Be able to make predictions regarding the relevance, credibility and influence of various documents or data sets
and to qualitatively analyze changes over time to qualitative metrics such as of absolute and normalized trend mass, velocity, acceleration, and co-variance with other trends.
Techniques -
Natural Language Processing will help to differentiate meaningful and relevant uses of symbols from unrelated due to alternate meaning, or metafor, or it's context.
The use of conceptual spaces that lie above the symbols themselves helps to form a conceptual topology that makes more sense logically.
We not only seek to increase relevance, but also to gain information into tone, tense, phrasing, sentiment, comprehensiveness, as well as some authority and assuredness cues.
Text mapping and link graph mapping will be the predominant fundamental aspects used for statistical modeling.
Other techniques include machine learning, which would have the algorithms learn things from the data such as how the data is structured heirarchicaly, and what new data search filters or priorities it should accomodate.
This means that the program will be able to "understand" what it learns by organizing it in a meaningful way, as well as will "react" and "want" to learn more about new things it discovers ,while staying relevant.
While most of the dynamic inputs are new occurences of blogs or comments from engines or aggregators, there are a variety of input sources that could form the basis of what is being modelled, or what aids to the innate knowledge for learning capability.
Document Inputs could include data from feed aggregators, search crawlers, APIs, toolbars, and other applications or line by line entry.
The aim of the tool is to learn new domains of knowledge, but we will need to teach it how to learn.
The meaning of each individual word, or phrase is determined practially by its usage.
It would be quite hard to have a machine teach itself how to learn.
To humans initially, and for computers, the actual symbols are meaningless. Most languages form arbitrary words. Sometimes variations of words can lead to clues to tense, possesion and the meaning of other words.
The whole basis of knowledge behind the symbols, is the relationships among the symbols.
Things such as how often a phrase is likely to be followed by or preceded by another phrase, given certain criterion.
The basic relationship types are as follows:
External
Internal
Additive
Alternative
Background
Causal
Contraction
Constrast
Dismissive
Elaboration
Enablement
Evaluation
Interpretation
Justification
Motivation
Parenthesis
Restatement
Similarity
Solutionhood
Temporal
And, the types of tense - absolute, future, nonfuture, nonpoast, nonyet, past, present, still.
Adverbial clause - He was quiet in order to avoid you. This sentence has meaning triggered by the words "in order to" that triggers a relationship between what and why.
It turns out, the variety of adverbs form the core of the relationships among words, phrases, clauses and sentences, thankfully.
The word was Adverbially modified. (Adverb modifies verb.)
This sentence has Adverbially Inefficient Usage. (Adverb Modifies Adjective)
That word was Synthetically Adverbialy Created. (Adverb Modifies Adverb.)
While we can definately learn a lot from the nouns and adjectives and their interrelationship with conjunctions and prepositions, dividing the adverbs up into various subclasses will be necessary to interpret relationships within large data sets.
The large sets that will form the data to be monitored as well as often serving a role as the collective backbrain of the AI will be -
The Wikipedia currently quite massive and comprehensive, and is also a fast growing medium that responds well and reflects current trends.
It can be used to form the basis of ontologies, and lexicons used by the backbrain.
Portions of the wikipedia will also be a source of documents for targeted mining projects.
Dmoz - Dmoz not only tells us one natural way of structuring the areas of knowledge or commerce...but it also gives us some clues to the authority and age of its documents, as well as enabling other documents to be related to dmoz, if they are in the same category.
This conceptual logical link can be distinct from an explicit hypertext link.
The conceputal logical links can arise from being in the same category at a respected source, or having similar patterns of keyword uses, or more generally over the unstructured web, having similar citation patterns, and inlink and anchor patterns.
Webster - Dictionaries can load the lexicon, and map it to parts of speech. A conceptual space of terms can be used to support relationships between nouns, (dog is an animal) (something can't be happy and sad at the same time.) as well as to learn different ways to categorize adjectives and adverbs based on related spaces.
Myspace, and the various other Social Networks allow the data set of its participants to be mapped to a special graph of the social networks of the user base.
Very interesting groups can be analyzed based on relationship density patterns, and strongly connected clusters. This can point to influential parties.
Keyword analytics, and specific mining for consumer brands, celebrities, political causes, lifestyle declarations can all be done specifically and separately.
The search engines give us data both in the results they serve for keywords.
Search engines also give us the more important measures in trendspotting, what are people searching for?
Data sets that not only give lists, but the orders and distances of the searches will add to the information available.
Our AI engine might needs to know who ranks number one for target terms which clearly can be linked to influence and profitability or who had the most top 10 hits over all relevant phrases for a space)
Ad search engines -
Keywords bought, and the price.
Keyword sets Co-bought
Ad Copy
Google Maps / MSN Maps
Other relevant databases to integrate into the backbrain are the :
Whois Database: (and older versions)
DNS DB
Archive.net
And any other massive amounts of webtraffic Data.
In a similar way, the following data sources can be mined for their keyword or structural content.
Chat rooms - loggin converations.
Blog-o-Sphere
Forums
Classifieds
Del.icio.us and other Social Bookmarking
Video and Podcast Downloads
Peer to Peer Search Phrases
P2P Files Available
P2p Downloads
Away Messages
Newsletters
Main Stream Media
Academic DBs, Citerank ¶ 7:50 PM 0 comments
Leading sites:
Myspace
Friendster
Ryze
Meetup
Myyearbook.com
Hi5
xanga.com
Friendwise – new myspace project
Yahoo 360
livejournal.com
zerodegrees.com/
Other General Social Networking Sites:
Friendzy
Tribe.net
FriendSurfer.com
PeepsNation.com
Emode.com
Tickle.com
squidoo.com/
Multiply.com
reunion.com/
connectu.com/
Dodgeball
Upcoming.org – owned by Yahoo
me.com/
friendtastic.com/
gazzag.com/
xuqa.com
mywallop.com – MSN project, unreleased
sconex.com/
www.sms.ac – SMS community
studybreakers.com/
student.com/
profileheaven.com
network.monster.com/
43things.com/
campusbeaver.com
downelink.com/
faceparty.com/
nexopia.com/
frappr.com
openbc.com/
passado.com
couchsurfing.com/
meetro.com
zaadz.com/
catch27.com
Private networks
Orkut – (run by Google)
Asmallword.net – High Society Exclusive Network
doostang.com
Biz networking:
itsnotwhatyouknow.com
joinnetworkplus.com
konnects.com
linksv.com – Silicon Valley
matcheroo.com/
http://www.zerodegrees.com/
ecademy.com/
jobs.realcontacts.com – Used for Job Search Among existing network.
businesspartnerships.ca/
refernet.net/
Niche Business:
BusinessPartnerships.ca
econozco.com/ - Spanish Language
neurona.com/ - Spanish
Less established:
networkingforprofessionals.com/
senior-management-network.co.uk/
http://www.schoolster.com/
Sales Leads:
jigsaw.com
spoke.com
Local Social Networking
metrochai.com – New York City
bebo.com – most popular (#1) in UK
buzz-oven.com – 2 Texas Cities
Wealink.com – China
iwiw.net – Hungary
connect.ee - Estonia
hyves.net - Netherlands
mixi.jp - Japan
Online Media (podcasts, video and news):
youtube.com
ourmedia.org
libsyn.com Liberated Syndication
audioblog.com
49media.com/
towerpod.com/
dl.tv
blip.tv
mediabistro.com
www.buzznet.com
revver.com
www.mediabuddies.com
http://www.slide.com/main
channelone.com
channel101.com
grouper.com
gather.com
podcastcentral.com
webtvlist.com
wwitv.com - World Wide Internet TV
podcastalley.com
tv4all.com
comfm.com/live/tv/
gizmodo.com
geeknewscentral.com
dl.tv (digital life)
nyc.node101.org
Blog and Self-Publishing Sites
Blogger.com
Livejournal.com
Business Partnership Sites:
Businesspartners.com
companypartners.com/
Social Bookmarking:
Del.icio.us
Del.irio.us
citeulike
Simpy
tagged
stumbleupon.com/
(IBM Dogear Project)
Social Search:
Rollyo.com
jookster.com/ - also social network
fleck.com/
rojo.com
Niche Social Networking:
wayn.com - Where are you Now? Travel
purevolume.com - music
omidyar.net - Philanthropy
Dogster.com, which targets dog owners
christianster.com
favors.org/FF- 50,000 participants
bookcrossing.com
fastcompany.com/cof- Magazine inspired
jewster.com
politicalfriendster.com
freecycle.org - Giving and Receiving unneeded materials.
theyrule.net – reveals network of top businesses and organizations
pooln.com - Carpools
rotteneggs.com - pranksters
Classified Listings
Craigslist.org
Google Base
Forums, MessageBoards:
Slashdot.org, Kuro5hin.org
delphiforums.com
Contract Work / Freelance
Scriptlance –
Rentacoder
Elance
Guru
ContractedWork
CGIlance.com
Offshorexperts.com
Dating Sites: (Free)
okcupid.com/
desi-planet.com – Indian Youth site
jdate.com
ashleymadison.com – discreet affairs
3D worlds / games
Habbohotel.com
There.com
imvu.com
secondlife.com/
teen.secondlife.com/
More listings:
netrelate.com
isociate.com
meebo.com/
plazes.com
IPTV News and Listings –
http://www.livetvchannels.com/default.asp
http://www.dailyiptv.com/?gclid=CNGY07-DloQCFSRJPgodtjFosw
http://www.neeptv.com
New unsorted social networks:
Kontiki.comjainworld.com
africancraft.com
ancestry.com
Microsoft Groove Networks
epinions.com
bolt.com
churchlife.tv, godipod company sells scripture-preloaded ipods.
dotcomedy.com
nbcfirstlook.com
"Online Media Exchange"
laptop.media.mit.edu - (check out the crank powered laptop for the
iknowtion
uthtv
omnik
absolutepunk.net
collegehumor
purevolume
shopbop
karmaloop
pandora
buzzoven, friendwise, heavy
dave.tv
vpod.tv
R.O.B.S.O.N.
Objects - (nouns)
Web Sites
Users - Persons
Places
Servers
Countries
Concepts - (nouns) Ratings
Effect
Weight
Direction
Distance
Duration
Words - Symbols (nouns)
Phrases
lexeme - eat, eats, eating, eaten, ate
Methods of Communication
Properties -
Passive action - Verb - a passive action that describes a relationship.
User A Owns site B.
Property - Adjectives - Properties of objects of Concepts
(core property, peripheral property)
Adverbial Properties -
Describes Actions -
Describes Adjectives / Properties
Describes Adverbs (very vs. slightly - diff adverb mod types)
Describes Clauses
Operator - (conjunction)
Equals, Is Greater than, Is Less than, Is near to, Contains, Is superset of, Has intersection with, Has no intersection, Combined with, Subtracted from, (union, intersection) Implies, <-- , If and only if.
and, or, but, nor, for, so, yet, while, when, where, whether
after, although, as, because, IF, once, since, then, though, till, until,while,do..while
eiter/or, neither/nor, so/as, whether/or
Prepositions.
Inside, wherein, on, about, around .
Properties of an object Related to a Place.
(postpositions - ago, thereafter)
Conjunctive Adverbs - Accordingly, Afterwards, Also, Consequently, Besides
However, Indeed, Likewise, Moreover, Finally,
Never, Nonetheless, otherwise, similarly, so, still
therefor, thus, paradoxically
Determiners - My, your, his, "the old lady's" Accusative Case, Nominative case - she.
Pronoun - Pronoun.
Interjection. Articles.
Simily, Metaphor, Kennings,
Elipsis - I ate the cake and judy the ice cream.
Semantic Feature - Boy {young}{male}{human}
Tense - Relation with Time.
Theta Role - He put the spoon in the cabinet.
participant, circumstancial
Aspect - did, was doing, do, was doing, had done, has/have been doing.
Active / Passive Mood. Chased, Was Chased By.
Active Functions - Functions within the system that operate on the data.
These are the methods within the classes.
The Case:
Most data retrieval applications are interested in the data only as a means to the output, or desired task the person is performing at that moment.
Although this has its useage in automating labor-intensive tasks, the task based, or functional approach to data retrieval and storage leaves a lot to be desired.
Without an in-house index or repository, all the data that is pulled in from the retrieval process is a disconnected list, who knows what it's related to, who made it, who owns it, who it was made for.
Also, there is seldom a general and easy way for automated processes or humans to extend the data, (and data types) of these web objects in a custom way.
The data structure presented in this platform borrows most of its concepts from object oriented programming, OOP.
This includes:
Classes (or data types) - This is where data is structurally related to other properties and data fields. Object oriented programming also has the functions that operate on that data as native methods of that class. Our discussions for classes and data-types are primarliy dealing with data members of a class, not its method members.
These include inheritance - allowing a class to take on the attributes of another class.
Extension - extending a class to include all data types, and then some.
Subclass - This is where a class is further segmented into sub-classes all of which have the "core" data members, but may lack all data members of the main class. A sub-class may have its own extended data members.
How companies can use this system.
Companies will use the system to store and inter-relate the web research they pull from their variety of sources, including:
Search Results from Scraping
Search Results from API
Feeds
Custom Crawling Results
Direct Human entering of Data.
Data from toolbar and browser history
Clearly, in the data environment, a user of data retrieval techniques will always be interested to search more sources using increasingly flexible and refined ways.
Additionally, depending on the type of data considered, and the context of the research and analysis, there will always be custom tasks and actions that need to be handled.
This includes editing and extending the data, advanced analytics of the data, reporting, integration of the data relationships with other data sets such as keyword databases, purchase data, etc.
There will always be room to increase the sources of input, as well as sources of output.
What this framework aims to do is to be able to sort, extend, describe, tag, weight, and categorize, assocate and relate the inputted data in meaningful ways.
What's a meaningful way?
This is why we take a general approach where web objects can be of any class, and have any attributes, properties and relationships to other web objects, or abstract concepts.
Once we have a general way to describe, extend and inter-relate data, all we need to do is draft Document Type Definitions that describe our web Objects, And Data Type Definitions for the properties those data members have.
Different uses of the program will:
Pull from different data sources.
Use different Document types, classes, and relationships types to extend and inter-relate the data.
Apply custom analytical functions to filter, sort, and apply custom metrics and automated properties with various sets of Data.
Finally, the reporting or output phase, which will often include a combination of standard reporting on qualities of the web relationships, as well as custom reporting for advanced modules that offer additional analytic processes.
Overall, this system allows the maximum amount of information to be available for all of the advanced functions that are needed for the output of the data retrieval process.
Flexibility of Licensing.
An organization can adopt this data framework to store and (more importantly) make universally available its data.
If the organization invests significant amounts in data retrieval from search engines, blogs, feeds, APIs and cusotm crawlers, that can be seen as an independent module, that simply interfaces with the open platform, of the RELWOM system.
Similarly, if the organization invests heavily in latent analytic processes, complex linear transformations and projections and iterative processes that converge to data points for detecting implicit inter-relationship, rather than the explicit inter-relationships built into RELWOM.
These advanced modules can exist independently from the RELWOM platform, yet, be completely interoperable.
In fact, the expanded sets generated by Proprietery filters, sorters and analytics algorithms can become properties, pointers and even relationship types inside of RELWOM.
Needless to say, independent modules for reporting, or publishing the results would be able to make full use of the RELWOM logical layer, while being independent of it.
This allows an open environment for sharing data sets of objects, of classes, and of relationship types (a special kind of class).
While, at the same time, allows a proprietary environment for advanced analytics, filters, sorting tools to uncover non-explicit relationships between data, and then use the open system to make these explicit.
This provides the ability of organizations to share and collaborate on data sets, document types, property types, relationship types, and other logical structures in RELWOM.
It also allows organizations to offer their indepent modules for input, processing or output as proprietary add-ons to the RELWOM framework.
Applications to mine Blogs.
How, when and where do topics break out to become highly popular?
What predicts or corrolates with topics breaking above their levels and getting increased and possibly mainstream coverage.
How early are the signs of such developments?
Affiliate Relationships.
What Affiliates are early adopters of new merchant programs / products?
What Affiliates Are influential?
Applications for Social Networks:
Who are the most influential people related to a particular interest?
How dedicated are the influencers? Do they have a blog, and group memberships?
How do social networks break down into logical clusters of influence, behavior, and demographics?
How can merchants make use of this information to better speak to their market, and their needs?
How does the duration of friendships, effect the predicted co-friendship patterns? How do the status of frequent updates to profile or extremely active comment areas corrolate to other activities within social networks?
Linguistic Applications:
How do people that use small words, poor grammar, and butchering of words like playaz, wazzap, relate differently to their social networks?
Are these patterns of language likely to be closely clustered in networks, or be fairly distributed among younger users?
How do people that use big words, latin words, misspellings, metaphor, uncommon grammar tense, case and form relate to their immediate and extended networks, and the network at large?
Political Applications:
What issues motivate certain groups to the extent that it would either be the deciding factor in actively deciding to vote, or a swaying factor to one who is usually resolved to vote?
How can these coalitions and politically / economically aligned groups be found and monitored?
What values, backgrounds, temperaments, intents are what drive these movements?
By understanding the common goals, as well as the areas of conflict within actionable groups, public office holders can more effectively make policy to please their voters.
Raising Funds - By surveying a person's participation in user-generated media, one can predict the causes they are most likely to contribute to, or the associations they are most likely to receive benefit from.
Their voting patterns can be predicted based on ther core set of behaviours.
The voter himself will also feel they belong to a smaller group, that actually represents them, even if it is an independent faction running largely through one or both parties in the 2-party system.
Voters will be very excited to give their loyalty to their union reps, or their syndicate representatives, based on the new coalitions they can form with fellow voters, and even financial contributors from abroad.
This is the concentration of power that will give their chosen leaders actual power to promote the interests of its constituents, they will trade their votes for favors as their base has influence.
This is markedly different than having to compromise in the ballot box between two moderates, and having no way to "block vote" or coordinate for more power with actionable aligned groups.
Local government is likely to be the easiest to influence to a large degree. ¶ 8:15 AM 0 comments
Optimal linking among target pages, booster pages, and external pages.
Optimized Link structure
In this review, we assume the target page for increased web authority score is the page we wish to position with the highest LinkRank.
Additional company owned target pages could be referenced from external pages of type (1.) and be assimilated into the link hub network as a link of type (3).
1. External Links not requiring a backlink. (These are sometimes referred to by web spam detractors as "hijacked links".
This includes genuine link references, paid links, one component link of a triangular link trade, press release features, news features, and certain kinds of directory listings.
These are the set of "backlinks" where the optimizer has limited control over that go into establishing a target's initial popularity.
2. Internal pages: (Target Page and Booster Pages) – In this example, the target page is http://target.com, and all subpages additional pages on other sites will be treated as booster pages.
The target pages along with the booster pages collectively encompass the pages that the optimizer has absolute control over.
3. Potential link hub partners that do require a backlink, or some kind of linking compensation to their target pages: These pages are typically willing to offer links from their boosting pages and / or their target pages for favorably mutual linking.
1.
Links coming from external sites not requiring a backlink should all point to the target page.
Having any of these links point to the booster page to the exclusion of a direct link to the target will be a sub-optimal link structure. If there is a very small number of links on that page, even securing multiple links such as to the booster page + the target page will typically dilute the reference as well.
Exception. If there is a large amount of outgoing links on the hijacked page, (making a singular reference to the target relatively diminished) and it is possible to secure multiple links on the same hijacked page, it is advisable to reference not only the target page, but also additional booster pages. This could include a number of booster pages until the percentage of company properties is high enough to receive a good portion of the page's pagerank. The requirements and desired amount of proportion will depend on the current page's pagerank as well as its number of outgoing links. Example, for a low ranked site, it is advisable to use a larger percentage of on-page links to target and boosting pages.
In most cases, only one link is offered by news services, directories, main page links offered by webmasters, etc. So, we can assume that linking to the single target is the typical strategy.
2.
The internal pages of one or more domains will be divided up into Booster Pages, and (preferably) one target page.
The following guidelines determine the optimal interlinking structure among these links.
All booster pages link to the target page, and ONLY the target page.
There are no links among the boosting pages. (or external links)
The target points to some or all of the boosting pages.
Many SEOs are accustomed for the preference of one-way links over reciprocal links. Since this is a unique case where the linked to page is set up to be a booster page for one and only one page, and would cause no PR leak to external or other internal pages, the reciprocal link represents an optimized link structure.
For this reason, it is advisable for the target to reference as many booster pages as possible. This also ensures the booster pages will be indexed if using large document sets.
The reason why one way linking is usually preferred to reciprocal linking is usually because they are referring to sites that have OTHER links on them, in which case the reciprocal link in turn passes pagerank to those pages.
In our special case of linking to booster pages, 100% of their outgoing links (just one link) come back to the target link. For this reason, the reciprocal link re-enforces the original target's pagerank.
How do we allow using only one-link-only booster pages, when many other links to navigation, external references and page items are needed?
For instance, how can a navigational Sub-Page be a Booster Page (link ONLY to Target.com and nothing else), and at the same time serve as a valuable navigation sub-page citing other sub-pages?
The use of the rel=nofollow tag for both image links in a navigational scheme, as well as textual links in the body of the document will allow us to create links that spiders do not follow and engines don't pass PageRank scores.
This is absolutely recommended for the "leaves", of the site-map, such as the Sub-pages of the that aren't navigational, and only have content.
The Navigational and Leaf pages can be said to be truely optimized boosting pages only if the target page (target.com) can link to ALL subpages, in both the leaves and navigational pages.
This is because if the navigational page turns into a booster page, the engines won't be able to find the sub-pages unless they are directly linked to from the target page (or some other non-booster non-primary target), Note, the spider will always come to the target first, since that's the only external incoming link permitted by our system.
If the taret.com homepage can reference most of the pages on the site itself, then the main sub-pages can remain optimized booster pages.
If this presents display concerns for the main page, some navigational pages can be maintained for the purposes of spidering and indexing.
Although this removes pages from truely optimized link hubs, there are many other boosting pages that will not be effected.
Note: having the target still point back to as many boosters as possible is still needed to support the maximal optimization.
3.
If one or more pages are designated as targets either because they have incoming backlinks (from hijacked, paid, or genuine link references), and / or they are seeking to mutually share their rank with current target pages, the following approach can be applied.
Link the existing target page to all additional target pages in a complete graph. This means every target page links to every target page.
No changes are needed from external backlinks, nor to boosting pages of any of the target pages.
In particular, it is not needed to add links to all targets to all booster pages.
The PageRank re-distribution effect is accomplished in a symmetrical way simply by interlinking the targets.
¶ 8:34 PM 0 comments
Axioma - This one requires a download, and actually pulls from a major Engine's index. It supports grabbing entines from Y!, G, and MSN.
It has one GREAT feature that I actually haven't been able to find on any other search engine.
This feature is the option to search only home pages, such as site.com.
This can be very helpful when you're looking for domains specifically, frequently for businesses.
Often, us people in the business aren't looking for information, or side-pages.
We're looking for domains only, and we don't want our results messed up with multiple results for the same old domains, much less side-pages from highly ranked domains that have nothing to do with our topic on their domain.
So, I'm not looking for about.com's 400,000th most popular page that happens to contain my keywords or parameters, I'm looking for businesses that are specifically about these search parameters, and about them to the point that their homepage is optimized for this term.
It has a lot of good filters and scales to be able to sort your results based on different characteristics.
Grokker - This one is very well done. There is a graphical view and a linear standard view.
Mooter - has a basic graphical layout of the top clusters.
Exalead - This let's you specify that a page links to a certain page WHILE also searching for other parameters, such as site: , or URL contains, or a keyword search. Very useful for backlink searching.
Sphere - Not so sure about this one, its really new.
ROllyo - Too lazy to search the the web results yourself? Let top celebrities and jounalists search the web for you!
Social search allows you to associate yourself with communities that predict what sites are relevant to you.
So, if you work at jaguar, your association with the car salesmen group let's the search engine know you mean the CAR.
If you work at the zoo, and have that group selected, the SE knows you mean the animal.
The personalization of search has arrived.
Live.com - A brand new interface by microsoft for their search engine. This also serves a purpose similar to MyYahoo where the user can add various news stories and features to their home page.
One interesting option is the creation of Macros that let you search only sites that are listed, and pre-definate search filters for re-use.
The cool thing is that you can use the Macros of other people.
This is especially useful to prevent similar groups from having to find their results among all the unrelated results.
Del.icio.us - This social search is great. When can I integrate my clickstream with this?
Trexy - this logs your clickstream for the purposes of recovering site's you've already been to, but can't remember where or how you found them.
It also has a feature to let you know if your site has been visited before by other Trailblazers as they are called.
COmplete with the obligatory toolbar, Trexy is worth a look.
Clusty - ah, one of my favorites. I can't say enough good things about this clustering engine.
I don't know where they get their index, but it's great.
I've read that the microsoft clustering search engine uses only the snippets, and the 3-word permutations contained therein to create it's clusters.
That puts a lot of emphasis on the snippet, possibly to the detriment of densely used uncommon phrases used in the document.
But, the results are pretty good for most topics I've tried.
Ask.com - Props to Ask.com for advertising their Search Engine's advanced features on Television.
They show how you can expand or refine your search to more inclusive phrases that are related to your topic, or refine your search using more specific phrases that are contextually a sub-set of your search phrase.
I'm not sure how it comes up with the suggestions. I imagine it comes from other searches conducted on their search engine, but I'm not sure how they build the relationships between broader searches and refined searches.
If anyone has any idea how this works, let me know.
Gigablast - Their bits perform a similar function to ask's advanced features.
What do you know that teoma and ask are now one and the same.
Teoma has a special link rank popularity that is specific to the topic at hand.
This prevents higly ranked fashionistas and stylists from being able to have highly ranking commentary on Foreign Policy just by making a page about it.
Overall, a very good advancement from teh way google does it.
It's true that google in a sense does this with their Anchor bucket, but this is only for very common and competitive searches, and only applies when links are supported with literal anchors.
The way ASK does it seems much better.
Its actually quite similar to the HITS algorith, also known as the CLEVER algorithm, I believe developed by IBM.
Instead of working with global pagerank calculated from the ENTIRE web... the HITs algorithm uses a normal search engine to get a list of example sites, and then analyses popularity of those sites compared with one another.
It also gives each site a Hub-Rank, and an Authority Rank.
This is in line with the bow-tie theory of the internet.
This is that most sites are linked to a huge strongly connected cluster, that is comprised of the most comprehensive portals, and hubs.
A hub is definated as a site that links to many good authorities.
An authority is a site that is referenced on many hubs.
Sounds like a nice circular definition, right?
Well, fortunately, induction, recursion, and interation and convergence of these scores let's us do just that.
Hubs are references, connectors, portals, directories, links pages.
Their hub-ness or hub score is determined by the quality or authoritativeness of those links it contains.
An authority page is a page that has high quality content related to a narrow subject.
It isn't normally characterized as having a large amount of outgoing links on it, although this could be the case.
It is characterized by having a high amount of inlinks from quality hubs.
Since naturally some sites are made with the intent to point people to other destinations...and other sites are set up to BE destinations, I think this algorithm is very CLEVER.
Its also good that it doesn't need to index the entire web to find out how popular I am for a particular subject.
Eurekster - ¶ 8:04 AM 0 comments
Like the highly useful clusty.com clustering engine, Microsoft has developed a tool for pre-sorting search results based on link structure and keyword heuristics involving densities, co-citation, semantic relationships between words.
http://rwsm.directtaps.net/
This cluster engine technology was developed largely in China by microsoft labs in that region.
I am a big fan of clustering, and feel it is an evolution of search which currently dominates data retrieval.
kartoo also has an interesting visual graph of your search results that can allow you to see the related keywords and websites that are in the "space" of search you provided.
The goals of this is partition the search results into discrete conceptual spaces that make sense for the user.
Ask.com has even made this concept the main message in some of its television commercials. That's the formet AskJeeves Search Engine, Ask.com.
It markets the idea of its "Expand your Search", and "Refine your Search" features that come up when your search result is related to other common search results.
This tool is very useful in suggesting other keywords that are synonyms or related in some way.
Gigablast.com has a very good version of this technology refered to at the top of their page under Related Searches called Giga Bits.
Snap Search Engine actually suggests keyword variants as your type your SEARCH.
Clusty, the free cluster application of vivisimo, is just the beginning of the kind of technology these companies are working on.
Vivisimo has some private software that they user to crawl or trawl the web and then apply more sophisticated clustering, filtering and analysis functions with their data.
Clients who are looking to manage their reputation online are very keen on using this kind of technology to monitor their brand and the response to their marketing initiatives, and those of their competitors.
Vivisimo, Umbria, Cymfony and Intelliseek are the new leaders of reputation management online.
These companies have developed software that crawls the internet and analyzes the various results according to their client's brand kewyords or product names.
Currently they are being used to show market sentiment, the appearance of new voices, the overall number of postings, as well as the corrolation between publicity in the blog world and in the Mainstream Media.
Neilson (also owns alexa.com) just bought intelliseek / buzzmetrics.
This is a very healthy area, just getting write-ups in The Economist and WIRED.
Vivisimo reportedly charges $20,000 just to manage one brand keyword EVERY MONTH.
Although the public blog feed aggregators and blog monitoring tools are doing well, one of the totally un-exploited areas of this data monitoring are sites like myspace.com, friendster, youtube, and other popular destinations for the young and influential consumers.
Word of Mouth Marketing, along with its popular trade association, WOMMA, is getting a lot of "buzz" these days due to some of the success of the model.
Whether its called viral marketing, or word-of-mouth, or new interactive media, these techniques are becoming more standard part of the web marketing landscape.
Myspace.com is not only a great place for marketers to find their target audience and steer the discussion in their direction, but also is a fabulous demonstration of an effective viral campaign.
The myspace viral campaign was so simple, some may have missed it.
When you are bored at home or work and reading your buddy's away messages, you saw "View my profile at Myspace.com, and get your free profile today." Or something like that.
This simple add-on into a person's away message creates a highly targeted teaser and incentive to click.
Every person will click due to the personal relationship with their buddy, but will also have the option to create their own page.
Since a lot of middle and high school kids are online all the time after school, and college kids are known to literally be online all the time, even if they are away at class, their away message serves as the perfect viral message.
Viral marketing has gotten a lot of buzz lately and is confused with fads like the dancing baby on Ally Mcbeal, or some other funny but unuseful story surrounding a product.
Most often, these jokes are unrelated to the product, and there is no real utility in the message. (other than to pass it on.)
Passing on a fad is quite different than passing on a truly useful service.
I think would-be viral marketers need to determine what their value is, and how they can communicate that to their audience.
Activity on blogs, groups, other user's profiles, as well as updating of their own profile is all activity about to be mined by this industry.
They will then corrolate people to their immediate friends, their extended network, the network at large, and will dissect the keywords, music groups, celebrities, brands, products, television shows, movies, books that they mention on their profile.
This will help advertisers know their market's inclinations to better speak to them, or to find the best concentration of these consumers.
Also, another important area of research will be to determine what prosumers are the most influential.
If their comments or postings are complimented, tracked back, blogged about, "given props", or start a long discussion, this is more influential than someone without these characteristics.
Other attributes one could ascribe with this data is the originality of a profile.
This could be in relation to one's friends, one's "top 8", an extended friend network, their high school or college, or their region, or the network at large.
Another important tool is the regional analysis of trends.
Marketers will want to know how different regions speak different languages and use different patterns of grammar.
Slang words, brand names, ideas, web sites, and culture can be shown to diffuse from one area to another.
Metrics on cross-over and spill-over to the mainstream can be analyzed to show when a fad becomes a lasting trend.
This also has applications not only for the consumer space, but for the business and investment areas as well.
If a company has never gotten so much of a mention, and is now being written up on the top blogs and popping up on people's myspace profiles, maybe it's time to take notice.
Many consumers are worried that their activity is being mined and sold to anyone willing to foot the bill.
These people would be unhappy to know that web hosting companies and internet service providers regularly sell their data to anyone willing ot pay for it.
This shows traffic patterns for competitor sites, as well as click-stream.
Other search engine companies are clearly keeping a database of all searches, and moving into the area of integrating the google toolbar with search results.
The idea is if most people click "back" a few seconds after visiting a page, google needs to take that into consideration for it's relevance for that keyword.
We'll have to see how Google keeps this one free from fraud and manipulation. ¶ 7:55 AM 0 comments
Top SEO Tips.
1) Use hypens, not underscores in URLs.
2) having keyword in /foldername/ and title to increase chances of a double listing.
3) From an external link, one way links are preferred. From one of your links, or a page with many of your links on it, a recirprocal link is best.
4) Optimized linking structure has all of your booster pages linking to your target, and only your target.
5) Connecting 2 targets has the same effect has linking all boosters to both targets.
6) Canonical issues, make sure all versions 301 to one version of your site.
7) If possible, secure links to your pages along with authoritiy links. This makes the pages your links are on seem like a reputable hub, and makes it resemble other top hubs that reference many top authorities.
Co-citation will help your page be corrolated with the authority pages.
8) Place your keyword phrases in bold. For multi-word phrases This helps engines make sense of keyword permutations that are a 'phrase' compared to its surrounding text, and will also give priority to anything bold. Same applies to capitalization, and font size.
9) Google sitemaps can let you prioritize your pages as far as importance, and
classify them as to update frequency for spidering. This means your updated pages are more likely to be freshly indexed.
10) DHTML, Flash, Javascript are more trouble than they are worth on any page you are trying to position. Forget using them in a large way on your main page. If you must use them, pretend like they don't exist and build an HTML page around those components that has enough content and links in plain HTML.
11) Dynamic URLs are bad. Google has no way of knowing if its a session ID, affiliate ID, tracking code, or some actually unique page.
Having a plain-vanilla URL removes all doubt.
Use foldernames for pages you are trying to position.
Open ended URLs are even worse, where by a mod_rewrite, or open-ended string parameter allows you to type in any ending string to the URL to throw a 200, not a 404.
When this happens, you can get new URLs you create for them indexed by linking to them, or keying them into the google toolbar.
12) Anchors - the most important for highly competitive terms.
If google finds enough 'hits' of your keyword phrase in its anchor text database, (like 30 or 40k+), then it won't even look at its on-page database to find the pages to sort.
It will do the filters and link rank sorting on its top anchor choices and serve those results.
13) Related to 12. Massive content generation pages work well for longer, rarer, keywords, often including names of states or cities / locations.
This is because since almost no other pages have that exact keyword combination, or almost no hugley popular ones do, that off-page factors are far less important.
This means that the on-page factors become more decisive in a situation like this.
This is due to the fact that the link rank is based on sorting relevant result. When there are fewer results, much less to sort, so the sorting has less effect overall.
14) Using keywords that are semantically related to your keyword can re-enforce your topical or broad match for a keyword.
Find out google's synonyms or highly corrolated keywords by using a ~tilde
before your ~search.
15) A page with no links, is treated by google as a page that links to every page on the internet. Even pages with links have a small "dampening factor", that give a portion of their link rank to a 'random' page, or create an equal probability distribution among all pages.
This is an algorithmic solution to the problem that if pages with no links simply absorbed their PR entirely, and didn't pass any (because they have no links), that they would accumulate all the pagerank, and be considered the most popular, simply because they would horde link rank if their link rank wasn't somehow diffused as it is in the other pages with links.
The model for pages with links suggests that x% of the time the user will click on a link, and 100-x% of the time, he will go to a page on the web entirely at random. Here, 100-x is the dampening factor.
Linking to one and only one page (hopefully that links back)is the strongest focused link rank passing that a link can do.
This however does appear "unnatural" in large closely interconnected clusters.
Beware that clustering and collusion algorithms are out there looking for things like this, but they are designed to detect the huge spammers getting non-relevant results, not the small optimizers getting higher relevant results.
For this reason, using it on a small scale and in a slightly more natural and 'random' way is recommended.
16) Nofollow - Nofollow any links if you want to reference a bad neighborhood, or something with low perceived trust rank.
Trust rank is how google lets humans do it better by having people evaluate a list of sites as being not-spam, or "good", or "trusted".
The pages that these pages link to are deemed good, or trusted by association.
17) HTML Validation. Get Tidy plugin for firefox. Oh, I mean, Tidy extension for firefox. Use other validators to check dead links and other page-level errors.
The most common errors are using table alignments like middle, instead of center, or the other way around, that browsers understand but are not standard HTML.
Other warnings include problems involve unclosed tags, references to unknown variables.
Technically, Javascript is supposed to be preceded by extra \ back slashes whenever used with HTML.
Make sure you fix your errors and take a look at your warnings as well.
18) Try to get .edu / .gov links, and check the .edu inlinks of potential links.
19) When using sub-domains, be sure to link to images only on the sub-domain. It will reduce your chances of a multiple listing in the first page if you link to off-sub-domain images, on the root domain.
20) More on dynamic URLs. If you must use them, don't use things like ID, SessionID, etc. ¶ 7:36 AM 0 comments
