Googlebot!'
Google, Wherefore Art Thou Google? Sites Abandoned by Googlebot! © August
30, 2004 As a search engine optimization specialist I often optimizeexisting web
pages for small business clients, upload them tothe site and see pages
re-indexed by Google within a week.This only happens with existing business
sites that have beenonline for a few years. Google seems to be updating
theirindex as often as every other week at this point and olderestablished sites
that are already indexed seem to be re-crawled on that twice a month schedule on
a fairly routinebasis.Two clients that hired me for recent work saw their
rankingsshoot to the top for a newly targeted search phrase in aweekend when I
did optimization on a Thursday and they wereranked instantly by Saturday. Now
keep in mind that thisdoesn't happen for everyone, only those that have been
onlinefor some period and already have significant content thatsimply needs
tweaking and proper title and metatag informationadded. They usually have
relatively good existing PageRank anddo well for other RELEVANT search phrases
already. I offer thatwarning only to avoid instilling false hopes in anyone
hopingto achieve the same instant ranking boost overnight.Those clients that do
succeed in this way are often thrilledwith the results accomplished in such
short order. I'd loveto be able to offer that type of ranking boosts to
everyone,but some are more equal than others when it comes to easy,inexpensive
SEO tune-ups that rev up your rankings overnight.Your mileage may vary.WHY DO
NEW SITES SUFFER?What is going on with newer sites that don't get crawled
formonths? I've got a client, a newer attorney directory thatoffers tons of
great information in the form of articles onspecific areas of law, links to
incredibly valuable andrelevant legal sites and over 600,000 attorneys listed
bypractice area and state. Yet the site has not been re-crawledby Google for
over 3 months! Now this would not be such a bigissue for many sites, but this
site is relatively new and we'veoptimized all the titles, tags & page text,
created a completesite map and placed links to all these resources on the
frontpage.I know that the site is not being crawled because Google'scached copy
of the front page shows it before we did thework four months ago, without the
new links and withouttitle tags. We've submitted the site by hand,
(manually)once a month for three months via the Google Add URL
page.http://www.google.com/addurl.html When the hand submissionfailed to get it
re-indexed for four months, we submittedthe sitemap page, which has not been
crawled at all. Googleshows only ONE page on this site, when in fact it
hasthousands of pages, a sitemap and dozens static pages!Part of the problem is
that this site must be dynamic, sincea database of over 632,000 attorneys must
be accessed,retrieved and served for any of those law firms searched forto be
returned to the site visitor. Google warns owners ofdynamic sites that Googlebot
may not crawl dynamicallygenerated pages with "?"" question marks in the URL.
This isto avoid crashing the server with too many concurrent pagerequests from
Google's spider.http://www.google.com/webmasters/2.html#A1 The solution to this
dynamic URL problem has been discussedwidely in search engine forums and
solutions have been bandiedabout including software provided by SEO's, URL
re-writetechniques for dynamic pages on APACHE
servershttp://www.alistapart.com/articles/urls/ and PHP
pageshttp://www.stargeek.com/php-seo.php to generate search enginefriendly
URL's. Others recommend simply adding static HTMLsitemap pages as alternatives
for the search engine spiders. In this instance the client's developer simply
said "Ican'tdo that (PHP solution) on this server". So we resorted toputting up
the static HTML sitemap pages with hard-codedURLS to the main 54 pages of the
site athttp://lawfirm411.com/Law-Firm-411-sitemap.html This shouldget at least
those fifty pages crawled by Googlebot, butGoogles' spider appears not to be
crawling this site at all.How do we know this? See for yourself by using the
followingquery in the search box at Google: allinurl:www.lawfirm411.comwhere the
result page shows ONE page in the results. If youtry that query on your own site
(replace your own domain namefor lawfirm411.com), you'll see the results lists
ALL yourpages.The site home page was crawled by Google four months ago, whenthey
took their "Cached Snapshot" of the page. You can seethis by visiting the Google
cached page
here:http://66.102.7.104/search?sourceid=navclient&ie=UTF-8&q=cache:www.lawfirm411.comwhere
the date of this snapshot is "Apr 20, 2004 07:42:19 GMT"and they haven't been
back since. The page in that snapshothas none of the newly added links, an
outdated title tag, andold content.This problem is not unique to this site. One
client we workedwith two years ago had a dynamically generated, framed
site!Those two site structures have always given search enginestrouble. Their
site was not crawled at all and only the frontpage showed up. Our solution was
to create a second domain(owned by the client), which had static HTML pages that
precisely mirrored the content of the client's framed,dynamically generated
site. Guess what happened afterGooglebot crawled the static site? Google indexed
the framedsite in full and then banned the static site from the index!Not an
approach we advocate, but the one that worked for thisclient.We're still
searching for ways to get Googlebot back toLawFirm411.com before creating that
new static site, butdecided to share this odd experience with the SEO
communitybefore going to any extremes. Google provides over 70% ofmost search
engine referred traffic to ALL of our clientsand we realized we can't site idly
by and see a major clientlanguish because Googlebot didn't like what it found at
theclient site on the first visit four months ago.This issue dogs newer sites in
other places as well. The OpenDirectory Project has also become notoriously slow
in addingnew sites to the directory and in this case, has not pickedup this site
even after 6 regular monthly submissions. Theweb playing field may have begun
tilting toward older,established sites and away from new ones. © August 30,
2004 As a search engine optimization specialist I often optimizeexisting web
pages for small business clients, upload them tothe site and see pages
re-indexed by Google within a week.This only happens with existing business
sites that have beenonline for a few years. Google seems to be updating
theirindex as often as every other week at this point and olderestablished sites
that are already indexed seem to be re-crawled on that twice a month schedule on
a fairly routinebasis.Two clients that hired me for recent work saw their
rankingsshoot to the top for a newly targeted search phrase in aweekend when I
did optimization on a Thursday and they wereranked instantly by Saturday. Now
keep in mind that thisdoesn't happen for everyone, only those that have been
onlinefor some period and already have significant content thatsimply needs
tweaking and proper title and metatag informationadded. They usually have
relatively good existing PageRank anddo well for other RELEVANT search phrases
already. I offer thatwarning only to avoid instilling false hopes in anyone
hopingto achieve the same instant ranking boost overnight.Those clients that do
succeed in this way are often thrilledwith the results accomplished in such
short order. I'd loveto be able to offer that type of ranking boosts to
everyone,but some are more equal than others when it comes to easy,inexpensive
SEO tune-ups that rev up your rankings overnight.Your mileage may vary.WHY DO
NEW SITES SUFFER?What is going on with newer sites that don't get crawled
formonths? I've got a client, a newer attorney directory thatoffers tons of
great information in the form of articles onspecific areas of law, links to
incredibly valuable andrelevant legal sites and over 600,000 attorneys listed
bypractice area and state. Yet the site has not been re-crawledby Google for
over 3 months! Now this would not be such a bigissue for many sites, but this
site is relatively new and we'veoptimized all the titles, tags & page text,
created a completesite map and placed links to all these resources on the
frontpage.I know that the site is not being crawled because Google'scached copy
of the front page shows it before we did thework four months ago, without the
new links and withouttitle tags. We've submitted the site by hand,
(manually)once a month for three months via the Google Add URL
page.http://www.google.com/addurl.html When the hand submissionfailed to get it
re-indexed for four months, we submittedthe sitemap page, which has not been
crawled at all. Googleshows only ONE page on this site, when in fact it
hasthousands of pages, a sitemap and dozens static pages!Part of the problem is
that this site must be dynamic, sincea database of over 632,000 attorneys must
be accessed,retrieved and served for any of those law firms searched forto be
returned to the site visitor. Google warns owners ofdynamic sites that Googlebot
may not crawl dynamicallygenerated pages with "?"" question marks in the URL.
This isto avoid crashing the server with too many concurrent pagerequests from
Google's spider.http://www.google.com/webmasters/2.html#A1 The solution to this
dynamic URL problem has been discussedwidely in search engine forums and
solutions have been bandiedabout including software provided by SEO's, URL
re-writetechniques for dynamic pages on APACHE
servershttp://www.alistapart.com/articles/urls/ and PHP
pageshttp://www.stargeek.com/php-seo.php to generate search enginefriendly
URL's. Others recommend simply adding static HTMLsitemap pages as alternatives
for the search engine spiders. In this instance the client's developer simply
said "Ican'tdo that (PHP solution) on this server". So we resorted toputting up
the static HTML sitemap pages with hard-codedURLS to the main 54 pages of the
site athttp://lawfirm411.com/Law-Firm-411-sitemap.html This shouldget at least
those fifty pages crawled by Googlebot, butGoogles' spider appears not to be
crawling this site at all.How do we know this? See for yourself by using the
followingquery in the search box at Google: allinurl:www.lawfirm411.comwhere the
result page shows ONE page in the results. If youtry that query on your own site
(replace your own domain namefor lawfirm411.com), you'll see the results lists
ALL yourpages.The site home page was crawled by Google four months ago, whenthey
took their "Cached Snapshot" of the page. You can seethis by visiting the Google
cached page
here:http://66.102.7.104/search?sourceid=navclient&ie=UTF-8&q=cache:www.lawfirm411.comwhere
the date of this snapshot is "Apr 20, 2004 07:42:19 GMT"and they haven't been
back since. The page in that snapshothas none of the newly added links, an
outdated title tag, andold content.This problem is not unique to this site. One
client we workedwith two years ago had a dynamically generated, framed
site!Those two site structures have always given search enginestrouble. Their
site was not crawled at all and only the frontpage showed up. Our solution was
to create a second domain(owned by the client), which had static HTML pages that
precisely mirrored the content of the client's framed,dynamically generated
site. Guess what happened afterGooglebot crawled the static site? Google indexed
the framedsite in full and then banned the static site from the index!Not an
approach we advocate, but the one that worked for thisclient.We're still
searching for ways to get Googlebot back toLawFirm411.com before creating that
new static site, butdecided to share this odd experience with the SEO
communitybefore going to any extremes. Google provides over 70% ofmost search
engine referred traffic to ALL of our clientsand we realized we can't site idly
by and see a major clientlanguish because Googlebot didn't like what it found at
theclient site on the first visit four months ago.This issue dogs newer sites in
other places as well. The OpenDirectory Project has also become notoriously slow
in addingnew sites to the directory and in this case, has not pickedup this site
even after 6 regular monthly submissions. Theweb playing field may have begun
tilting toward older,established sites and away from new ones. ABOUT
THE AUTHOR Mike Banks Valentine is the SEO for
http://www.lawfirm411.comContact him at
http://www.seoptimism.com/SEO_Contact.htm Improve Your Small Business Online at
our Ecommerce Tutorialhttp://website101.com/Free-Tutorials/index.html
You may view the latest post at
http://www.richproject.co.cc/?p=4059
You received this e-mail because you asked to be notified when new updates are
posted.
Best regards,
admin
k_malee@hotmail.com
ไม่มีความคิดเห็น:
แสดงความคิดเห็น