Blog Archives

Microsofts Struggling Search Engine Bing

“For Microsoft, any limits imposed on Google might help it improve the fortunes of its struggling search engine, Bing.”

While Nick and Eric’s article is more about Google than Bing, the statement does make us think about Bing as a search engine…

Poor Indexing, Poor Search Results

For the average web searcher Bing, along with the sister Microsoft search engines Yahoo and Search MSN, provides a poor experience to internet users.

Read the rest of this entry

Bing Banned

Bing and MSN Bots Are Banned

I have banned Bing, Yahoo and MSN search engine spiders from my sites! I’m tired of the constant rule breaking and over-crawling by Bing and MSN search bots.

Bing is a Rule Breaker

Microsoft claims Bing honours robots.txt rules. In my experience that is a blatant lie. Bingbot / msnbot simply ignore robots.txt rules and crawl whatever they want. Some of the specific rules broken include;

  • crawling system folders
  • crawling image folders (msn-media bot). Image folders and extensions jpg,  png,gif, bmp are disallowed
  • crawling RSS feeds. All RSS feeds are disallowed; rss.xml, /feed/, etc
  • crawling comment forms; DOMAIN/comment/184 – the path /comment/ is disallowed in robots.txt

The last straw was today. 2 days ago I added Bing and MSN user agent strings to disallowed bots in robots.txt across all my sites; this morning I see these bots read robot.txt then ignored it totally, and crawled the sites anyway.

Have You Seen Bad Activity by Bing?

Read the rest of this entry

Rulebreaker Bingbot and MSN

Bingbot and MSN bots are Rulebreakers

I’m tired of watching Bingbot and msnbot breaking rules and crawling disallowed files, folders and paths. Microsoft say their bots obey robots.txt rules – They don’t. Bingbot/msnbot occasionaly read the robots.txt file, then immediately afterwards continue on to crawl items specifically listed e.g;

1st Rule Breaker Example – Comments

image of bing and msn bot logoComment paths are disallowed:

  • Disallow: /comment/
  • Disallow: /*/comment/
  • Disallow: /comment/reply/

And the result, Bing crawls these paths

  •   /comment/169   2004/07/13  14:24
  •   /comment/179   2004/07/13  14:25
  •   /comment/201   2004/07/13  14:26

Read the rest of this entry

Forget Google, go Yahoo and Bing

Google’s Latest Faulty Algorithm

Panda 2.5 must rank as the most unsuccessful of all their updates. Speculative estimates of badly affected websites lie in the 60% range. Was your site or blog one of these?

If you have been adversely affected, don’t despair, there are things you can do.

Forget Google for now:

Yes, forget Google. Right now don’t waste your money on SEO services – the next update will more than likely need something totally different and intensive SEO at this point is likely to do more damage than good! (sorry to all the SEO services and specialists out there, part of the reason for Panda 2.5 is a result of heavy SEO). Ensure the other aspects of your site are good. Money spent getting your site builder/manager (not SEO specialist) to check your website will be a better long-term investment.

Concentrate on Bing and Yahoo

Bing and YahooThere are other ways to search the net. Concentrate on the other search engines for now. Both Bing and Yahoo are much more conducive to real website quality than Google has become. They may not index all your pages, nor are they the first choice for most internet users; however with Google now serving up really poor results with shockingly insignificant term matching, a good many net surfers are likely to turn to one of these engines (I have – after getting real trash from very specific search terms these past few days, I am now using Yahoo and Bing as default Search Engines, and finding content I want).

Create accounts with Bing And Yahoo, and list your websites – don’t forget to submit a site map!

Pay Attention to Structure Read the rest of this entry