Blog Archives

Bing Banned

Bing and MSN Bots Are Banned

I have banned Bing, Yahoo and MSN search engine spiders from my sites! I’m tired of the constant rule breaking and over-crawling by Bing and MSN search bots.

Bing is a Rule Breaker

Microsoft claims Bing honours robots.txt rules. In my experience that is a blatant lie. Bingbot / msnbot simply ignore robots.txt rules and crawl whatever they want. Some of the specific rules broken include;

  • crawling system folders
  • crawling image folders (msn-media bot). Image folders and extensions jpg,  png,gif, bmp are disallowed
  • crawling RSS feeds. All RSS feeds are disallowed; rss.xml, /feed/, etc
  • crawling comment forms; DOMAIN/comment/184 – the path /comment/ is disallowed in robots.txt

The last straw was today. 2 days ago I added Bing and MSN user agent strings to disallowed bots in robots.txt across all my sites; this morning I see these bots read robot.txt then ignored it totally, and crawled the sites anyway.

Have You Seen Bad Activity by Bing?

Read the rest of this entry

Advertisements

Rulebreaker Bingbot and MSN

Bingbot and MSN bots are Rulebreakers

I’m tired of watching Bingbot and msnbot breaking rules and crawling disallowed files, folders and paths. Microsoft say their bots obey robots.txt rules – They don’t. Bingbot/msnbot occasionaly read the robots.txt file, then immediately afterwards continue on to crawl items specifically listed e.g;

1st Rule Breaker Example – Comments

image of bing and msn bot logoComment paths are disallowed:

  • Disallow: /comment/
  • Disallow: /*/comment/
  • Disallow: /comment/reply/

And the result, Bing crawls these paths

  • 157.56.93.219   /comment/169   2004/07/13  14:24
  • 157.56.93.219   /comment/179   2004/07/13  14:25
  • 157.56.93.219   /comment/201   2004/07/13  14:26

Read the rest of this entry

Microsoft Security Essentials

Microsoft Security Essentials Under Microscope

Microsoft Security Essentials LogoA look at Microsoft Security Essentials, the free anti-virus application from Microsoft. Is Microsoft Security Essentials any good? Will it protect a Windows PC from most common threats? Can MSE compete with commercial security applications?

Over the next few months we will see.

Annoyed with Commercial AV Software.

I have become increasingly annoyed with commercial anti-virus applications. They have become overpriced, use too much system resources, interfere with other applications, or slow down internet access. Worse still, none of the apps are able to detect every virus or malware… An example is Trojan Generic 24, which seems to be only detected by AVG (but doesn’t stop or remove it). Trend Micro Titanium and Norton AV don’t find all versions of this dangerous trojan.

Read the rest of this entry

Win 7 Unstable – Gigabyte Q1000C Netbook

Windows 7 and Netbooks

Gigabyte Q1000C NetbookI have had a Gigabyte Q1000C Netbook on the workbench this week. The mini laptop came with Windows 7 Starter pre-installed. First off let me comment on ‘7 Starter’: I do not like these starter operating systems, for that matter all the ‘Home’ versions of every Windows operating system, XP, Vista and 7. They always seem to have problems. It would have been preferable for Microsoft to provide the basic Pro version at reduced price to OEM’s and the public market.

However the problems with Netbooks (at least this example) seems more about the lack of ability of the Intel Atom CPU, and insufficient RAM (2GB), to adequately run Windows 7.

Win 7 Instability Problems

Microsoft is aware of problems running Windows 7 on Netbooks. Their response is “There are too few problems to warrant a fix by a service pack or regular update, instead each issue will be addressed by a Hot-Fix”. Not really an adequate response to the numerous issues raised in online tech forums by owners of Netbooks.

Windows 7 LogoThe problems with this example:

  • Windows Freezes: An ongoing problem that has persisted with the machine since the owner purchased it. Since arriving on the bench on Monday, this has occurred every day, initially so often it was almost impossible to do a Windows update (The owner had struggled with this, and ended up disabling updates – even Service Pack 1 had not been installed).
  • Blue Screen of Death: No fewer than 5 instances of the ubiquitous Windows ‘friend’, the blue screen. After all these years, one would have expected Microsoft to move away from the basics that cause this error – a flawed basic system. After Windows XP, Microsoft should have taken the route Apple took with OSX – Unix.

Fixes Applied:

Read the rest of this entry

Open Source Software Popularity

Open Source Becoming Mainstream

“Open source is a development method for software that harnesses the power of distributed peer review and transparency of process” (Open Source Initiative)

Considering a poll being conducted on eWEEKeurope open source software is gaining penetration with business users.

OpenSource Poll Results

OpenSource Poll Results

47% of respondents significantly favour open source applications, and a further 15% are trying it out.

Slightly more tha 63% of responders make use of OpenSource to some extent.

A further 8% state they might use it but are locked into propretory software.

Only 13% said they they do not trust it.

Operating Systems:

Refering to a Softpedia article, Linux makes up 40.6% of the operating systems in use, while Microsoft Windows still holds a narrowing lead of 57.5%

Open Source in South African Public Sector

As far back as 2003 open source software started gaining support at official levels in South Africa. In 2003 the CSIR (Council for Scientific and Indsustrial Research) Launched an project to assist in the development of open source in the region (SouthAfrica.info).

A number of Government departments today rely on opensource, including The Northern Cape Provincial Government, Department of Land Affairs, and SA Revenue Services (Linux and Oracle). Education also makes extensive use of open source applications for in-school computer learning programs.

Facebook and Twitter Opensource.

I wonder how many of the approximate 800 million Facebook users realise the worlds most used social media network is based on open source technology.

Then there are the more than 600 million users of Twitter, another major open source appplication. How many of the 13% who ‘do not trust’ open source are regular tweeters, and spend hours every week on Facebook?

WordPress

Read the rest of this entry

Laptop Windows Recovery

No Windows Disk With Laptops

I recently came across a post regarding a problem experienced in trying to recover a broken Windows Vista Operating System on a Laptop.

windows 7 logo

Windows 7

The laptop was supplied without a windows installation disk; which is an increasingly common scenario. Laptop manufacturers (OEM – Original Equipment Manufacturers) seem to feel that the built-in recovery function is sufficient, and saves them a few cents of the cost of supplying a disk.

Sadly, the recovery system does not always work… damaged hard drives, viruses, and numerous other lesser factors can prevent this from working as expected.

The unhappy laptop owner put the blame on Microsoft; “since Microsoft stopped supplying installation disks with laptops” I contend this is not Microsoft’s fault, it is the OEM laptop manufacturers fault.  Microsoft does not manufacture laptops. In fact Microsoft made Vista readily available, and 7 can be installed from any disk using the matching version and the PC owners ‘key’.

Microsoft does not sell Windows CD’s, they sell the licence, i.e. the right to use the operating system. Anyone can download the operating system from their website for free, and even run it on trial for a limited period of time. When you decide to keep the version you are happy with, purchase the licence, enter the activation code, and you have a full and legal Windows installation.

I agree that not having the disk on hand is annoying. Perhaps it is time Microsoft demanded OEM’s provided an installation disk included with the package. Consumer demand is of course the real answer. Refuse to buy any equipment that does not come with a legal version of the operating system, whetehr it is new or used.

It is not exactly a new phenomenon, Going as far back as Windows XP and perhaps even earlier, OEM’s were providing their own disks, which often had a customised Windows installation, rather than the standard version which Microsoft provides on their own disks. Very often these OEM’s (and I mention Dell, and Hewlett Packard) had hard coded the BIOS, which was then locked, making it extremely difficult for owners, even technicians, to install any other version of operating system. They of course will contend this is to “improve the user experience of their products” and other such nonsense! This practice was not only limited to laptops, but the desktops from some manufacturers also had this limitation built-in.

I believe they do this in order to lock their customers into their own support network! While it is reasonable to expect the owner of ANY equipment to be required to make use of a manufacturers support service during the WARRANTY period, making it extremely difficult for the owner to choose another service agent outside of warranty is a violation of consumer rights, definitely in my own country, and I am certain this applies in the USA and much of the EU as well. Consumers have a right to choose!

All is not lost however

As mentioned, the operating system can be obtained from Microsft or other sources. Just download it (using another PC if the one with the problem has totally ceased to work, burn it to a disk a writeable DVD will be required) and run this on the PC (or laptop) required. You may need to choose which version to use.

On your laptop or desktop, there will be a ‘certificate of authenticity’ which describes the version the licence is for, and the ‘key’ to enter when requested. That’s it, the installer will continue and the system will soon be back up and running.

Real life examples: Read the rest of this entry

Flash Player Plug-in Crashing

Flash Plugin Crashing

Adobe Flash Player logo

Adobe Flash Player plugin crashes in Win XP systems are usually caused by a combination of older graphics card drivers using Microsoft Direct Draw 9  (DX 9) and an up to date version of the plugin.

Assuming the actual hardware components are OK – and if the display is working they probably are, the user has a choice of two routes – driver update or OpenGL.

Read the rest of this entry