Raw log files


Why look at what is in log files ?

Raw log files contain all the hits generated by the traffic on your website. So any file, page, image that was loaded on your site will have an entry in the web log file. Most Search Engines or sites will send information in the hit entry of the visitor they send to your site. Google, MSN and most other search engines will put a referrer information in the hit entry of the incoming visitor. This referrer information usually contains what the search query was and sometimes where you were listed. This is important web site traffic statistics you can use to improve your web site traffic.

Log files can be used to find the visits of the bots for example. Knowing the pages that got crawled and how to get them crawled is important. Knowing that Googlebot, msnbot and other bots found your pages allows you to know what is going on with your work and if these are properly linked and found by the different web spiders.

With the server error codes like 404 and 301, you can see if some errors were made creating links on your site or from other sites. My friend Cygnus noticed he was getting a lot of 404 hits for a non existent page. He created a page to get the visitors on a real page instead of his standard 404 error page and convert some of these visits to sales.

It's also good to be able to see if some sites are using your bandwidth. Some webmasters will just use picture on your site to use on their sites, it's called hotlinking. They don't copy it, they link straight to your image so you waste bandwidth giving the file to their visitor. If the image is big and their traffic is high it can bust your hosting bandwidth limit.

When you make a page for a key phrase you don't always expect to rank for other search queries. Sometimes you can rank for something you did not expect and if you tweak that page for something else, you can lose that unexpected traffic. Looking at the page hits and search used, you can see these unexpected searches and prevent losing them.

When you get hit for a listing on page 3 of Google for example, it means that you might get much more traffic being on 2nd or first page. Knowing about deep listings and being able to tweak them to bring them closer to the 1st page is a must.


 • These are only conclusions from observations I made. I do not pretend to say these are facts •


Bot Hunter
Am I white-hat?
Log files
Making pages
Nofollow
Safesearch
SEO