Whether and how to add a robots.txt file and sitemap.xml sitemap

What is a robots.txt file and sitemap.xml most webmasters know. Do not know what you reveal that:
robots.txt is used to instruct search engine bots (and other) what is allowed and what is not. Of course, if these bots will respect the limits imposed upon them is another matter. Googlebot is supposed to obey. Additionally, this file can point to where the site map, or sitemap.xml.
File robots.txt must be in the root directory, and be available at: http://adres_strony.pl/robots.txt.

sitemap.xml is a file containing a site map in XML format. The file name may be different. But it must also be accessible from the main level, eg http://adres_strony.pl/sitemap.xml. In addition, the map can be a list of other maps. I used this example in the pages of translation in another language. On this page is just such a solution.
Whether and how to add a robots.txt file and sitemap.xml sitemap

I will not delve into how to construct the two files. You can even read about it here: Files  robots.txt  and  sitemap.xml. Control behavior of web crawlers.

With the sitemap, Googlebot knows much better what has changed on the site and the address at which new content is located. By. my tests and observations, this gives a slight profit. Google often looks to the side and index new pages more quickly. Of course not count on miracles. The main factor influencing the rate of the index is its "popular website" means the quantity and quality of links leading to the site.

For this part, I expect a new entry for the index within 24 hours. This page is not super-Linked. But in the case of my most popular page (Bucket Forum), A new subject indexes are already in a few minutes, like fresh page Linked with forums index.

Finally, let me add that we ping the site map (let google know that was updated). This is done by calling address:
http://www.google.com/webmasters/sitemaps/ping?sitemap =http://adres_strony.pl/sitemap.xmlOr use the website: Submit Your Sitemap to Google, Yahoo!, Bing & Ask.com ...

When it comes to generating maps to an XML file, it is best to do a script in PHP that will be generated automatically and it will always be current. If this is not possible, you can use the free online tools: XML Sitemap Tool.
Please note that this tool works on the side of our browser, so you have to be wylogowanym on which the map is drawn, otherwise the robot can follow the links, which should not have access.

Related entries:

  • Why not to do a page based on frames

    Why not to do a page based on frames

    Can write a little weird in 2012 on the frames. But take a chance. Statistics on my part shows that there are still some desperados who are going to use frames. Alternatively, find out about it. Since he started to create pages based on a fairly long time frame and continued this unfortunate dealings, have the knowledge and experience sufficient to deter a potential amateur frames. Defects created in the page layout ...
  • What is SEO

    What is SEO

    Search engine optimization (SEO - Search Engine Optimization) is the position of influencing the results of search engines. Nowadays, SEO refers mainly to Google and is not surprising, since it is this search engine generates the most traffic on the Internet. SEO because of the chosen method can be divided into two groups: White Hat SEO (white hat) is a term for ...
  • Unethical SEO - Black Hat SEO

    Unethical SEO - Black Hat SEO

    Unethical SEO, English-speaking market called Black Hat SEO is a very popular method of shill sites ranking on Google. In principle, the same positioning is already quite controversial, because artificially affect the position of your website in search engines. Thus, to determine whether the positioning is ethical or not, is quite smooth. But surely some techniques clearly among those from the black hat. Popular methods used in ...


This entry has not yet commented.

Add a comment

* - Required field