Author: admin | Monday, May 14, 2012, 22:18 hours | Read 49150 times
robots.txt is used to instruct search engine bots (and other) what is allowed and what is not. Of course, if these bots will respect the limits imposed upon them is another matter. Googlebot is supposed to obey. Additionally, this file can point to where the site map, or sitemap.xml.
File robots.txt must be in the root directory, and be available at: http://adres_strony.pl/robots.txt.
sitemap.xml is a file containing a site map in XML format. The file name may be different. But it must also be accessible from the main level, eg http://adres_strony.pl/sitemap.xml. In addition, the map can be a list of other maps. I used this example in the pages of translation in another language. On this page is just such a solution.
I will not delve into how to construct the two files. You can even read about it here: Files robots.txt and sitemap.xml. Control behavior of web crawlers.
With the sitemap, Googlebot knows much better what has changed on the site and the address at which new content is located. By. my tests and observations, this gives a slight profit. Google often looks to the side and index new pages more quickly. Of course not count on miracles. The main factor influencing the rate of the index is its "popular website" means the quantity and quality of links leading to the site.
For this part, I expect a new entry for the index within 24 hours. This page is not super-Linked. But in the case of my most popular page (Bucket Forum), A new subject indexes are already in a few minutes, like fresh page Linked with forums index.
Finally, let me add that we ping the site map (let google know that was updated). This is done by calling address:
http://www.google.com/webmasters/sitemaps/ping?sitemap =http://adres_strony.pl/sitemap.xmlOr use the website: Submit Your Sitemap to Google, Yahoo!, Bing & Ask.com ...
When it comes to generating maps to an XML file, it is best to do a script in PHP that will be generated automatically and it will always be current. If this is not possible, you can use the free online tools: XML Sitemap Tool.
Please note that this tool works on the side of our browser, so you have to be wylogowanym on which the map is drawn, otherwise the robot can follow the links, which should not have access.
Comments:This entry has not yet commented.
Add a comment
- Black Hat SEO
- black hat soe
- how to add a site map
- site map
- Unethical SEO
- positioning the base
- White Hat SEO
czas generowania - całkowity: 0.2088 sek.
czas generowania - MySQL: 0.184 sek.
czas generowania - PHP: 0.0248 sek.
czas generowania - translator: 0.1775 sek.
Ilość przetłumaczonej treści: 99 %
ilość zapytań MySQL: 56
wykorzystana pamięć RAM: 1 283,18 kb