WebsiteDays.com create your web site free, Hosting, Coding, E-commerce, Free Website Builder. Download free website builder to create your website! Personal, and ECommerce website builders...

WebsiteDays.com for india ernakulam

Click on your region below to contact a Websitedays.com designer, or Websitedays.com sales person for info about Online Website builder system.

You can also send an email to

and we can forward your enquiry to respective designers in your area.

Agartala Agra Ahmedabad
Ahmednagar Aizawl Ajmer
Akola Alibag Aligarh
Allahabad Alleppey Almora
Alwar Ambala Amravati
Amritsar Amroha Anand
Anandpur-Sahib Ankleshwar Arpora
Aryankavu Aurangabad Auroville
Ayodhya Badlapur Balipara
Ballabgarh Bangalore Bareilly
Belgaum Bellary Bhangar
Bharatpur Bharuch Bhavnagar
Bhawanipatna Bhilai Bhopal
Bhubaneswar Bhuj-Rudramata Bicholim
Bijapur Bijnor Bikaner
Bodh Gaya Bokaro Steel City Brahmapur
Bundi Calicut Kozhikode Cannanore
Chandannagore Chandausi Chandigarh
Chandipur Changanacherry Chengannur
Chennai Madras Cherthala Chhatarpur
Chinsurah Chitradurga Chittoor
Cochin Kochi Coimbatore Cuddalore
Cuttack Darjiling Dehra Dun
Delhi Dewas Dhanbad
Dharamsala Dindigul Dungarpur
Durgapur Elamanchili Ernakulam
Erode Faridabad Gandhinagar
Gangtok Gauhati Ghaziabad
Gobindgarh Gorakhpur Greater Noida
Guntur Gurgaon Guruvayur
Gwalior Haridwar Hazaribag
Hisar Hosur Howrah
Hubli Hyderabad Igatpuri
Imphal Indore Itanagar
Jabalpur Jaipur Jaisalmer
Jalandhar Jamnagar Jamshedpur
Jodhpur Kakinada Kaladi
Kalimpong Kalpetta Kalyan
Kanchipuram Kannur Kanpur
Kanyakumari Kapadvanj Karauli
Karnal Karur Karwar
Kashmir Kassar Kausani
Khajuraho Kharagpur Kishangarh
Kodaikanal Kolhapur Kolkata Calcutta
Kollam Komarapalayam Kootanad
Kota Kothamangalam Kottakkal
Kottayam Kovilpatti Kullu
Kumaon Kumbakonam Kumta
Lonavala Lovedale Lucknow
Ludhiana Madurai Mahe
Mamallapuram Mahabalipuram Manali Mangalore
Manjeri Mardol Margao
Matheran Mathura Meerut
Moradabad Morbi Mumbai Bombay
Munnar Mussoorie Mysore
Nabha Nadiad Nagaur
Nagpur Nainital Narendranagar
Nasik Navsari Nawalgarh
Nellore Noida Ooty
Panaji Panchgani Panchkula
Panipat Panvel Patiala
Patna Pattambi Perumbavoor
Phagwara Pilani Ponda
Porbandar Pune Puri
Pushkar Raipur Rajapalayam
Rajkot Rajpipla Ramanathapuram
Ranchi Ratnagiri Rishikesh
Roorkee Saharanpur Salem
Sanawar Sangli Satara
Satna Sawai Madhopur Secunderabad
Shillong Shimla Sidhi
Sirsa Sivakasi Solan
Solapur Sonepat Srinagar
Sullia Surat Surendranagar
Tezpur Thalassery Tellicherry Thane
Thanjavur Thekkadi Thiruvalla
Thiruvananthapuram Trivandrum Thrissur Tiruchirappalli
Tirumala Tirunelveli Tirupati
Tiruppur Tirur Trichur
Tumkur Tuticorin Udaipur
Udupi Ulhasnagar Vadodara
Valsad Varanasi Vellore
Vijayawada Virudunagar Visakhapatnam
Vrindavan Warangal Yamunanagar
Yercaud

»   The Proper Way To Use The robot.txt File

Article submitted in category: Web Design And Development
Tags: | disallow | file | user-agent | index | robottxt file | robottxt | spiders | * disallow | user-ag

The Proper Way To Use The robot.txt File by: Jimmy Whisenhunt

When optimizing your web site most webmasters don’t consider using the robot.txt file. This is a very important file for your site. It let the spiders and crawlers know what they can and can not index. This is helpful in keeping them out of folders that you do not want index like the admin or stats folder.

Here is a list of variables that you can include in a robot.txt file and there meaning:

User-agent: In this field you can specify a specific robot to describe access policy for or a “*” for all robots more explained in example.

Disallow: In the field you specify the files and folders not to include in the crawl.

The # is to represent comments

Here are some examples of a robot.txt file

User-agent: *

Disallow:

The above would let all spiders index all content.

Here another

User-agent: *

Disallow: /cgi-bin/

The above would block all spiders from indexing the cgi-bin directory.

User-agent: googlebot

Disallow:

User-agent: *

Disallow: /admin.php

Disallow: /cgi-bin/

Disallow: /admin/

Disallow: /stats/

In the above example googlebot can index everything while all other spiders can not index admin.php, cgi-bin, admin, and stats directory. Notice that you can block single files like admin.php.

About The Author

Jimmy Whisenhunt is the webmaster at VIP Enterprises http://www.vipenterprises.org

vipenter@vipenterprises.org

This article was posted on February 06, 2005