It is often a good idea to  restrict search engine spiders or crawlers from accessing certain parts of your website.  This is done with a file called a ROBOTS.TXT file.  Fortunately, Prestashop 1.5 does this for you.  The following tutorial will walk you through the process of getting the file created.

How to create a robots.txt file in Prestashop 1.5

  1. Login to the Administration Panel as an Administrator
  2. Hover over the menu bar where it says PREFERENCES, and then click on SEO & URLS in the drop down menu.  You will see an existing list of URLs.  Scroll down further until you see ROBOTS FILE GENERATION. The following screenshot indicates the button to click that will generate the ROBOTS.TXT file:


    prestashop15-robotsfile


  3. The information outlined in blue is important as it lets you know where the file is required to be located.  By default, in Web Host Hub's servers, the location of the file is in the PUBLIC_HTML folder if the website is the primary domain.  If it is an addon domain, then the path to the ROBOTS file will look like this:

    /public_html/add-on-domain-name.com

    If you are not hosting with Web Hosting Hub, please consult your Technical Support or documentation about the location of your files.
  4. Click on the GENERATE ROBOTS.TXT FILE button and the ROBOTS file will automatcially be created. 
  5. The ROBOTS.TXT file will generate the necessary information to limit access to search engine spiders and crawlers.  Here's the top of the file:


    # robots.txt automaticaly generated by PrestaShop e-commerce open-source solution
    # http://www.prestashop.com - http://www.prestashop.com/forums
    # This file is to prevent the crawling and indexing of certain parts
    # of your site by web crawlers and spiders run by sites like Yahoo!
    # and Google. By telling these "robots" where not to go on your site,
    # you save bandwidth and server resources.
    # For more information about the robots.txt standard, see:
    # http://www.robotstxt.org/wc/robots.html

The rest of the file includes the entries to keep the robot from going where it doesn't need to go.  The most important part to take from this is that limiting the access helps to save BANDWIDTH and SERVER RESOURCES.

# robots.txt automaticaly generated by PrestaShop e-commerce open-source solution
# http://www.prestashop.com - http://www.prestashop.com/forums
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/wc/robots.html
Did you find this article helpful?

We value your feedback!

Why was this article not helpful? (Check all that apply)
The article is too difficult or too technical to follow.
There is a step or detail missing from the instructions.
The information is incorrect or out-of-date.
It does not resolve the question/problem I have.
How did you find this article?
Please tell us how we can improve our Help Center:
Email Address
Optional, but our team may contact you for more information.
Did you find this article helpful?

Post a Comment

Name:
Email Address:
Comment:
Are you a bot?
Submit

Please note: Your name and comment will be displayed, but we will not show your email address.

News / Announcements

Update to SSL Certificates - Certificate Warnings
1969-12-31 11:00 am EST
Hits: 635
What is the Heartbleed bug?
1969-12-31 11:00 am EST
Hits: 261

Related Questions

Here are a few questions related to this article that our customers have asked:
Ooops! It looks like there are no questions about this page.
Would you like to ask a question about this page? If so, click the button below!

Help Center Search

Current Customers

Email: support@WebHostingHub.com Ticket: Submit a Support Ticket
Call: 877-595-4HUB (4482)
757-416-6627 (Intl.)
Chat: Click To Chat Now

Ask the Community

Get help with your questions from our community of like-minded hosting users and Web Hosting Hub Staff.

Not a Customer?

Get web hosting from a company that is here to help.