Crawl Delay

Asked by:
MacBros / 5 Points
2011-08-17 6:05 pm EST
Website Troubleshooting
I'm trying to add Crawl Delay which was recommended by you but when I add it I get a 500 error

User-agent: *
Crawl-delay: 30

Never Mind. I put it in the .htacces file instead of the robots.txt file..

To ask this user for more information, please first login.

To submit an answer, please login.

Best answer chosen by Web Hosting Hub staff


1,846 Points
2011-08-17 8:11 pm EST
Hi MacBros,

I recommend our article on troubleshooting 500 internal server errors. It covers issues in the .htaccess.

It sounds like you got this one sorted out on your own, and updated your question?

We're here if you need any further support, simply ask!

- Brad

To submit a comment on this answer, please first login.

Want to share this Question?

Related Articles

It looks like there are no related articles.
Would you like to ask a question about this page? If so, click the button below!

Help Center Search

Current Customers

Email: Ticket: Submit a Support Ticket
Call: 877-595-4HUB (4482)
757-416-6627 (Intl.)
Chat: Click To Chat Now

Ask the Community

Get help with your questions from our community of like-minded hosting users and Web Hosting Hub Staff.

Not a Customer?

Get web hosting from a company that is here to help.