It's like when Google index the site they have a program running on their servers which visits a page on NewRider and copies the text to process for their search engine. They can run multiple of these programs at once so it's like 100 people all clicking to view 100 different pages all at once. It puts a brief load on the server and you have to wait a bit longer for the page you've asked for. Google and Bing can slow things down especially if they are visiting at the same time ( they are at the moment). Then you get less well behaved search bots from other indexing sites, especially China and they can try and index 1000+ pages all at the same time and that can hammer the site. It's those I'm trying to reject.