Many top sites run on NGINX these days. While NGINX can make your site run smoother and handle more load, it can’t stop web bots on its own. The good news is you can easily block any bot (good or bad) by making a small change to your configuration file.
To get started, you should figure out which bots are hitting your server and the number of times it is happening. The above command line lets you know how many hits you are getting from “A6-Indexer.” You can change that to any other bot name. The below command lets you know the IPs that are hitting your server, so you can run an IP trace on them.
watch "tail -n 500 /var/log/nginx/access.log | cut -d' ' -f1 | sort | uniq -c | sort -gr"
Once you have determined which IPs you would like to block, you should add the following to your server block (default.conf). As you can see, we are only blocking a few bots, but you can adds as many as you like. It is worth nothing that not all bots should be blocked.
Which bots are you blocking on your server?