Said It Here

How to stop excessive bot traffic and AI scrapers

Massive volumes of automated scrapers overwhelm sites with tens of thousands of public pages, driving hosting costs up significantly. This overwhelming bot traffic ruins the reliability of web data and forces owners to look for specialized blocking services to regain control.

OverviewLatest thread: Nov 2026
People affected22 accounts (4 forums)
Workarounds found50 items recorded
Discussion timelineSep 2026 – Nov 2026
Freshness statusActive Consensus
Editorial ConsensusSynthesized across Hacker News, r/shopify, r/sysadmin and 1 more · Sep 2026 – Nov 2026
Verification standards →

Core Friction Pattern

Analysis of 22 peer accounts across online developer and community forums reveals that “How to stop excessive bot traffic and AI scrapers” repeatedly surfaces due to edge-case mismatches between standard defaults and real-world environments. Standard troubleshooting frequently falls short, compelling practitioners to develop custom workarounds.

Top Peer-Reported Fix
using Vercel and Cloudflare to block traffic

+49 alternative community-tested workarounds cataloged below.

Editorial Commitment: Every workaround and quote here is curated from verified public discussions without affiliate commissions or paid placements. Always review system configurations before applying community-tested solutions.

What the numbers show

Counted from the threads quoted below. How we count

  • People have been saying this in most months since September 2026.
  • This comes up 98.2× more often in r/Wordpress than across the site as a whole.

What people tried

Every workaround mentioned in the threads below. We haven’t tested any of them — and nobody here is claiming they worked.

50 items+ Contribute

Found another way around this?

Share your steps and trade-offs so fellow engineers don't have to reinvent the wheel. No account required.

Active Consensus (Latest: Nov 2026)Indexed across peer discussions from Sep 2026 to Nov 2026
Evaluated against breaking changes & deprecations
  1. 1
    using Vercel and Cloudflare to block traffic
    Did this help you?
  2. 2
    putting the app behind an auth gate with cloudflare turnstile
    Did this help you?
  3. 3
    switching to hcaptcha
    Did this help you?
  4. 4
    using truesign.ai to block crawlers
    Did this help you?
  5. 5
    Introducing a WAF rule in Cloudflare to block visits via wildcard patterns
    Did this help you?
  6. 6
    Adding nginx rules to return a 404 status on pagination URLs
    Did this help you?
  7. 7
    Disabling archives and feeds from being indexed or crawlable using SEO plugins like Yoast or Rank Math
    Did this help you?
  8. 8
    Checking access logs by user agent to analyze bot behavior
    Did this help you?
  9. 9
    Locking down IT access and trying to secure company data
    Did this help you?
  10. 10
    Pre-anonymizing data before letting AI systems process it
    Did this help you?
  11. 11
    Manually applying the human or bot session filter in the dashboard
    Did this help you?
  12. 12
    Using Cloudflare WAF and custom WAF rules to block traffic
    Did this help you?
  13. 13
    Setting up rules and cache to manage server load
    Did this help you?
  14. 14
    blocking entire countries or geographical areas using Cloudflare
    Did this help you?
  15. 15
    asking the web host to block traffic from certain regions
    Did this help you?
  16. 16
    setting up Cloudflare security rules to block common threats
    Did this help you?
  17. 17
    Setting up Cloudflare with Bot Fight Mode / Block AI Bots enabled
    Did this help you?
  18. 18
    Upgrading hosting plans or increasing PHP workers
    Did this help you?
  19. 19
    Migrating to better hosting companies (e.g., Kinsta, GreenGeeks)
    Did this help you?
  20. 20
    Setting rate limiting or Crawl-delay in robots.txt / server config
    Did this help you?
  21. 21
    Blocking specific IP ranges or countries (Geo-blocking)
    Did this help you?
  22. 22
    Asking support to temporarily filter abusive traffic or apply rate limiting at the server level
    Did this help you?
  23. 23
    Outright dropping the single IP address instead of trying to overcomplicate rate limiting.
    Did this help you?
  24. 24
    Using fail2ban to scan logs and automate drops.
    Did this help you?
  25. 25
    Using ipset to block known bogus IPs.
    Did this help you?
  26. 26
    Checking the IP's geolocation and blocking it if it's outside the target audience country.
    Did this help you?
  27. 27
    Leaving 'Under Attack' mode on permanently, despite it showing a loading screen to real users.
    Did this help you?
  28. 28
    Creating custom Web Application Firewall (WAF) rules to block or challenge specific IPs, countries, or paths.
    Did this help you?
  29. 29
    Caching HTML at the CDN level so bot requests never reach the origin server running PHP.
    Did this help you?
  30. 30
    Adding specific search engine bot blocks directly to the .htaccess file.
    Did this help you?
  31. 31
    Disabling XML-RPC to prevent brute force attempts.
    Did this help you?
  32. 32
    Using bot throttling plugins to temporarily block scraping behavior.
    Did this help you?
  33. 33
    Setting up Web Application Firewall (WAF) rules to block specific crawler user agents.
    Did this help you?
  34. 34
    Disabling plugins that inject sitewide JavaScript to reduce main thread blocking.
    Did this help you?
  35. 35
    Checking server access logs to identify the exact behavior of the bots.
    Did this help you?
  36. 36
    Looking into adding a txt file (robots.txt) to block crawler access.
    Did this help you?
  37. 37
    Migrating DNS and protection services to specialized platforms like Cloudflare.
    Did this help you?
  38. 38
    Implementing geo-blocking, IP bans, and strict access controls via tools like Netbird.
    Did this help you?
  39. 39
    Reissuing compromised email accounts to affected users.
    Did this help you?
  40. 40
    Setting up Cloudflare WAF or edge caching to block bad traffic before it reaches the origin server
    Did this help you?
  41. 41
    Installing security plugins like Wordfence, Bottle, or Fail2ban to block or throttle malicious IPs
    Did this help you?
  42. 42
    Implementing object caching via Redis or Memcached to reduce database load
    Did this help you?
  43. 43
    Blocking scraper IP addresses in Cloudflare
    Did this help you?
  44. 44
    Adding Cloudflare WAF or security rules to restrict access to REST API endpoints
    Did this help you?
  45. 45
    Filing DMCA copyright removal requests with Google and the copier's hosting provider
    Did this help you?
  46. 46
    Setting RSS feeds to show excerpts only rather than full posts
    Did this help you?
  47. 47
    Using rate limiting and bot mitigation tools
    Did this help you?
  48. 48
    adding robots.txt rules blocking bots from specific language folders
    Did this help you?
  49. 49
    implementing rate limits for user agents that ignore robots.txt
    Did this help you?
  50. 50
    disabling the translation plugin
    Did this help you?
Tried another approach not listed above?+ Share your workaround ↑

In their words

Unedited, grouped by where they were said, most upvoted first within each place, each linked to the thread it came from.

r/sysadmin4 people · September 2026 – November 2026

“The last few months we have seen a huge uptick in ai attacks combined with what looks like human intervention.. they automated creating accounts, we added a rate limit. They altered their code to stay under our rate limit. We increased the difficulty in our captcha they seemed to crack it anyway.”source ↗

Relative_Charge_9410 · r/sysadminNov 2026· 97 upvotes

“I looked in my logs and noticed one bot (with the same IP and same user agent) has been making requests to my server at the rate of one request per second for about 5 minutes.”source ↗

“I thought about using the recent module of iptables but I'm not sure if that is the best because if I pick too low of numbers then I block legit users and if I pick too high of numbers then my system might go to a crawl.”source ↗

Ok_Strike9189 · r/sysadminOct 2026· 4 upvotes

“I'm doing what I can to secuure my company's data, but I feel at this point it is going to be pointless because it feels like legislation is going to change to make trademark/copyright not a real enforceable thing.”source ↗

Siphyre · r/sysadminSep 2026· 1 upvotes

“Banning an ip may be an option but what if a legitimate user also uses that IP as well at a later time?”source ↗

Ok_Strike9189 · r/sysadminOct 2026· 1 upvotes

“I just spent the weekend updating CloudFlare rules to stop huge volumes of site scraper bots, including going full Bot fight mode. Best I achieved was a 2 hour lull before they moved to a new host.”source ↗

charmingpea · r/sysadminNov 2026· 1 upvotes
r/shopify2 people · September 2026 – October 2026

“these last months, maybe the last year, the sessions on my store went crazy : 100-200 simultaneous sessions non-stop, day and night.”source ↗

“My conversion rate in August : 0.66% with the bots, 1.57% without.”source ↗

christianJulesAl · r/shopifySep 2026· 21 upvotes

“Since it's my business I of course need to link my shop, but f*** AI slop and them making use of my stuff.”source ↗

“Is there a way I can block them off Shopify?”source ↗

“I'm not a coder or anything, saw a short Info of a txt that can be added to block the "friendly" bots?”source ↗

TherbisOfficial · r/shopifyOct 2026· 1 upvotes
r/Wordpress14 people · September 2026 – November 2026

“My concern is the contact information.”source ↗

“If someone can simply scrape the pages and collect both the listings and all of their contact details, they could essentially recreate a large part of the directory elsewhere.”source ↗

“What I’m trying to understand is what practical measures experienced WordPress developers use to make large-scale scraping of contact information significantly harder without making the site frustrating for legitimate users or hurting SEO.”source ↗

“I’m currently quite lost on the best solution for this, so I’d appreciate practical advice from people who have dealt with this problem.”source ↗

Mountain-Monk-6256 · r/WordpressSep 2026· 19 upvotes

“I noticed Ive been getting a high load of Traffic from China, its my top traffic source and im located in the USA. I read that I should block China traffic, but how do I do that on Wordpress? Also is it safe for SEO to block traffic from China?”source ↗

Borange81 · r/WordpressSep 2026· 18 upvotes

“My website crashes every other hour for an hour so down like 50% of the time UNLESS I have under attack mode on in Cloudflare which blocks bots. If I turn it off and the website crashes, it immediately comes back online a minute after turning it on.”source ↗

“Are these number normal? Is there a way to fix this without having under attack mode on 24/7? I feel like it annoys the viewers with that 5 second loading screen and also potentially effect google analytics.”source ↗

audit157 · r/WordpressOct 2026· 8 upvotes

“When I try to see the stats for my site I often get the "database connection error" message which would seem to indicate that the db engine is overwhelmed by the requests, forcing me to restart the mysql and apache daemons.”source ↗

Brassens71 · r/WordpressNov 2026· 6 upvotes

“i have a wordpress blog that contains some content and fields, i noticed that there is a competitor website that is running a script on my website and its copying its content 100% and the issue is it is ranking above me”source ↗

JosephGappy · r/WordpressNov 2026· 5 upvotes

“Woocommerce site on godaddy managed wp throwing constant 503s even with plugins disabled”source ↗

“In access log it shows dozens of different IPs from all over the world per second. Possible bot problem?”source ↗

cb8_ · r/WordpressOct 2026· 3 upvotes

“suddenly started seeing a lot of hits to the translated version of our site that Weglot produces. Like over 7,000 in under 30 minutes. I couldn't find any documentation on the Weglot site for settings I could change to fix the issue so I'm hoping to crowdsource some other ideas or things to look into.”source ↗

tired-NPO-worker · r/WordpressNov 2026· 3 upvotes

“We're experiencing a lot of bot traffic going to those addresses and are not sure why.”source ↗

“I put the Cloudflare rule in place this morning (2-3 hours before writing this) and according to Cloudflare the rule has been triggered 8,430 times since I created it!”source ↗

International-Dog869 · r/WordpressSep 2026· 2 upvotes

“My WP site for my self-published books has very low traffic counts, but these last two days, it's seeing hits from all over China.”source ↗

NVBoomer · r/WordpressOct 2026· 2 upvotes

“probably bots scraping content, happens to my art portfolio all the time”source ↗

TheSteepCrocodile · r/WordpressOct 2026· 2 upvotes

“Even the dmca won't help because he is creating new websites and domains all run the same script.”source ↗

JosephGappy · r/WordpressNov 2026· 2 upvotes

“Really? I’ve been trying to sort my server with rules and cache.”source ↗

pheyonagh · r/WordpressSep 2026· 1 upvotes

“I had this problem. It was low-rent AI bots hammering my shop. When bot-hammering uses up your server quotas GoDaddy’s throws 503s to your audience.”source ↗

Aggressive_Ad_5454 · r/WordpressOct 2026· 1 upvotes

“Had similar resource usage on my server with a very low traffic site. Moved it to a new server. Monitored requests with cloudflare and Google. Noticed a metric ton of search requests for non-existent locations under the domain.”source ↗

Crazy-Own · r/WordpressOct 2026· 1 upvotes

“I used the block anthropic function and set up all the waf rules so the site isn't going offline anymore. However after doing that everything slowed down and it says 10 duplicate queries and 10 slow queries and 40 second wait time with query monitor.”source ↗

“Blocking anthropic suddenly triggered a huge mess and I don't know why. Also the bot hits were 700k before blocking now they are 2 million. Though in the events pane it says blocked but they are still served data of 20gb.”source ↗

audit157 · r/WordpressOct 2026· 1 upvotes

“I'm battling this problem a content site I run and despite spending over $5k on attempting to solve the problem, its still cat and mouse.”source ↗

ContributionEasy6513 · r/WordpressNov 2026· 1 upvotes

“I have sent multiple DMCA reports to the hosting provider, cloudflare and the domain registrar waiting for their response.”source ↗

JosephGappy · r/WordpressNov 2026· 1 upvotes

“tbh this is first time dealing with this stuff and im afraid that google dmca need a lawyer or something should i be hiring?”source ↗

JosephGappy · r/WordpressNov 2026· 1 upvotes
Hacker News2 people · September 2026

“We host a web app which has around 50,000 statically generated public pages, and the amount of bot traffic is insane.”source ↗

“I’ve never seen anything like it before, our hosting costs are up like crazy, and besides all of this, I can’t trust anything I read or see on this web anymore.”source ↗

flyingcoder · Hacker NewsSep 2026· 6 upvotes

“I occasionally suffer this same problem when happy crawlers find my site and use truesign.ai to block them, so far successfully.”source ↗

babyDoomer · Hacker NewsSep 2026

Where this came up

People with this problem also raised

Related community dilemmas clustered by discussion consensus

6 related topics

Why are automated bot attacks and hacking attempts suddenly surging?

Sites on platforms like WordPress and Shopify are being overwhelmed by relentless daily bot attacks and scripts like wp2shell that exploit unmaintained pages and checkout links. This constant barrage prevents small business owners from operating normally, forcing them to spend money on services like Cloudflare while leaving them vulnerable to site cloning and identity theft.

8 workarounds4 accounts · 2 forums

How to stop an email spam bomb attacking a user inbox

An ongoing email spam bomb floods a user's inbox with hundreds of random newsletter and phishing messages a day in multiple languages. This relentless wave buries legitimate communications and overwhelms standard inbox management, leaving administrators searching for effective filtering and mitigation strategies.

5 workarounds3 accounts · 2 forums

Why does a WordPress site need a plugin for everything?

Too many plugins bog down site performance and turn basic features—like cloning posts—into bloated dependencies. This makes routine site maintenance exhausting and prevents pages from loading quickly.

23 workarounds13 accounts · 1 forum

How to stop cache warming from crashing the server

Running scheduled curl scripts across hundreds of URLs every thirty minutes creates too much parallel traffic as a site grows. This heavy request load hammers the origin server and can easily kill site performance when the sitemap expands.

3 workarounds2 accounts · 1 forum

Why is my website traffic suddenly spiking with no sales?

Sudden traffic surges with flat conversion rates and zero browsing activity usually indicate automated bot traffic. This phantom traffic distorts analytics and wastes server resources without generating actual revenue, leaving store owners unable to tell if real customers are visiting or how to filter the noise.

8 workarounds6 accounts · 3 forums

Why is Intelligo audio causing high CPU on domain controllers?

An Intelligo audio component installed on client workstations floods domain controllers with excessive lookup requests, driving lsass.exe CPU usage above 90% during business hours. Standard troubleshooting like adjusting heartbeat settings or server configurations fails to stop the traffic, leaving servers continuously overloaded.

9 workarounds3 accounts · 2 forums