Python: Bypass CloudFlare 'Verifying Browser' in Requests
Usually you can make HTTP requests on CloudFlare-protected websites, but depending on their Security Level (as
I'm Under Attack!), your requests won’t pass their firewall.
- Quick answer: cloudflare-scrape
Making requests via command-line you may get the response
503 Service Unavailable, it means your script is stopping at this page:
The ‘make it yourself’ way
Use PyExecJS to run JS code in Python, you’ll still need extra work to evaluate CloudFlare protection. Interesting module though.
The ‘ready to use’ way
pip install cfscrape
import cfscrape # Requests wrapper requests_flare = cfscrape.create_scraper() r = requests_flare.get("https://github.com") # or post() r.status_code # 200
- If you’re getting a CAPTCHA instead (common for TOR exit nodes), this technique won’t work.