Key Takeaways
Avoid detection in Playwright scraping: stealth settings, headers, fingerprint, and proxies. Reduce bot signals and blocks with best practices.
Why Playwright Gets Detected
Playwright drives a real Chromium (or Firefox/WebKit) browser, so it’s already closer to a real user than a simple HTTP client. Sites can still detect automation via browser flags (e.g. navigator.webdriver), fingerprint (canvas, WebGL, fonts), behavior (timing, lack of mouse movement), and IP (datacenter vs residential). This guide covers how to reduce detection when using Playwright for scraping. See browser fingerprinting, how websites detect scrapers, and web scraping without getting blocked. For production, combine with residential proxies and best proxies for web scraping.
Browser launch and context (reduce obvious bot flags)
Use a realistic viewport, locale, and timezone so the environment matches a normal user. Avoid headless if the site is very strict (use headless: false or the newer “headed/headless=auto” modes).
Example Playwright launch with basic stealth defaults:
import { chromium } from 'playwright';
const browser = await chromium.launch({
headless: true, // try false if a site is very strict
});
const context = await browser.newContext({
viewport: { width: 1366, height: 768 },
userAgent:
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36',
locale: 'en-US',
timezoneId: 'America/New_York',
});
const page = await context.newPage();See Playwright web scraping tutorial and headless browser scraping. Bypass Cloudflare and handling CAPTCHAs when challenges appear.
Proxies and IP (don’t look like a bot farm)
The IP is one of the strongest signals. Datacenter IPs are often blocked or limited. Use rotating residential proxies so traffic comes from real-user IPs.
Playwright proxy example:
import { chromium } from 'playwright';
const browser = await chromium.launch({
headless: true,
proxy: {
server: 'http://p1.bytesflows.com:8001',
username: process.env.BYTESFLOWS_USERNAME,
password: process.env.BYTESFLOWS_PASSWORD,
},
});Pair this with proxy rotation strategies so each session or request isn’t always from the same IP. See using proxies with Playwright, why residential proxies, avoid IP bans, and Proxy Checker. Best proxies for web scraping and Proxies.
Fingerprint and headers (look like a real browser)
Playwright sends browser-like headers by default. To further reduce fingerprinting:
- Keep a consistent viewport and user agent per session.
- Avoid switching devices (mobile/desktop) too frequently on the same account.
- Don’t disable too many features (cookies, JS, images) unless necessary.
You can inspect your headers with HTTP header checker and adjust. For high-security sites, consider:
- Using stable, realistic profiles (same UA + viewport + languages).
- Avoiding obviously automated patterns (e.g. requesting
/robots.txtthen hammering the site).
See preventing scraper fingerprinting, browser stealth techniques, anti-bot systems, and web scraping detection methods.
Behavior: delays, scrolling, and retries
Bots often request pages too fast or with no mouse/scroll. Add random delays between actions and, when it helps, simulate scroll or click so behaviour looks more human:
await page.goto(url, { waitUntil: 'networkidle' });
await page.waitForTimeout(1000 + Math.random() * 1500);
await page.mouse.move(200, 400, { steps: 10 });
await page.mouse.wheel(0, 600);
await page.waitForTimeout(800 + Math.random() * 1200);Combine this with:
- Max request rate per IP / per account.
- Backoff and retry when you see 429/5xx or captchas.
See web scraping without getting blocked, avoid IP bans, scraping dynamic websites, Playwright at scale, ultimate web scraping guide, and Scraping Test.
Further reading:
- Ultimate web scraping guide
- Best proxies for web scraping
- Residential proxies
- Proxy rotation
- Web scraping architecture
- Scraping data at scale
- Avoid IP bans
- Playwright web scraping
- Headless browser
- Bypass Cloudflare
- How websites detect scrapers
- Python web scraping guide
- Proxy pools
- Proxy Checker
- Scraping Test
- Proxy Rotator
- Robots Tester
- Ethical web scraping
- Web scraping legal
- Common web scraping challenges
- Web scraping without getting blocked
- Proxies
Next steps:\n\n1. Turn one of your existing Playwright scripts into a “stealth” version using the launch/context and delay examples above.\n2. Add residential proxies and proxy rotation, then watch how your block/ban rate changes.\n3. Use Proxy Checker, Scraping Test, and HTTP Header Checker to iterate on your setup.\n\nFor a broader picture, see ultimate web scraping guide, best proxies, and web scraping without getting blocked.