Why Does My Puppeteer Script Keep Breaking?

Puppeteer navigation timeouts, headless detection by Cloudflare and DataDome, CAPTCHA walls, selector drift after site updates, Chromium version rot, and the memory math at 10k+ monthly runs: a diagnostic for each failure with an error-to-fix routing table.

Puppeteer scripts fail in a characteristic sequence: first the timeouts, then the CAPTCHA wall, then the silent selector break that shipped bad data before anyone noticed. If you're maintaining a Puppeteer script against an enterprise portal, you will meet most of the failures below eventually. Each section opens with the problem the way you'd type it into a chatbot, gives the short-term fix, and names the structural cause.

For the cross-tool view, start with the anchor diagnostic. Selenium and Playwright versions of this page: here and here.

Error-to-fix routing table

"Puppeteer TimeoutError: Navigation timeout of 30000 ms exceeded"

The classic. page.goto or page.waitForNavigation never reached the state you asked for. Three causes, in descending frequency:

• The portal is slow. Enterprise portals hydrate in stages; 10-second loads are normal. Raise the budget for the slow pages instead of globally.
• Your `waitUntil` doesn't match the page. networkidle0 never settles on a page with polling or websockets; the navigation "times out" while working perfectly. Wait on the thing you actually need: page.waitForResponse for the XHR that carries your data, or page.waitForSelector for the element you'll touch next.
• A redirect went somewhere you didn't expect. Auth hops and session expiry send navigation into a login flow with none of your expected elements. Log the final URL on every timeout.

Notice the second fix. When you wait on page.waitForResponse, you've identified the backend endpoint that carries the data. The page render around it is packaging.

"Puppeteer headless detected / Cloudflare or DataDome blocking"

Puppeteer's default fingerprint is the best-catalogued in the industry: HeadlessChrome in the user agent, navigator.webdriver set, missing codec and plugin surfaces, distinctive canvas and WebGL output. Detection vendors cross-reference those tells with IP reputation and behavior. puppeteer-extra-plugin-stealth patches the obvious ones, headed mode in a virtual display patches more, residential proxies patch the IP, and every rung of that ladder costs more and decays faster, because the detection side ships updates continuously and the plugin follows.

The strategic question is the same as for Selenium and Playwright: if the target is a portal you or your customer legitimately log into, the disguise is the wrong layer. The portal's frontend calls backend endpoints over authenticated HTTP. Automation that speaks to those endpoints directly, which is what Supergood builds and maintains, has no fingerprint to hide, because there's no browser to detect. That's the core of Supergood vs browser automation.

"Puppeteer CAPTCHA wall"

When the fingerprint score crosses a threshold, the portal serves a CAPTCHA, and unattended automation stops. Solving services exist; they add cost, latency, and a terms-of-service problem, and portals rotate providers precisely to break them. A CAPTCHA is the portal saying "prove you're a human" to a script whose whole job is to not be one. The durable answer is to stop presenting as a browser at all: authenticated network-layer requests don't get served CAPTCHAs in the first place.

Page layout changes: the silent one

"My Puppeteer script runs green but the data is wrong." Selector drift in Puppeteer often doesn't throw. page.$ returns null, optional chaining swallows it, and the script keeps going with a missing field, or worse, a selector that now matches a different element, reading the wrong column into the wrong record. You pay for the run, then pay again to find and clean up what it wrote.

Defenses, in order: assert on every element you depend on (fail loudly, never optionally); prefer IDs and data-* attributes over positional CSS chains; and snapshot-test critical pages so a deploy-time diff warns you before the nightly run does. Those lower the frequency and raise the visibility. They don't change the underlying number: an actively developed portal redesigns meaningfully 1 to 3 times a year, each break costing 4 to 16 dev-hours. The recovery playbook is "re-inspect and patch"; the avoidance playbook is an integration with no DOM dependency at all, which is the trade examined in RPA vs API integration.

Latency and memory at scale

Every Puppeteer page is backed by a full Chromium process: a DOM tree, a JS heap, a render pipeline, hundreds of MB of RAM. Two consequences at volume:

• Concurrency is memory-bounded. A 16GB box realistically runs a few dozen concurrent sessions; past that you're queueing, recycling processes to fight leaks, and adding boxes.
• Every run pays the render tax. Full page load, JavaScript execution, dynamic-content waits. A 30-second login-navigate-extract workflow is typical; the backend equivalent is about a second.

Hosted browser sessions floor out around $0.15 each; a realistic multi-step workflow lands near $1.00 per completed call once retries are counted, and complex multi-step success rates as low as 9 to 19% multiply the real cost per completed action. At 10,000 runs a month the floor alone is ~$1,500/mo of session time for work a backend API performs for a fraction of that. Full model: the true cost of browser automation past 10,000 calls a month.

"My script broke after npm update"

Puppeteer pins a specific Chromium build per release; upgrading one without the other breaks launch or changes rendering behavior, and system dependencies (fonts, libs, sandbox flags) rot underneath long-running boxes. Pin both, read release notes, rebuild images regularly. Then note the shape of the problem: your integration has a runtime dependency on a browser binary neither you nor the portal controls. An API client's dependency surface is an HTTP library.

The worked example: 10,000 runs a month

One Puppeteer workflow, 10,000 runs a month, assumptions stated: 30s per run, hosted sessions at the $0.15 floor, two redesign breaks a year at 10 hours, monthly babysitting at 2 hours.

If the portal offers an official API covering the workflow, use it. Where it doesn't, Supergood generates and maintains REST and MCP APIs at the network layer the portal's own frontend uses: no Chromium to feed, no selectors to drift, no fingerprint to defend.

Related reading

• Why your enterprise automation keeps breaking — the cross-tool anchor diagnostic
• Why does my Selenium scraper keep breaking? — the WebDriver sibling
• Why does my Playwright automation keep failing? — the modern-framework sibling
• Supergood vs browser automation — the API alternative explained
• The true cost of browser automation past 10,000 calls a month — the cost model behind these tables

FAQ

Why does my Puppeteer script keep timing out? The page never reached the state you asked for: slow portal, hydration after XHRs, or a waitUntil condition (networkidle0 with polling) that can never settle. Wait on specific responses or selectors. Timeouts that start after a site deploy are usually drift, not slowness.

Why is Puppeteer detected as a bot? Headless Chromium's default fingerprint is the best-catalogued there is. Stealth plugins patch the obvious tells and decay as detection updates. For portals you legitimately use, authenticated network-layer access ends the arms race.

What do I do when the website changes and my selectors break? Re-inspect, update, and add assertions on every critical element, because Puppeteer drift often fails silently. Budget 4 to 16 dev-hours per break, 1 to 3 redesigns a year. Recurrence on a workflow that matters is the signal to remove the UI dependency.

How much memory does Puppeteer use at scale? A full Chromium process per page, hundreds of MB each; a 16GB box runs a few dozen concurrent sessions. At 10k+ monthly runs you're managing queueing and process recycling that an API integration simply doesn't have.

Should I keep patching or replace it? Patch one-offs at low volume. Replace the approach when drift recurs, detection arrives, or volume passes a few thousand runs a month. Removing the browser removes the failure class.