Can AI Crawlers Read Your Site Without JavaScript?

In this article

1. Why JavaScript is a problem for AI search
2. Which AI crawlers render JavaScript?
3. What AI crawlers typically miss
4. How to test what AI crawlers see (in 5 minutes)
5. How to fix it, from best to good enough
6. A note on common platforms
7. Frequently asked questions

Why JavaScript is a problem for AI search

When a browser loads a modern website, two things happen. The server sends HTML, then the browser runs JavaScript that can add text, images, prices and links to the page. A person sees the finished result. A crawler that doesn't run JavaScript sees only the first step.

Google solved this years ago. Googlebot queues pages for rendering in a headless Chrome browser and indexes the result. Most AI crawlers skipped that step. Rendering every page costs far more than fetching HTML, and AI companies crawl at huge scale. So if your product descriptions, pricing table or case study text arrive through JavaScript, AI search engines may be indexing an almost empty page.

That's why our LLM Readiness Check measures how much of your homepage text is in the raw HTML. It's one of the most common reasons a site that ranks well on Google gets little attention in ChatGPT or Perplexity.

Which AI crawlers render JavaScript?

The most detailed public test is Vercel and MERJ's December 2024 analysis of crawler traffic across Vercel's network. It found that "none of the major AI crawlers currently render JavaScript". ChatGPT's crawler fetched JavaScript files in 11.5% of requests and Claude's in 23.8%, but neither ran them. As of mid-2026, none of these companies has documented adding JavaScript rendering to its crawlers.

Microsoft has published far less than Google about how Bingbot, which grounds Copilot, handles JavaScript. Treat server-rendered HTML as the safe default for every engine, Copilot included.

What AI crawlers typically miss

• Single-page apps. Sites built as client-side React, Vue or Angular apps often send a near-empty <div id="root"></div> and build everything in the browser.
• Tabs, accordions and "read more" sections that fetch their content only when clicked. Content that's in the HTML and simply hidden with CSS is fine.
• Prices, specs and inventory loaded from an API after the page appears. These are often the facts buyers ask AI about.
• Reviews and testimonials loaded by third-party widgets.
• Structured data added by Google Tag Manager. Google can read it because it renders pages. AI crawlers can't.
• Titles, meta descriptions and canonical tags set by JavaScript instead of in the HTML.
• Links built with JavaScript click handlers instead of real <a href> elements, which stops crawlers from discovering your other pages.

How to test what AI crawlers see (in 5 minutes)

1. View the source. In Chrome, type view-source: before the URL, or press Ctrl+U (Cmd+Option+U on Mac). This is the raw HTML, the same thing GPTBot gets. Search it (Ctrl+F) for a sentence from the visible page, a price or a product spec.
2. Turn off JavaScript. Open DevTools, press Ctrl+Shift+P (Cmd+Shift+P on Mac), type "Disable JavaScript" and reload. Whatever disappears is invisible to most AI crawlers.
3. Fetch it like a crawler. From a terminal:

   curl -s -A "OAI-SearchBot" https://www.example.com/pricing | grep -i "your key phrase"

   No match means the phrase isn't in the HTML the crawler receives.
4. Compare word counts. Copy the visible page text into a word counter, then do the same with the text from the raw HTML. If the raw HTML has less than half the words, you have a rendering problem.
5. Run the free LLM Readiness Check for your homepage, or use the Chrome extension on any page. It reports the share of text that's readable without JavaScript.

Test your most important templates, not just the homepage: a product or service page, an industry page, pricing, and a case study.

How to fix it, from best to good enough

1. Server-side rendering or static generation (best)

Have the server send complete HTML. Modern frameworks support this directly: Next.js and Remix for React, Nuxt for Vue, Angular's server rendering, SvelteKit, and Astro. Static generation builds the HTML in advance, which suits marketing pages that don't change by the minute. Every visitor and every crawler gets the same complete page, and it usually loads faster too.

2. Render the important parts on the server

If a full migration isn't realistic, server-render the content AI needs to quote: headings, body copy, product specs, prices, FAQs and structured data. Keep interactive extras such as calculators, filters and chat widgets on the client.

3. Prerendering

A prerendering service or build step loads your pages in a headless browser and stores the finished HTML, which is then served to crawlers or to everyone. It's a practical fix for an existing single-page app. If you serve prerendered HTML only to bots, which Google calls "dynamic rendering", treat it as a stopgap. Google describes it as a workaround, not a recommended long-term solution, and the bot and human versions can drift apart.

4. Quick wins on any platform

• Move JSON-LD structured data out of Google Tag Manager and into the page template.
• Put tab and accordion content in the HTML and show or hide it with CSS, instead of fetching it on click.
• Make sure navigation uses real <a href="..."> links.
• Set the <title>, meta description and canonical tag in the server-rendered HTML.
• Publish an llms.txt and an XML sitemap that point to pages that work without JavaScript.

A note on common platforms

Traditional CMS platforms such as WordPress, HubSpot CMS and Webflow generate pages on the server, so the main text of a page is usually in the raw HTML. Problems tend to come from add-ons: page-builder widgets, sliders, review plugins, and content embedded from other services. Headless setups and custom JavaScript apps are where whole pages most often go missing, especially when they were built as a client-side app to save time. Whatever the platform, the five-minute test above tells you the truth.

Frequently asked questions

Free tool

How much of your homepage can AI read?

The free LLM Readiness Check measures how much of your homepage text is in the raw HTML, the part AI crawlers can read, and tests 15 AI crawlers' access.

Run the free check