Serve content in HTML, not JavaScript, for AI crawlers
Most AI crawlers don't run JavaScript. If your site renders client-side only, they see a blank page.
Why it matters
GPTBot, ClaudeBot, and PerplexityBot all fetch raw HTML — they do not execute JavaScript like Googlebot does. If your homepage is a 200-byte `<div id="root"></div>` shell with everything hydrated client-side, AI search engines literally see no content. This check is worth 25 points and is the #1 reason React SPAs get a low Crawlable score.
How to fix it
1. View source on your most important page
Right-click → View Page Source (not Inspect). If you can't find your headline, body copy, or product description in the raw HTML, you have an SSR problem.
2. Migrate to SSR or SSG
If you're on Vite + React Router SPA, move to TanStack Start, Next.js, Remix, or Astro. If you're on Next.js Pages Router with `useEffect` data fetching, switch to `getServerSideProps` or App Router server components. The goal: real text in the initial HTML response.
3. Pre-render the highest-traffic routes first
You don't have to migrate the whole app at once. Use SSG for marketing pages (`/`, `/pricing`, `/blog/*`) and keep the dashboard SPA. Marketing is where AI traffic lands anyway.
4. Test with curl
Run `curl -A 'GPTBot/1.0' https://yourdomain.com | grep -i 'your headline'`. If grep finds nothing, AI crawlers find nothing.
FAQ
- Will prerendering or a snapshot service fix this?
- Sort of — services like Prerender.io serve a static HTML snapshot to bots. It works but adds a moving part and most snapshot services don't recognize all AI bot user-agents. SSR is more reliable.
- What about React's `use client` directives?
- Components marked `'use client'` still server-render their initial HTML in Next.js App Router and TanStack Start. The directive controls where hydration happens, not whether SSR runs.
See your own score
Run a free Crawlable scan to find every check that needs fixing on your site — not just this one.
Check my site