Search-engine indexing

Make a hosted app crawlable: prerender routes at build time from your own build script, ship robots.txt and a sitemap, and serve on your own domain.

By default a hosted app is a single-page app: the same index.html shell is served for every route and the page is rendered in the browser. That is the right shape for an internal dashboard, but a crawler that doesn’t execute JavaScript sees an empty document — no title, no copy, no links. Most non-Google crawlers, including the ones behind social link previews, fall in that category.

Two things gate indexing, and an app needs both:

1. Real HTML per URL, which you get by prerendering at build time from your app’s own build script.
2. Your own domain. Cargo-owned hostnames — an app’s default hostname and every per-deployment preview URL — answer with X-Robots-Tag: noindex, which keeps them out of search results. Attaching a custom domain is the opt-in.

This page is about apps. A worker serves whatever headers its code sets, so if a worker returns HTML you want indexed, that’s yours to control.

Apps built on CargoRefineApp from @cargo-ai/app-sdk require a Cargo login, and anything behind a login can’t be indexed at all. A public app skips CargoRefineApp and renders its own tree — nothing at the platform level requires authentication.

Supported frameworks

Cargo automatically detects your frontend framework from package.json and configures the build accordingly:

Framework	Output directory	Env var prefix
Vite	dist	VITE_
Create React App	build	REACT_APP_
Gatsby	public	GATSBY_
Astro	dist	PUBLIC_

If no framework is detected, Cargo defaults to Vite conventions (dist output, VITE_ prefix).

SSR-first frameworks

Frameworks like Next.js, Remix, SvelteKit, and Nuxt are server-side rendering frameworks by default, and Cargo doesn’t detect them: an app using one gets the no-framework defaults, so the build output is read from dist/. Cargo hosts static sites only, so the app needs a build script that produces a static export and leaves it in dist/:

• Next.js: set output: "export" in next.config.ts. Adding trailingSlash: true exports each page as <route>/index.html, which is the layout routing resolves first; without it the export writes <route>.html, which resolves too. The export lands in out/, so move it — for example "build": "next build && rm -rf dist && mv out dist".
• Nuxt: use nuxt generate instead of nuxt build, and make sure its output ends up in dist/.
• SvelteKit: use @sveltejs/adapter-static with its pages and assets options set to dist.
• Remix: use a static adapter configured to write to dist/.

A build that finishes without creating dist/ fails with an error naming the directory. Cargo still recognizes these frameworks’ env var prefixes (NEXT_PUBLIC_, NUXT_PUBLIC_, PUBLIC_), so environment variables work as expected.

Prerender from your build script

On deploy, Cargo runs npm ci --ignore-scripts and then your app’s build script whenever it does more than vite build, so a prerender step in that script runs server-side as part of the build:

package.json

{
  "scripts": {
    "build": "vite build && vite-react-ssg build"
  }
}

There is nothing to opt into beyond the script itself. A build script that is only a plain Vite build — vite build, or create-vite’s tsc && vite build and tsc -b && vite build — keeps the default build, exactly as if it weren’t declared, and the build log says so. Apps without a build script get the detected framework’s default build command (npx vite build for Vite and for undetected frameworks, npx astro build for Astro, and so on).

Your script also runs with the platform and app environment variables in its process environment, so a prerender step running as a plain Node script can read them; the default build reads them from .env.production instead.

Two rules hold either way:

• Output has to land in the detected framework’s output directory (dist/ unless the table above says otherwise), and index.html must exist. The build fails with a user-facing error otherwise, since the app’s routes fall back to that shell.
• Your build script owns the whole build. Cargo doesn’t run the framework build before or after it, so the script needs to produce the client bundle too — not just the prerendered HTML.

Platform environment variables are available with each supported prefix — a Vite app reads VITE_CARGO_API_URL, a Next.js app reads NEXT_PUBLIC_CARGO_API_URL, and so on. Your own environment variables must also use a recognized prefix to be exposed to the build.

If your build script does more than vite build — a different --mode, a custom output directory, a lint step — that now runs on deploy where it previously didn’t. A failed build leaves the currently live deployment serving, and the build log shows what broke.

Routing

A path that names a file (/logo.png, /about.html) is always served as that file, and / serves index.html. For a path without an extension, Cargo serves the first of these that your build produced:

Order	File	Status	Written by
1	pricing/index.html	200	Next.js with trailingSlash: true, Astro, Gatsby
2	pricing.html	200	Next.js without trailingSlash
3	404.html	404	your own not-found page
4	index.html	200	the SPA shell, so the browser’s router picks the page

/pricing and /pricing/ resolve identically. So a prerendered site gets clean URLs and a client-rendered one keeps its shell, with nothing to configure and nothing to declare. There’s no setting: it follows the files, which means a preview URL serves pages exactly the way the live site will, and promoting a deployment switches layouts the moment its files are live.

Two things follow from the order:

• Ship a 404.html to get a real 404. Without one, an unknown path serves your shell with a 200 — a soft 404. That’s what a client-rendered app wants; a prerendered site usually doesn’t.
• An SPA that ships a 404.html gets it for client routes, because step 3 comes before step 4. If your router owns unknown paths, don’t ship one.

A file that genuinely doesn’t exist (/logo.png on a site without one) answers 404 from the storage layer and never reaches your app.

If your app renders a “not found” view client-side on a path that resolved to step 4, set <meta name="robots" content="noindex"> on it from the client — the response itself is a 200, so nothing else tells a crawler to skip it.

Ship robots.txt and a sitemap

Anything in Vite’s public/ directory is copied verbatim into the build and served at the root. Nothing extra is needed:

public/robots.txt
public/sitemap.xml

public/robots.txt

User-agent: *
Allow: /

Sitemap: https://www.example.com/sitemap.xml

Both are served with Cache-Control: no-cache so they revalidate on every request and a promote takes effect immediately. Hashed assets under assets/ are the only files cached long-term.

Put the metadata in the prerendered head

Cargo serves index.html exactly as your build produced it — no tags are injected. Every prerendered page needs its own title, description, canonical URL, and Open Graph tags (Next.js writes these from each page’s metadata export):

about/index.html

<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>About — Example</title>
    <meta name="description" content="What we do and who we do it for." />
    <link rel="canonical" href="https://www.example.com/about/" />
    <meta property="og:title" content="About — Example" />
    <meta
      property="og:description"
      content="What we do and who we do it for."
    />
    <meta property="og:url" content="https://www.example.com/about/" />
    <meta property="og:image" content="https://www.example.com/og.png" />
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="/src/main.tsx"></script>
  </body>
</html>

Serve it on your own domain

An app’s default hostname is a subdomain Cargo owns, and Cargo serves those with X-Robots-Tag: noindex. So a custom domain isn’t a nice-to-have here — it’s what makes the app indexable at all, and it puts the content on a domain whose authority is yours.

Declare it on the app and let the deploy attach it. If Cargo also holds the domain, declare that too and merge the records the attach returns straight into its zone:

infra/resources.ts

import { defineApp, defineDomain } from "@cargo-ai/cdk";

const site = defineApp("website", {
  path: "./apps/website",
  domains: ["www.example.com"],
});

defineDomain("example.com", {
  // Omit `adopt` to buy it: registering charges credits and isn't refundable,
  // so the plan's `+ create domain:example.com` line is the purchase approval.
  adopt: true,
  dnsRecords: [site.domainRecords],
  redirectUrl: "https://www.example.com",
});

site.domainRecords is every record the hostname needs — the _cargo-verify TXT, the certificate’s validation CNAME, and the www CNAME to Cargo — so one deploy attaches the domain and publishes those records. dnsRecords manages only what it lists: the records are merged into the live zone, so mail records and anything added in the UI stay put, and there’s no need to restate them here. The one exception is a record that can’t share a name with a declared CNAME — the parking A a registrar leaves at www — since DNS gives that name to the CNAME alone; the plan shows it as a deletion before you confirm. Until every hostname is attached — after a --draft deploy, which attaches none — the zone is left as it is rather than published without the records still to come. The bare domain can’t point at the app, since the app has no fixed IP address; redirectUrl forwards it to www.

Removing a hostname from domains leaves it attached — the deploy never takes a live site offline on its own, and its records stay in domainRecords so the zone keeps serving it. Detach it explicitly. Detaching stops the app listing its records, but nothing a deploy publishes for an app is a deploy’s to delete, so they stay in the zone; plan names them, and the Cargo UI is where they go. For a domain registered elsewhere, attach through the API instead and add the records at your DNS provider:

curl -X POST https://api.getcargo.io/v1/hosting/custom-domains \
  -H "authorization: Bearer $CARGO_API_TOKEN" \
  -H "content-type: application/json" \
  -d '{ "kind": "app", "appUuid": "<uuid>", "hostname": "www.example.com" }'

The response carries the DNS records to add: a _cargo-verify.<hostname> TXT record that proves you control the domain, validation records for the certificate, and a cnameTarget to point the hostname at. Add them, then POST /v1/hosting/custom-domains/<uuid>/refresh-status until the status reaches active. Nothing is served on the hostname until the TXT record resolves. The TXT value is unique to this attachment, so a record left over from an earlier one won’t match, and you can delete it once the domain is active.

Hostnames need at least three DNS labels, so attach www.example.com rather than the bare example.com.

Every deployment also gets a permanent preview URL (deployment-<uuid>.app.getcargo.run) serving an identical copy of the site. Those carry the same noindex as the default hostname, so they can’t compete with your domain as duplicates. Point <link rel="canonical"> at your custom domain anyway: it’s what consolidates a URL that gets shared or linked from more than one host.

Get it discovered

Indexable HTML on your own domain makes the site eligible, and that’s all it does. Cargo doesn’t announce it: no search engine is pinged, no sitemap is submitted for you. A new domain that nothing links to can stay uncrawled indefinitely, so one of these has to happen:

• Submit the sitemap. In Google Search Console, add the domain as a property, verify it with the DNS TXT record, and submit https://www.example.com/sitemap.xml. URL Inspection also requests indexing for one URL at a time, which is worth doing for the homepage. Bing Webmaster Tools is the equivalent if Bing matters to you.
• Get one link from a page that’s already crawled. A marketing site, a docs page, a public repo — anything already in the index gives a crawler a path to the new domain.

Then wait. Discovery takes days to weeks, and crawled is not the same as indexed — Google decides, and thin pages often stay out. curl -sSI https://www.example.com/ confirms the response carries no X-Robots-Tag, but only Search Console’s page indexing report answers whether a URL is actually in the index.

Checklist

• Public app that doesn’t require a Cargo login
• A build script in package.json that prerenders (so it does more than vite build)
• One <route>/index.html or <route>.html per page, every route listed in the sitemap
• public/robots.txt allowing crawlers, pointing at the sitemap
• Per-page title, description, canonical, and Open Graph tags in the prerendered head
• Custom domain attached and active — required, since the default hostname is noindex — and every canonical pointing at it
• Sitemap submitted in Google Search Console, since nothing indexes a site it hasn’t discovered