# AI-built app risk checklist: 40 checks before customers rely on it

URL: https://pixelactstudio.com/resources/ai-built-app-risk-checklist

> AI-built app risk checklist: 40 yes-or-no checks on secrets, authorization, input, migrations, monitoring, tests, and AI spend, plus what to fix first.

The AI-built app risk checklist is 40 yes-or-no checks for apps built fast with AI coding tools, from Cursor and Claude Code projects to vibe-coded apps made in Lovable, Bolt, or Replit. It covers what a demo never tests: exposed secrets, missing authorization, unvalidated input, unmanaged migrations, silent errors, thin tests, risky dependencies, and uncapped AI API spend.

It is for founders and technical leads at seed and Series A startups whose first version is live, or close to it. Use it as a security checklist for AI-generated code and as a wider check of whether the app is production ready, before customers, investors, or an enterprise security questionnaire ask. Building fast with AI tools was a sensible trade. This list covers what comes next.

## How should you use the AI-built app risk checklist?

Run the checklist in one sitting with the code and the provider dashboards open, answer every check yes or no, and fix what fails by severity rather than by count.

- Work through it with someone who can read the code and open the hosting, database, and AI provider dashboards. Most checks take a few minutes each.
- Answer yes only when you have checked it today. “I think so” counts as no, and so does a check nobody on the team can answer.
- Send test requests against staging with test accounts, never against another customer's real data.
- Count the no answers in each section, then use the guide after the checks to decide what to fix first.

For the reasoning and the commands behind many of these checks, read our guide to [the risks of AI-generated code in production](https://pixelactstudio.com/blog/ai-generated-code-risks).

## Are secrets and API keys out of the browser and git history?

- [ ] Is your production JavaScript bundle free of secret keys, such as values starting with `sk_live_`, `sk-`, or `sb_secret_`? Next.js and Vite write any `NEXT_PUBLIC_` or `VITE_` variable that client code uses into the JavaScript every visitor downloads, so a secret key there is already public. Search the built files, such as `.next/static` or `dist`, for the key's value, because the variable name is gone after the build.
- [ ] Has every key that ever appeared in git history been rotated at the provider, not just deleted from the code? Deleting the line leaves the key in history, where anyone with a clone of the repository can find it with `git log -p`. Only rotation makes the old value useless.
- [ ] Do calls that need a secret key, such as those to OpenAI, Anthropic, Stripe, or your email provider, run on the server rather than in the browser? A call made from the browser needs the key in the browser, where anyone can copy it and spend on your account. Our [teardown of an exposed API key](https://pixelactstudio.com/blog/exposed-api-keys) shows how to move the call behind an authenticated server route.
- [ ] Are your database connection string and any Supabase secret or `service_role` key used only in server code? Supabase's secret and `service_role` keys act through a Postgres role with the `bypassrls` attribute, and a database URL usually connects as the role that owns the tables, which row-level security does not restrict by default. Either one in client code lets a visitor read and change every table.
- [ ] Do production, staging, and local development use separate API keys, databases, and webhook secrets? Shared keys let a test script or a laptop send real emails, charge real cards, or delete production rows. Separate keys also let you rotate one environment without taking down the others.
- [ ] Does CI run a secret scanner, such as Gitleaks or TruffleHog, on every pull request? A coding agent may paste a key into a file to get a feature working on the first try, and a scanner flags it before merge. Fixing a pull request takes minutes, while a key that reaches a shared branch has to be rotated.

## Do sign-in and authorization hold up on the server?

- [ ] Does every API route and server action check the session on the server before it reads or writes data? API routes and server actions are public HTTP endpoints, so anyone can call them with curl, signed in or not. An endpoint that skips the check answers whoever finds its URL in the browser's network panel.
- [ ] If you sign in as user A and resend a request with user B's record ID, do you get a 404 or 403 instead of B's data? Reaching another user's records by changing an ID is a form of broken access control, which ranks first in the [OWASP Top 10:2025](https://top10.owasp.org/2025/A01_2025-Broken_Access_Control). When one endpoint fails this test, assume every record in that table is exposed.
- [ ] Are queries scoped by the user or organization ID taken from the session, never by an ID sent in the request body or URL? A query that filters by an `ownerId` from the request trusts the caller to tell the truth. Read the ID from the session in one shared helper that every query uses, so the rule does not depend on each endpoint getting it right.
- [ ] Is row-level security turned on for every table in an exposed schema, with policies narrower than `using (true)`? [Supabase's row-level security guide](https://supabase.com/docs/guides/database/postgres/row-level-security) warns that a table in an exposed schema without RLS is readable and writable by any role with a grant on it. RLS with a `using (true)` policy still lets every user that policy covers read every row.
- [ ] Do admin pages and admin endpoints check the user's role on the server, not only in the navigation? A regular account that types `/admin` or calls an admin endpoint directly can reach user lists, refunds, or impersonation. Role checks belong in the same server code that does the work.
- [ ] Are sign-in, sign-up, and password reset rate-limited per IP address and per account? Without limits, a script can try thousands of passwords against one account or send thousands of reset emails through your provider. A flood of reset emails can also hurt your sending reputation, which pushes real emails into spam.

## Does the server validate input and handle data safely?

- [ ] Does the server validate every request body against a schema that sets limits and rejects unknown fields? Anyone can skip your form and send `role: 'admin'`, a negative quantity, or a 10 MB name. A strict schema, such as Zod's `z.strictObject`, refuses fields and values the endpoint does not expect.
- [ ] Do all database queries use parameters or the ORM's query builder, with no SQL assembled from strings? Generated code sometimes drops to a raw query for one tricky filter, and one string-built query is enough for SQL injection. Search for `$queryRawUnsafe`, `sql.raw`, and template strings passed to `query(`.
- [ ] Is text that users can edit rendered as plain text, with no `dangerouslySetInnerHTML` or `v-html` on it? Rendering user content as HTML lets one user run script in another user's session, which is cross-site scripting. If you need rich text, sanitize it with an allowlist library such as DOMPurify.
- [ ] Do file uploads check type and size on the server and land in a private bucket served through short-lived signed URLs? A public bucket with guessable paths exposes every invoice, ID scan, or contract anyone uploaded. Size limits also stop one upload from filling storage or exhausting server memory.
- [ ] Do webhook handlers verify the provider's signature before acting on the payload? An unverified Stripe endpoint accepts a forged `checkout.session.completed` event from anyone, which can mark an unpaid order as paid. Stripe signs every event so your server can check it with the endpoint's signing secret.

## Are the database, migrations, and backups ready for real data?

- [ ] Is every schema change stored in the repository as a migration and applied by CI or the deploy, not by hand? Changes clicked into a database dashboard or pushed with `prisma db push` leave production with a schema no migration records. The next generated migration then conflicts with what is really there.
- [ ] Can you build an empty database from the migrations alone and get the same schema as production? This proves the migration history is complete. Diff `pg_dump --schema-only` of both databases, and treat any difference as drift that will surprise the next deploy.
- [ ] Have you restored a production backup into a separate database in the last 90 days? Until you restore one, you do not know whether backups work or how long recovery takes. While you are in the settings, check whether point-in-time recovery is on for your plan and how far back it reaches.
- [ ] Are destructive changes, such as dropping or renaming a column, split into separate expand and contract releases? A rename in one deploy breaks the old code still serving requests during rollout and makes rollback impossible. Our guide to [safe releases and rollbacks](https://pixelactstudio.com/blog/safe-releases-and-rollbacks) walks through the sequence.
- [ ] Do unique, not-null, and foreign key constraints in the database enforce the rules your code assumes? Code that checks for an existing email before inserting still lets two simultaneous requests create duplicate accounts or double charges. A unique constraint makes the database refuse the second one.
- [ ] Does every list endpoint have a maximum page size, backed by an index on the columns it filters and sorts by? A query that returns every row is instant with 500 rows in a demo and times out at 500,000. The `pg_stat_statements` extension in Postgres lets you rank queries by total execution time, so you know which to fix first.

## Are errors handled, logged, and seen by someone?

- [ ] Does an error tracker such as Sentry capture exceptions from both the server and the browser, with source maps uploaded? Without one, you learn about a broken signup from a customer email, often days later. Source maps make each error point at the original file and line instead of minified code.
- [ ] If signup or checkout broke an hour ago, would a named person already have an alert? Error tracking that nobody watches changes nothing. Send alerts for the paths that make money to a channel someone reads, and prove it works by throwing a deliberate error in staging.
- [ ] Do calls to payment, email, and AI providers have timeouts and a handled failure path? Code written only for the success case leaves requests hanging when a provider slows down, until the platform kills them. A timeout with a clear message or a queued retry keeps their outage from becoming yours.
- [ ] Do users see a plain error message while stack traces, SQL, and internal IDs stay in your logs? Raw errors in API responses show an attacker your framework, table names, and query shapes. Customers only need to know what happened and what to do next.
- [ ] Are logs structured, tagged with a request ID, and free of passwords, tokens, and personal data? A request ID lets you follow one failure across the server, the job queue, and the provider. Tokens in logs turn your logging vendor into one more place secrets can leak.
- [ ] Do background jobs and webhook handlers retry a limited number of times, then record the failure somewhere that alerts someone? A job that fails silently can stop invoices, emails, or data syncs for days before anyone notices. A failed-jobs table or dead-letter queue with an alert makes the failure visible the same day.

## Do tests and dependency checks catch problems before release?

- [ ] Do end-to-end tests cover signup, sign-in, the action customers pay for, and checkout? These few paths make or lose money, so they get tests first. Five to ten Playwright tests on them are a better first investment than a coverage target spread across helper functions.
- [ ] Does an automated test sign in as one user, request another user's record, and fail the build if the data comes back? Authorization breaks quietly, because the app still works for whoever is testing it. Automating the cross-user request from the sign-in section turns the next regression into a failed build instead of a customer report.
- [ ] Do tests, type checks, and linting run in CI on every pull request and block the merge when they fail? Tests that only run on a laptop stop running in the first busy week. A required check makes every change, AI-assisted or not, clear the same bar before it reaches production.
- [ ] Do the generated tests on your critical paths assert what the code should do, not just what it does today? A model asked to write tests for existing code can lock that code's bugs in, for example by asserting that any signed-in user may delete any invoice. Read the assertions on those paths yourself.
- [ ] Does `npm audit --omit=dev` report no high or critical vulnerabilities, with Renovate or Dependabot opening regular update pull requests? Published vulnerabilities are the easiest holes to find, because the advisories describe them. Small regular updates cost far less than one large upgrade after a year of drift.
- [ ] Has someone confirmed that every dependency is the package you meant to install, with a real repository and a release history? Models sometimes suggest package names that do not exist, and attackers register those names so the install runs their code. A package that installs is not proof, so check its repository and publish history, and run `npx knip` to list packages nothing imports.

## Are AI API costs capped and tracked?

- [ ] Does every route that calls a model require a signed-in user and enforce a per-user rate limit? A public route that forwards requests to OpenAI or Anthropic is an open proxy that anyone with curl can spend through. A limit such as 20 requests an hour per user, answered with a 429, caps the damage.
- [ ] Is a hard spend limit set at each AI provider, per project where possible, rather than only an email alert? [OpenAI's spend limits guide](https://developers.openai.com/api/docs/guides/spend-limits) states that spend alerts do not enforce a cap, while a hard limit for the organization or a single project stops traffic with a 429 error. Check what each other provider offers, and make the app show a clear message when a limit is reached.
- [ ] Does each AI feature cap the input length it accepts and the number of output tokens the model may return? Without caps, one user can paste a 200-page document into a chat box and get a long answer back, and you pay for both by the token. Set `max_tokens` in Anthropic's Messages API, `max_completion_tokens` in OpenAI's Chat Completions API, or `max_output_tokens` in its Responses API, from what each feature needs.
- [ ] Do you record token usage and cost per request, per user, and per feature? Provider dashboards show the total bill, not which feature or customer drove it. Logging the usage fields each API response returns lets you spot abuse within a day and price plans on real numbers.
- [ ] Do model retries stop after a fixed number of attempts with backoff, and do agent loops have a step limit? A loop that retries until the output parses, or an agent that keeps calling tools, bills every attempt and can run all night on one request. The official OpenAI and Anthropic SDKs already retry twice by default, so count those attempts before adding your own loop, and give every agent a hard step limit.

## How do you tell if your AI-built app is production ready?

An AI-built app is not ready for customers to rely on while any check on the urgent list below fails, whatever the rest of the score. Read your results by severity, not by total. One urgent no outweighs five elsewhere, because those failures can be exploited or run up a bill today, and leaked data or a large bill is hard to undo.

### Which failures should you fix this week?

- A secret key or database URL in the browser bundle, or a key from git history that has not been rotated.
- An endpoint that returns another user's data, an admin route a regular account can reach, or an exposed table without row-level security.
- A route that calls a paid AI API and is missing sign-in, a per-user rate limit, or a hard spend limit.
- A webhook handler that acts on payloads without checking the signature.
- A critical `npm audit` advisory in your framework or another package that runs on your server.

Rotate exposed keys first, since rotation takes minutes and makes a leaked value worthless. Then close the authorization gaps, because each one exposes every customer's data at once.

### What can wait a few weeks?

Other gaps in error tracking, tests, migrations, and dependency upkeep seldom cause harm on their own. They decide whether later fixes ship safely, so schedule them this month, starting with error tracking, which shows whether the urgent fixes broke anything. The [technical debt triage checklist](https://pixelactstudio.com/resources/technical-debt-triage-checklist) helps rank them once the urgent fixes ship.

### When does an outside review make sense?

Get an outside review when your team cannot fix the urgent failures this week, when nobody can answer a whole section, or when a fundraise or enterprise deal depends on the answers. A [Pixelact software audit](https://pixelactstudio.com/services/software-audits) is a fixed package from $2,500: about 3 to 5 days of review, a written report that ranks risks and fixes, a build path with rough costs, and a walkthrough call. If releases already break every week, [software rescue](https://pixelactstudio.com/services/software-rescue) starts with a rescue audit at the same price, which ends in a fix-or-rebuild call.

To see the format first, [request an audit](https://pixelactstudio.com/request-an-audit) and ask for the sample report we made from one of our own products.

## Frequently asked questions

### Is my AI-built app production ready if it passes every check?

Passing every check means the gaps that commonly appear in apps built fast are closed. It does not prove the app is secure, compliant, or ready for heavy traffic, and it does not replace a penetration test. Treat a clean result as a baseline and rerun it after each major feature. The [startup security checklist](https://pixelactstudio.com/resources/startup-security-checklist) covers what customers and investors ask about, and the [performance and scalability checklist](https://pixelactstudio.com/resources/performance-scalability-checklist) covers growth.

### Do I need an engineer to complete this checklist?

For part of it, yes. Provider spend limits, environment separation, backup settings, and alert routing need only dashboard access. The bundle search, the cross-user request, input validation, and the migration rebuild need someone who can read the code, run a build, and send a request with curl. If nobody on the team can do those, they are where an outside reviewer adds the most.

### Do a lot of failed checks mean we should rebuild the app?

Often not. Most failures here, such as unrotated keys, missing rate limits, or a schema outside version control, can be fixed in place without changing how the product works. A rebuild is worth weighing when the data model has no tenant boundary or every fix breaks something else. Our guide to [deciding whether to rebuild or refactor](https://pixelactstudio.com/blog/rescue-or-rebuild) sets out how to tell which case you are in.

### Should we keep building with AI coding tools?

Yes. Keep the speed and add the review that production needs. A person who knows the system reviews every change to sign-in, payments, personal data, or the schema. CI runs tests, a secret scanner, and a dependency audit on every pull request. Pull requests stay small enough to read. Our [AI code review workflow](https://pixelactstudio.com/blog/ai-code-review-workflow) describes one way to set this up.

### How much does an outside review of an AI-built app cost?

A Pixelact software audit costs from $2,500 as a fixed fee paid upfront, so you know the full cost before the review starts. It covers the app's code, architecture, infrastructure, and workflow. If the audit leads to a build with us within 60 days, 50% of the audit fee is credited against that build.