Skip to content

Checklist

AI-built app risk checklist: 40 checks before customers rely on it

We built fast with AI tools. What is likely to break once real customers rely on it?

About software audits
Checks
40
Sections
7
Last updated

The AI-built app risk checklist is 40 yes-or-no checks for apps built fast with AI coding tools, from Cursor and Claude Code projects to vibe-coded apps made in Lovable, Bolt, or Replit. It covers what a demo never tests: exposed secrets, missing authorization, unvalidated input, unmanaged migrations, silent errors, thin tests, risky dependencies, and uncapped AI API spend.

It is for founders and technical leads at seed and Series A startups whose first version is live, or close to it. Use it as a security checklist for AI-generated code and as a wider check of whether the app is production ready, before customers, investors, or an enterprise security questionnaire ask. Building fast with AI tools was a sensible trade. This list covers what comes next.

How should you use the AI-built app risk checklist?

#

Run the checklist in one sitting with the code and the provider dashboards open, answer every check yes or no, and fix what fails by severity rather than by count.

  • Work through it with someone who can read the code and open the hosting, database, and AI provider dashboards. Most checks take a few minutes each.
  • Answer yes only when you have checked it today. “I think so” counts as no, and so does a check nobody on the team can answer.
  • Send test requests against staging with test accounts, never against another customer's real data.
  • Count the no answers in each section, then use the guide after the checks to decide what to fix first.

For the reasoning and the commands behind many of these checks, read our guide to the risks of AI-generated code in production.

Section 010 of 6 checked

Are secrets and API keys out of the browser and git history?

  • Next.js and Vite write any NEXT_PUBLIC_ or VITE_ variable that client code uses into the JavaScript every visitor downloads, so a secret key there is already public. Search the built files, such as .next/static or dist, for the key's value, because the variable name is gone after the build.
  • Deleting the line leaves the key in history, where anyone with a clone of the repository can find it with git log -p. Only rotation makes the old value useless.
  • A call made from the browser needs the key in the browser, where anyone can copy it and spend on your account. Our teardown of an exposed API key shows how to move the call behind an authenticated server route.
  • Supabase's secret and service_role keys act through a Postgres role with the bypassrls attribute, and a database URL usually connects as the role that owns the tables, which row-level security does not restrict by default. Either one in client code lets a visitor read and change every table.
  • Shared keys let a test script or a laptop send real emails, charge real cards, or delete production rows. Separate keys also let you rotate one environment without taking down the others.
  • A coding agent may paste a key into a file to get a feature working on the first try, and a scanner flags it before merge. Fixing a pull request takes minutes, while a key that reaches a shared branch has to be rotated.

Section 020 of 6 checked

Do sign-in and authorization hold up on the server?

  • API routes and server actions are public HTTP endpoints, so anyone can call them with curl, signed in or not. An endpoint that skips the check answers whoever finds its URL in the browser's network panel.
  • Reaching another user's records by changing an ID is a form of broken access control, which ranks first in the OWASP Top 10:2025. When one endpoint fails this test, assume every record in that table is exposed.
  • A query that filters by an ownerId from the request trusts the caller to tell the truth. Read the ID from the session in one shared helper that every query uses, so the rule does not depend on each endpoint getting it right.
  • Supabase's row-level security guide warns that a table in an exposed schema without RLS is readable and writable by any role with a grant on it. RLS with a using (true) policy still lets every user that policy covers read every row.
  • A regular account that types /admin or calls an admin endpoint directly can reach user lists, refunds, or impersonation. Role checks belong in the same server code that does the work.
  • Without limits, a script can try thousands of passwords against one account or send thousands of reset emails through your provider. A flood of reset emails can also hurt your sending reputation, which pushes real emails into spam.

Section 030 of 5 checked

Does the server validate input and handle data safely?

  • Anyone can skip your form and send role: 'admin', a negative quantity, or a 10 MB name. A strict schema, such as Zod's z.strictObject, refuses fields and values the endpoint does not expect.
  • Generated code sometimes drops to a raw query for one tricky filter, and one string-built query is enough for SQL injection. Search for $queryRawUnsafe, sql.raw, and template strings passed to query(.
  • Rendering user content as HTML lets one user run script in another user's session, which is cross-site scripting. If you need rich text, sanitize it with an allowlist library such as DOMPurify.
  • A public bucket with guessable paths exposes every invoice, ID scan, or contract anyone uploaded. Size limits also stop one upload from filling storage or exhausting server memory.
  • An unverified Stripe endpoint accepts a forged checkout.session.completed event from anyone, which can mark an unpaid order as paid. Stripe signs every event so your server can check it with the endpoint's signing secret.

Section 040 of 6 checked

Are the database, migrations, and backups ready for real data?

  • Changes clicked into a database dashboard or pushed with prisma db push leave production with a schema no migration records. The next generated migration then conflicts with what is really there.
  • This proves the migration history is complete. Diff pg_dump --schema-only of both databases, and treat any difference as drift that will surprise the next deploy.
  • Until you restore one, you do not know whether backups work or how long recovery takes. While you are in the settings, check whether point-in-time recovery is on for your plan and how far back it reaches.
  • A rename in one deploy breaks the old code still serving requests during rollout and makes rollback impossible. Our guide to safe releases and rollbacks walks through the sequence.
  • Code that checks for an existing email before inserting still lets two simultaneous requests create duplicate accounts or double charges. A unique constraint makes the database refuse the second one.
  • A query that returns every row is instant with 500 rows in a demo and times out at 500,000. The pg_stat_statements extension in Postgres lets you rank queries by total execution time, so you know which to fix first.

Section 050 of 6 checked

Are errors handled, logged, and seen by someone?

  • Without one, you learn about a broken signup from a customer email, often days later. Source maps make each error point at the original file and line instead of minified code.
  • Error tracking that nobody watches changes nothing. Send alerts for the paths that make money to a channel someone reads, and prove it works by throwing a deliberate error in staging.
  • Code written only for the success case leaves requests hanging when a provider slows down, until the platform kills them. A timeout with a clear message or a queued retry keeps their outage from becoming yours.
  • Raw errors in API responses show an attacker your framework, table names, and query shapes. Customers only need to know what happened and what to do next.
  • A request ID lets you follow one failure across the server, the job queue, and the provider. Tokens in logs turn your logging vendor into one more place secrets can leak.
  • A job that fails silently can stop invoices, emails, or data syncs for days before anyone notices. A failed-jobs table or dead-letter queue with an alert makes the failure visible the same day.

Section 060 of 6 checked

Do tests and dependency checks catch problems before release?

  • These few paths make or lose money, so they get tests first. Five to ten Playwright tests on them are a better first investment than a coverage target spread across helper functions.
  • Authorization breaks quietly, because the app still works for whoever is testing it. Automating the cross-user request from the sign-in section turns the next regression into a failed build instead of a customer report.
  • Tests that only run on a laptop stop running in the first busy week. A required check makes every change, AI-assisted or not, clear the same bar before it reaches production.
  • A model asked to write tests for existing code can lock that code's bugs in, for example by asserting that any signed-in user may delete any invoice. Read the assertions on those paths yourself.
  • Published vulnerabilities are the easiest holes to find, because the advisories describe them. Small regular updates cost far less than one large upgrade after a year of drift.
  • Models sometimes suggest package names that do not exist, and attackers register those names so the install runs their code. A package that installs is not proof, so check its repository and publish history, and run npx knip to list packages nothing imports.

Section 070 of 5 checked

Are AI API costs capped and tracked?

  • A public route that forwards requests to OpenAI or Anthropic is an open proxy that anyone with curl can spend through. A limit such as 20 requests an hour per user, answered with a 429, caps the damage.
  • OpenAI's spend limits guide states that spend alerts do not enforce a cap, while a hard limit for the organization or a single project stops traffic with a 429 error. Check what each other provider offers, and make the app show a clear message when a limit is reached.
  • Without caps, one user can paste a 200-page document into a chat box and get a long answer back, and you pay for both by the token. Set max_tokens in Anthropic's Messages API, max_completion_tokens in OpenAI's Chat Completions API, or max_output_tokens in its Responses API, from what each feature needs.
  • Provider dashboards show the total bill, not which feature or customer drove it. Logging the usage fields each API response returns lets you spot abuse within a day and price plans on real numbers.
  • A loop that retries until the output parses, or an agent that keeps calling tools, bills every attempt and can run all night on one request. The official OpenAI and Anthropic SDKs already retry twice by default, so count those attempts before adding your own loop, and give every agent a hard step limit.

0 of 40 checked

How do you tell if your AI-built app is production ready?

#

An AI-built app is not ready for customers to rely on while any check on the urgent list below fails, whatever the rest of the score. Read your results by severity, not by total. One urgent no outweighs five elsewhere, because those failures can be exploited or run up a bill today, and leaked data or a large bill is hard to undo.

Which failures should you fix this week?

#
  • A secret key or database URL in the browser bundle, or a key from git history that has not been rotated.
  • An endpoint that returns another user's data, an admin route a regular account can reach, or an exposed table without row-level security.
  • A route that calls a paid AI API and is missing sign-in, a per-user rate limit, or a hard spend limit.
  • A webhook handler that acts on payloads without checking the signature.
  • A critical npm audit advisory in your framework or another package that runs on your server.

Rotate exposed keys first, since rotation takes minutes and makes a leaked value worthless. Then close the authorization gaps, because each one exposes every customer's data at once.

What can wait a few weeks?

#

Other gaps in error tracking, tests, migrations, and dependency upkeep seldom cause harm on their own. They decide whether later fixes ship safely, so schedule them this month, starting with error tracking, which shows whether the urgent fixes broke anything. The technical debt triage checklist helps rank them once the urgent fixes ship.

When does an outside review make sense?

#

Get an outside review when your team cannot fix the urgent failures this week, when nobody can answer a whole section, or when a fundraise or enterprise deal depends on the answers. A Pixelact software audit is a fixed package from $2,500: about 3 to 5 days of review, a written report that ranks risks and fixes, a build path with rough costs, and a walkthrough call. If releases already break every week, software rescue starts with a rescue audit at the same price, which ends in a fix-or-rebuild call.

To see the format first, request an audit and ask for the sample report we made from one of our own products.

Checklist updates

Get an email when this checklist changes

We revise checklists when tools or advice change. Leave your email and we'll tell you when this one is updated. You'll also get the monthly newsletter, and every email has an unsubscribe link.

By subscribing, you agree to get the Pixelact newsletter by email. Every email has an unsubscribe link. Read the privacy notice.

FAQ

Questions about this checklist

Straight answers on what it covers, who should run it, and what to do with the results.

Passing every check means the gaps that commonly appear in apps built fast are closed. It does not prove the app is secure, compliant, or ready for heavy traffic, and it does not replace a penetration test. Treat a clean result as a baseline and rerun it after each major feature. The startup security checklist covers what customers and investors ask about, and the performance and scalability checklist covers growth.

For part of it, yes. Provider spend limits, environment separation, backup settings, and alert routing need only dashboard access. The bundle search, the cross-user request, input validation, and the migration rebuild need someone who can read the code, run a build, and send a request with curl. If nobody on the team can do those, they are where an outside reviewer adds the most.

Often not. Most failures here, such as unrotated keys, missing rate limits, or a schema outside version control, can be fixed in place without changing how the product works. A rebuild is worth weighing when the data model has no tenant boundary or every fix breaks something else. Our guide to deciding whether to rebuild or refactor sets out how to tell which case you are in.

Yes. Keep the speed and add the review that production needs. A person who knows the system reviews every change to sign-in, payments, personal data, or the schema. CI runs tests, a secret scanner, and a dependency audit on every pull request. Pull requests stay small enough to read. Our AI code review workflow describes one way to set this up.

A Pixelact software audit costs from $2,500 as a fixed fee paid upfront, so you know the full cost before the review starts. It covers the app's code, architecture, infrastructure, and workflow. If the audit leads to a build with us within 60 days, 50% of the audit fee is credited against that build.

Next step

Need help closing the gaps?

Tell us what you're building and what the checklist turned up, or book a call if you're not sure where to start.

On the discovery call

  1. 01What prompted this now, and where things stand
  2. 02What a good outcome looks like
  3. 03Whether we're a fit, and what a next step could be

45 minutes. No slides, and no long questionnaire.