Scale Stack Resource
Pre-Scale Architecture Checklist
27 failure modes we catch in every architecture review — before they turn into an outage, a surprise bill, or a rebuild.
This is the same checklist we run across every architecture review we do at Scale Stack. It is written for founders and engineering leads who feel their system is about to hit a scale, reliability, or cost wall — and want a concrete way to figure out where.
How to use it
- Score each item: green (covered), amber (partial), red (missing).
- Anything red in Reliability or Security is a day-one fix. Everything else sequences behind it.
- If you have more than five ambers in a single section, you have a systemic gap — not a to-do list.
1 · Reliability
The difference between a platform people trust and one they work around is almost always here. Every item is a place where silent data loss or double-writes usually start.
No dead-letter queue (or DLQ not monitored) for webhook ingestion
Failed Stripe / HubSpot / Meta events disappear. You hear about it from the customer, not your dashboard.
Retries without idempotency keys
Creates duplicate charges, duplicate contacts, or double fulfilments under load or partial failures.
Inbound webhook timeouts shorter than downstream processing
Your upstream marks the webhook failed and retries; you process it twice — or not at all.
No alerting on payment / event processing backlog growth
Queues back up quietly for hours. You only find out when reconciliation breaks at end of day.
Firestore / Postgres multi-doc writes without transactional boundaries
Partial writes leave the system in an inconsistent state that no retry can recover from.
2 · Scalability
These are the patterns that look fine at 10k users and turn into a full rebuild at 100k. Catching them early is a 10× cheaper than catching them on the billing invoice.
Read-heavy patterns with no caching or denormalized read model
Every page load hits your primary store. Latency and cost grow linearly with traffic.
Hot document / hot key patterns without sharding
Leaderboards, counters, and popular resources throttle Firestore / Postgres at the worst moment.
N+1 reads inside cloud function loops
Silently explodes your invocation and read billing at scale. Usually invisible in local testing.
Single-region setup for multi-region traffic
Global users see 200–500 ms overhead on every request. Conversion and retention pay for it.
Synchronous, chained cloud functions where an event bus belongs
Each hop adds cold start, cost, and a new failure mode. Pub/Sub or a queue is almost always cheaper and faster.
3 · Data & pipelines
Operational and analytical workloads sharing a pipeline is the single most common root cause we see behind mysterious production incidents.
Analytics events flowing through the same path as operational writes
A slow BigQuery insert should never be able to back-pressure a user-facing checkout.
No schema versioning on events
A well-meaning producer change silently breaks three downstream consumers on deploy day.
Firestore / Supabase used as a queue
Missing ordering and at-least-once guarantees means you eventually lose or reorder events.
No backfill / replay path for missed or late events
One bad deploy becomes three days of manual SQL to reconstruct state.
PII leaking into analytics or logs without a redaction layer
A one-line bug becomes a privacy incident. The fix belongs at the pipeline, not in each producer.
4 · AI & automation
AI features are now a line item on the P&L. These are the guardrails that keep them from becoming the biggest one.
LLM calls without cost / latency budgets or per-tenant caps
One abusive tenant or one infinite loop and the month's AI spend lands in a single afternoon.
RAG pipeline with no evaluation set
You ship a model or prompt change with no way to detect quality regressions until users complain.
Prompt logic living in client code
Impossible to iterate safely. Every change is a full release cycle instead of a config flip.
No guardrails on tool-use agents for destructive actions
Refunds, deletes, and sends need explicit approvals, dry-run modes, and audit trails — not vibes.
Automation triggers in Zapier / Make without source of truth in code
Business logic ends up distributed across tools no one owns. Onboarding new engineers gets painful fast.
5 · Security & cost
The checklist items here are not glamorous. They are also the ones that show up on the post-mortem when something goes very wrong.
Service accounts with overly broad IAM
Write or admin where read-only is enough. The blast radius of any compromise scales with this.
Secrets in .env.local committed to git or shared in chat
Rotate now, audit with gitleaks, and move to a managed secrets store before the next hire joins.
No spend alerts on Firebase / Google Cloud / Vercel
Surprise bills are almost always preventable with a 30-minute budget + alert setup.
No cost attribution per feature / tenant
You cannot tell which product line is profitable. Pricing and roadmap decisions get made blind.
CORS / App Check / RLS misconfigured
Public keys end up allowing full-table reads. This is the #1 preventable data exposure we see.
6 · Delivery & team
Reliability is a team property, not a code property. Two last items that quietly decide how fast you can fix everything above.
No staging environment with realistic data volume
Performance and race-condition bugs only surface in production, at the worst possible time.
CI without type-checks or tests on infra-touching code
Functions, Firestore rules, Supabase migrations, Terraform — all high-blast-radius, all under-tested.
Want us to walk this checklist against your system?
Free 30-min architecture call. We score each item together, then send you a written async review — yours to keep whether we work together or not.
Call 03 7073 2727 to talk it through.
Tip: press ⌘+P to save a printable copy.