Overview
Diwalco helps families build a shared family tree, invite relatives to claim their place in it, and share memories privately. I'm the core engineer and majority contributor: the Express/Bun API, realtime chat, referrals and contests, offline-first client runtime, CI/CD, observability and release safety.
- Company
- Diwalco Limited (New Zealand), remote
- Role
- Founding Software Engineer, May 2025 - Present. Joined as the current platform was being built.
- Ownership
- Core engineer and majority contributor. 42 PRs authored, 40 merged; work peer-reviewed from Feb 2026.
- Product
- A private, collaborative family tree and family feed; a web app, installable as a PWA
- Surfaces
- Consumer web app (PWA), admin console, REST and realtime API
Stack
- Bun 1.4
- Turborepo
- Express 5
- Socket.IO
- Zod
- Drizzle ORM
- PostgreSQL 16
- Neo4j
- Next.js 16
- React 19
- TanStack Query
- SWR
- IndexedDB
- AWS
- Docker Compose
- GitHub Actions
- GHCR
- Trivy
- Sentry
- PostHog
Context
“One app for your family's past, present, and future.” You build a family tree for free starting from yourself, invite relatives who claim their own node (with admin approval), belong to several trees (your father's side, your mother's side), and share photos, milestones and memories in a private family feed. Trees have Admin, Moderator and Member roles, and users own their data.
The engineering shape: family relationships are a graph, but memberships, referrals, payouts and consent need relational integrity. Users are mostly on phones and patchy networks. Growth runs on invites, referrals and a milestone-based referral contest, which brings fraud and abuse pressure.
How to read the labels
Work is presented as it actually exists. Every claim on this page carries one of these labels.
- Shipped
- Merged to the production branch
- Pre-release
- Merged to the default branch, awaiting the next promotion to production
- Design
- Architecture and design complete; not yet implemented
What I owned
| Area | My role | Status |
|---|---|---|
| API (Express 5 on Bun): 96 versioned endpoints across 12 modules, shared response envelope, Zod validation | Majority contributor | Shipped |
| Auth: Cognito JWT/JWKS verification on HTTP and sockets, verified-token and user caches, phone OTP sign-in | Author | Shipped |
| Realtime chat (Socket.IO): authenticated sockets, membership-gated rooms, typing, read receipts, typed shared event contracts | Author (client and server) | Shipped |
| Memories feed: posts, multi-photo carousels, likes, threaded comments, share | Author | Shipped |
| Invites, referrals, milestone contests, auditable grand-prize draw, UPI prize claims | Author (admin contest config co-built) | Shipped |
| Offline-first client runtime: unified API clients, endpoint registry, user-scoped IndexedDB cache, service-worker rewrite | Author, on a teammate's PWA foundation | Shipped |
| Versioned policy consent | Author | Shipped |
| Policy evidence (append-only, hash-bound, release-aware) | Author | Shipped |
| CI/CD, containers, secrets, deploy and rollback | Owner of all pipeline, Docker and host scripts | Shipped |
| Test suite and CI contract gates | Introduced and own them; tests generated with AI, reviewed by me before merge | Shipped |
| Observability: Sentry in 3 apps, manual tracing on Bun, credential redaction and replay scrubbing | Author | Shipped |
| Bounded image pipeline: browser worker pool, server admission, WebP renditions | Author | Shipped |
In-app bug reporting with my npm package react-bug-report | Author | Shipped |
| Neo4j introduction, driver package, refactor of the cross-store transaction helper | Author / refactor | Shipped |
| PostgreSQL to Neo4j transactional-outbox projection | Architecture and design complete; not yet implemented | Design |
| AI-assisted family-tree creation | Designed and implemented | Pre-release |
Architecture
Client
Next.js 16 PWA (web)
SWR and TanStack Query, a user-scoped IndexedDB cache, Web Worker image renditions
HTTPS /api/v1 and Socket.IO
API
Express 5 API on Bun
Request pipeline: request-id → rate limiters → Cognito JWKS auth → Zod → services
Ordered cross-store commits
Data and services
PostgreSQL 16
System of record
Neo4j
Family graph
AWS S3
WebP renditions
Cognito, SES, Secrets Manager
Identity, email, secrets
Observability: Sentry across web and API, plus PostHog.
Pull request
Typecheck, 12 contract gates and tests
Multi-arch image build
BuildKit secret mounts keep secrets out of layers
Trivy gate
HIGH and CRITICAL findings stop the release
Scan-tagged digests
Only scanned digests become release tags
Release candidate
Resolved per commit and re-validated
Signed manifest
Ten files describe exactly what deploys
EC2 host, Docker Compose
Serialized deploy with atomic release and config switches
Health and readiness
Both datastores must answer before traffic moves
On failure: Automatic rollback. Phase-aware, and held to the same readiness bar
Deep dives
Two stores, one source of truth
ShippedDesign- Problem
- Family relationships need graph traversal; memberships, referrals and payouts need relational integrity.
- Approach
- PostgreSQL is the system of record: a 42-table schema with partial unique indexes for business invariants such as “one accepted referral per invitee” and “one DM per user pair”, plus CHECK constraints on money and state machines. Neo4j holds the family graph (people, trees and unions as nodes). I introduced Neo4j with a bounded, validated driver package and refactored the cross-store transaction helper. Writes commit in a fixed order, with rollback on any failure before commit.
- Honest boundary
- This is not an atomic distributed transaction; a narrow window remains between the two commits.
- Next step
- I architected and designed a PostgreSQL to Neo4j transactional-outbox projection: ordered graph projection, idempotent retries and a graph that can be rebuilt from PostgreSQL. The design is complete; it isn't implemented yet.
AI-assisted family-tree creation
Pre-release- Problem
- Typing a family tree node by node is tedious; people naturally describe their family.
- Approach
- The workflow takes a natural-language family description, uses an LLM only to interpret it into structured people and relationships, runs deterministic application validation, shows the result for user review and correction, and persists only the confirmed tree across PostgreSQL and Neo4j.
- Principle
- The model interprets; validation and the user decide what's written. Schema-valid output isn't the same as a correct family.
DescribePerson
The user writes their family in plain language
InterpretModel
An LLM turns it into structured people and relationships
ValidateCode
Deterministic application rules check the structure
ReviewPerson
The user corrects and confirms the tree
PersistCode
Only the confirmed tree is written, across PostgreSQL and Neo4j
Correctness under concurrency
Shipped- Contest enrolment race
- Concurrent requests read stale winner counts and could over-award a capped milestone. Fixed with a row lock (
SELECT … FOR UPDATE) inside the enrolment transaction. - Double referral attribution
- Phone verification did a read-then-write. Replaced with a single conditional
UPDATE … WHERE not-yet-verified RETURNING. - Referral completion stuck pending
- A nested call took a second pool connection while a transaction held one: a connection self-deadlock. The side effect now runs after commit.
- Read receipts
- The read cursor is monotonic (time plus a UUIDv7 tiebreak) and broadcasts only when it actually advances.
- Grand-prize draw
- Idempotent (row lock and one award per contest), fraud-flagged users excluded in SQL, deterministic candidate ordering, an injectable crypto RNG for tests, and a SHA-256 hash of the candidate set stored and audit-logged in the same transaction.
Identity and abuse resistance
Shipped- Verification
- Cognito JWT verification against JWKS with a pinned issuer,
token_useand client-ID binding, on both HTTP and the socket handshake. - Speed
- Bounded LRU caches for verified tokens and user rows, with reverse-index invalidation by user ID. Observed average authenticated request time fell from ~1.8 s to ~0.4 s (observed, not an SLO). Infrastructure outages (database, JWKS) surface as 500s instead of masquerading as 401 sign-in churn.
- Abuse controls
- 10 purpose-built rate limiters (global, chat, mutations, contest polling, OTP send keyed by destination, OTP verify, referral lookups, signup attribution, feedback, friend invites). Turnstile on OTP send, timing-safe internal-secret comparison, and unbiased cryptographic referral codes with collision retry.
- Hardening
- Turnstile action and hostname binding (blocks cross-site token replay), and a per-destination SMS limiter on the auth routes for toll-fraud protection beyond IP limits.
Release engineering with honest rollback
Shipped- Scan before tag
- Images are pushed to throwaway scan tags and gated by Trivy (HIGH/CRITICAL); only scanned digests become release tags. SBOM and provenance attestations; all 81 third-party Actions pinned by commit SHA; BuildKit secret mounts keep build-time secrets out of image layers.
- Promote, don't rebuild
- Production reuses the pre-release server image only if a diff over every build input is empty; otherwise it rebuilds and rescans.
- Deploy
- A release candidate is resolved per commit, re-validated and described by a signed 10-file manifest. The host deploy is serialized by a lock, with atomic release and config “generation” switches, then health and readiness checks, then phase-aware automatic rollback held to the same readiness bar. Database migrations are deliberately forward-only, so schema changes must be backward compatible.
- Secrets and containers
- AWS Secrets Manager syncs to 15 file-backed Docker secrets, written with symlink refusal, descriptor-based (TOCTOU-safe) validation and an atomic generation flip. Containers run with a read-only root filesystem, tmpfs, all capabilities dropped,
no-new-privileges, a non-root user, CPU, memory and PID limits, and digest-pinned base images. - Builds and hosting
- Native amd64 and arm64 builds (replacing QEMU emulation within a day), a private-network deploy path, GHCR retention cleanup, and a Free-plan artifact-quota guard.
CI that tests its own checks
Shipped- From zero
- Before March 2026 the repo had no tests and no CI. I introduced both: 0 → 160 test files (measured 2026-10-02). The tests are generated with AI and every one is reviewed by me before merge; the gates below are what keep them, and the rest of the code, honest.
- Contract gates
- 3 → 12 gates: API-contract drift, toolchain and dependency-catalog parity, env example, smoke and schema parity, secrets-registry parity, release-payload and config-path parity, and public-asset budgets. Migrations are applied to a fresh PostgreSQL in CI and fail on schema drift.
- Checks that check
- A meta-gate fails CI when any
check:*script exists that isn't wired into the pipeline. Gate self-tests copy real files, mutate one line and assert the gate fails; vacuous-assertion guards stop a scan from passing by matching nothing. - Why
- Partly to catch AI-generated code drifting from architectural rules. I also wrote a rulebook for AI coding agents whose rules map to these gates.
Observability on a runtime the SDK doesn't fully support
Shipped- Manual tracing
- Sentry runs across web, admin and API. Bun reimplements
node:http, so OpenTelemetry auto-instrumentation yields no Express transactions; I added manual request tracing with browser trace propagation. - Source maps you can trust
- Bun's own debug IDs collided with Sentry's, so the build strips, injects and verifies them. The uploaded source map is checked byte-for-byte against the
server.jsin the scanned image. - Sampling by measurement
- Health checks, session polling and static assets were ~83% of observed trace volume (development observation), so they're excluded from tracing. A 15-second health check alone would otherwise create ~5,760 transactions per container per day (calculated). Session Replay runs only on errors, with all text and inputs masked.
- Redaction
- One shared credential redactor applied at four SDK hooks, replacing three drifted copies, plus a transport that scrubs replay envelopes.
A bounded image pipeline
Shipped- Browser
- Reads dimensions from the header without decoding, resizes during decode in a module Web Worker on
OffscreenCanvas, and uses a worker pool capped at 3 (2.85× at 3 vs 2.91× at 4), sized down by device memory. 2.80× faster multi-photo conversion in a controlled desktop Chromium benchmark (6 × 12 MP JPEGs: 4,183 ms → 1,495 ms). - Server
- Capacity is reserved before any byte is buffered (global, per-user and bytes-in-flight ceilings), with abort-aware multipart parsing, a separate lane for heavy images, one WebP rendition (2,160 px edge) re-encoded server-side, and orphaned uploads cleaned up even on aborted requests and during shutdown. Web and API share one upload contract.
Offline-first without leaking data between users
Shipped- The service worker never caches
/api, because URL-keyed caches carry no user identity. API reads are cached in IndexedDB under a hashed user scope with TTL and byte budgets, sensitive paths excluded. Writes fail fast offline, and every cache is wiped at each auth boundary. One endpoint registry (78 endpoints) and three allowed transports are enforced in CI.
Graceful shutdown on Bun
Shipped- Shutdown releases sockets, database pools and in-flight upload cleanups with a deadline, and readiness checks both datastores.
- Finding
- On Bun, open WebSocket connections must be force-closed in the same tick as
io.close(), or the HTTP server never finishes closing.
Consent you can prove
Shipped- Versioned consent
- Required and grace-period enforcement.
- Evidence model
- Append-only. Database triggers block edits and deletes; each acceptance is bound by a compound foreign key to the exact content hash the user saw; policy versions get deterministic UUIDv8 IDs (SHA-256); policies publish before new code takes traffic and re-publish on rollback; marketing consent is stored separately from mandatory policy.
Talking to families
- I led early product conversations with individuals and families: introduced and demonstrated the product, gathered feedback, and helped people sign up and get started.
Smaller wins
- Replaced ~580 lines of hand-rolled feedback UI with my open-source
react-bug-report(shipped). - Reinstated versioned database migrations with a fresh-database CI check (shipped).
- Migrated auth from a third-party auth provider to AWS Cognito (shipped, Aug 2025).
- Built the v1 marketing site from scaffold to public launch in 2025 (since redesigned by a teammate).
Timeline
- May 2025Joined; v1 marketing site
- Jun 2025First API: PostgreSQL schema, Express routes, Zod validation, invites with open links and join requests, S3 uploads
- Jul 2025Monorepo restructure; Neo4j added alongside PostgreSQL; shared types and logger packages
- Aug 2025Auth moved to AWS Cognito; marketing site goes live
- Sep - Nov 2025Invite dashboard; Memories feed (likes, threaded comments, carousels); profiles with photo cropping
- Jan 2026Realtime chat
- Feb 2026Referral system, contest enrolment, phone OTP sign-in, referral fraud checks
- Mar 2026Offline-first API runtime, response envelopes, versioned consent, first CI workflow and contract gates
- Apr - Jul 2026Milestone contests, invite analytics, ownership and membership integrity, graceful shutdown, configurable milestone contests, grand-prize draw
- Aug 2026Production CI/CD (env contracts, images, scan gates, signed deploys, rollback); Sentry in 3 apps; bug reporting
- Sep 2026 onwardBounded image pipeline, native multi-arch builds, an artifact-quota guard, security hardening and the consent-evidence stack; the AI-assisted tree builder in pre-release
Numbers
Measured 2026-10-01. Each figure says what it counts; none of them are user, revenue or uptime claims.
- PostgreSQL tables (108 indexes, 14 CHECK constraints, 72 foreign keys)
- 42
- Shipped
- Versioned REST endpoints / route modules
- 96 / 12
- Shipped
- Purpose-built API rate limiters
- 10
- Shipped
- Automated test files, generated with AI and reviewed by me (measured 2026-10-02)
- 0 → 160
- Shipped
- CI contract gates, plus a meta-gate
- 3 → 12
- Shipped
- GitHub Actions workflow files; 81 of 81 third-party actions pinned by commit SHA
- 6
- Shipped
- File-backed Docker secrets synced from AWS Secrets Manager
- 15
- Shipped
- Apps instrumented with Sentry
- 3
- Shipped
- Observed trace volume that was health checks, polling and static assets (development observation), now excluded
- ~83%
- Shipped
- Observed average authenticated API request time after token and user caching
- ~1.8 s → ~0.4 s
- Shipped
- Multi-photo conversion throughput, controlled desktop benchmark
- 2.80×
- Shipped
- Lines of custom feedback UI replaced by
react-bug-report - ~580
- Shipped
What I'd do differently
- Put tests and CI contract gates in from the first commit. They arrived ten months in, so they had to fit around decisions that had already set.
- Decide how two datastores stay consistent before adding the second. Neo4j arrived with ordered commits and rollback; the outbox design that closes the remaining window came later.