LeadPool
Self-hosted, enterprise-grade platform for selling lead data to B2B clients.
- Language
- TypeScript
- Stars
- 0
- Last updated
From the README
- Pools — one per data type (auto warranty, medical, solar, …), each with its own admin-defined columns.
- Imports — CSV/XLSX, millions of rows, streamed and deduplicated by phone/email identity.
- Clients — their own portal: browse pools as products, pick an age tier (1 day … 120+ days), filter, see live availability + price, order, download CSV or Excel.
- Mixed orders — several age tiers in one order, split by percentage of a total ("50% fresh · 30% this month · 20% aged") or by exact lead counts; each tier is filled from its own age window, priced on its own and refunded on its own if short.
- Repeat orders — any order (one tier or a mix, with filters) placed automatically every day, on chosen weekdays or monthly, at a time in the client's own zone; paid from the balance, sent to their email / WhatsApp / Teams / cloud; skipped (never charged) when the balance is short and paused after 3 skips.
- Lead returns — clients send back bad leads within a window after delivery, up to a share of each order; staff approve per lead, approved leads are credited at the price paid and taken off sale. One switch turns the feature on or off for everyone (Settings → Lead returns), and it can be switched off per client.
- Sortable, filterable tables — click any column heading to sort (again to reverse, a third time to clear); the filter button next to it narrows by text, a pick-list, a number / money range or a date range (Today, 7 / 30 / 90 days, or from–to). Big lists (orders, returns, wallet history, audit log, a pool's leads) sort and filter in the database across every record, not just the page on screen; each table remembers its view while you move around.
- Hard rule — a client never receives the same lead, or the same person (any matching phone or email), twice.
- Suppression — clients upload their own lists; staff maintain a global DNC applied to everyone.
- Money — prepaid balances in micro-dollars, holds at order time, automatic refunds, append-only ledger.
- Approval flow — per client: auto-deliver instantly, or send each request to the admin queue.
- Two-factor sign-in — authenticator-app codes (TOTP) + single-use recovery codes for staff and client users; admins can require it for all staff.
- Reports — revenue, leads, orders, fill rate and price per lead for any period (vs. the previous one), by client, pool and lead age; export as a multi-sheet Excel workbook or daily CSV.
- Email notifications — order delivered / declined / unfilled, funds added, low balance (clients); approval needed, import finished (staff). Transactional outbox with retries; each user can mute them.
- Delivery channels — send order files to clients by Email, WhatsApp, Microsoft Teams, OneDrive or Dropbox: staff connect each once, set per-client destinations, send from any order or automatically on delivery; clients can manage their own destinations in the portal.
- Two looks — Modern (frosted clay, light/dark) or Classic (flat enterprise CRM), menu on the side or top, full or compact width — per browser, in both apps.
Architecture
apps/api Fastify 5 REST API (OpenAPI docs at /api/docs in dev)
apps/worker pg-boss jobs: imports, fulfillment, CSV exports, suppression, index builds, email,
channel sends (packages/db/src/channels), maintenance
(Excel copies of an order are built by the API on first request, then cached)
apps/admin Staff console (Vite + React 19 + TanStack Router/Query + Tailwind 4) → :5173
apps/portal Client portal (same stack and design system) → :5174
packages/db SQL migrations, Drizzle schema, fulfillment engine, wallet, exports, scripts
packages/shared Zod API contract, money/format helpers, identity normalisation, permissions
packages/ui "Frosted clay" design system shared by both apps
PostgreSQL 18 is the only infrastructure dependency (the job queue also lives in Postgres). Leads are LIST-partitioned per pool; order picking walks an index freshest-first in bounded pages, so cost depends on the order size — not on how many leads a pool holds.
Quick start — everything runs in Docker (Windows, macOS, Linux)
The only thing the machine needs is Docker Desktop (or Docker Engine with Compose 2.22+). The database, migrations, API, worker, admin console and client portal all run in containers, so the same folder runs the same way on any computer.
cp .env.example .env # Put 64 random hex characters in IDENTITY_PEPPER — for example from: docker run --rm node:24-alpine node -e "console.log(require('crypto').randomBytes(32).toString('hex'))" docker compose up -d --build # database, migrations, API, worker, admin, portal docker compose exec -w /app api pnpm db:seed # first time only: demo data (~40 s) docker compose watch # leave running while you work: saved files sync in, apps reload
| App | URL | Demo login (password ChangeMe!2026) |
|---|---|---|
| Admin console | http://localhost:5173 | admin@leadpool.local (super admin) · maya@leadpool.local (manager) · ops@leadpool.local (viewer) |
| Client portal | http://localhost:5174 | jordan@apexauto.example (auto-deliver) · dana@harborhealth.example (approval flow) · chris@blueriver.example |
The API (http://localhost:4000) and the database (port 5432) are reachable from this computer only.
How live editing works: docker compose watch copies every file you save into the containers — the API
and worker restart, the admin and portal hot-reload in the browser. Changing pnpm-lock.yaml (a new
dependency) rebuilds the image automatically. The containers keep their own Linux node_modules; nothing
from the host's is used. After changing docker-compose.yml or the Dockerfile, start with
docker compose up -d --build — a plain up -d recreates the containers from the last built image, which
leaves out everything synced in since then.
Everyday commands
| Command | What it does |
|---|---|
docker compose up -d --build |
Start or update everything (rebuilds only what changed; migrations run first) |
docker compose watch |
Live code sync while you edit |
docker compose ps · docker compose logs -f api worker |
Status · follow logs |
docker compose restart api |
Restart one service |
docker compose down |
Stop everything — data stays (the pgdata volume and ./storage) |
docker compose down -v |
Stop and delete the database |
Tools, run inside the stack
exec api … runs in the API container, which holds every package; -w /app runs from the repo root.
| Command | What it does |
|---|---|
docker compose exec -w /app api pnpm db:seed [--reset] |
Demo data: 3 staff, 4 pools, ~200k leads, 5 clients, 30 days of real orders (--reset wipes first) |
docker compose exec -w /app api pnpm db:migrate |
Apply pending SQL migrations (packages/db/migrations; also automatic on up) |
docker compose exec -w /app api pnpm typecheck |
TypeScript across the monorepo |
docker compose exec -w /app api pnpm test |
Unit tests + fulfillment concurrency tests (creates a throwaway leadpool_test DB) |
docker compose exec -w /app api pnpm lint |
Biome lint + format check |
docker compose exec api pnpm smoke |
End-to-end HTTP check against the running stack (103 assertions: archiving + restoring pool columns and tiers, import, orders, mixed orders, repeat orders, lead returns and their on/off switches, table sorting + filtering, dedup, CSV + Excel exports, suppression, approvals, emails, reports, money — each run imports its own fresh leads and tops up the demo client; expects the demo client Apex Auto Protect on instant delivery and lead returns switched on in Settings) |
docker compose exec api pnpm smoke:mfa |
Two-factor end-to-end check (76 assertions; uses throwaway users and cleans up) |
docker compose exec api pnpm smoke:channels |
Delivery-channels check (40 assertions: channel save/test, secrecy, client destinations, manual + automatic sends through the Email channel in file mode, portal access and permissions; restores what it touched) |
docker compose exec api node --import tsx scripts/create-admin.ts --email … --name … --password … |
Bootstrap / reset a super admin — also clears their 2FA (break-glass) |
docker compose exec -w /app/packages/db api pnpm bench |
EXPLAIN + timings for the pick/preview queries |
docker compose exec postgres psql -U leadpool |
SQL shell |
Testing from other devices (phones, other laptops on the same Wi-Fi/LAN): open
http://<this-computer's-IP>:5173 (admin) or :5174 (portal) — find the IP with ipconfig (Windows) /
ipconfig getifaddr en0 (macOS); on Windows the computer's name works too. The apps reach the API through the
dev-server proxy. If another device can't connect, allow Docker (or TCP 5173–5174) through the firewall for
the network you're on. Anyone on that network can reach the apps, so change the demo passwords before
testing on a shared network.
Move it to another computer
Everything is in three places: the project folder, .env (it holds IDENTITY_PEPPER — the data's
phone/email keys only match with the same value, so it must travel with the data) and the data (the
database plus ./storage).
# old computer — export the database into the project folder docker compose exec -T postgres pg_dump -U leadpool -d leadpool -Fc -f /tmp/leadpool.dump docker compose cp postgres:/tmp/leadpool.dump ./leadpool.dump # copy the project folder (with .env, storage/ and leadpool.dump) to the new computer, then there: docker compose up -d postgres docker compose cp ./leadpool.dump postgres:/tmp/leadpool.dump docker compose exec -T postgres pg_restore -U leadpool -d leadpool --no-owner /tmp/leadpool.dump docker compose up -d --build
The same dump restores into the production stack (docker-compose.prod.yml, service postgres) when you
go live.
Without Docker
Node 24 LTS + pnpm (corepack enable): pnpm install, then either docker compose up -d postgres or
pnpm db:local (embedded Postgres 18 in ./.pgdata), then pnpm db:seed and pnpm dev. Every tool command
above works as pnpm … from the repo root. Don't run these apps on the host while the Docker apps are up
(same ports). Moving data from the embedded database into Docker: start the embedded one on a spare port
(pg_ctl -D .pgdata -o "-p 5433" start, from @embedded-postgres/*/native/bin), then
docker compose exec postgres pg_dump -h host.docker.internal -p 5433 -U leadpool -d leadpool -Fc -f /tmp/db.dump
and docker compose exec postgres pg_restore -U leadpool -d leadpool --no-owner /tmp/db.dump.
How the guarantees are enforced
| Guarantee | Mechanism |
|---|---|
| Never the same lead twice | deliveries primary key (org, pool, lead) — inserts have no ON CONFLICT, a violation aborts the transaction |
| Never the same person twice | Every phone/email is normalised and HMAC-hashed into leads.keys; picks exclude any key in client_delivered_keys |
| No race between a client's orders | Per-client advisory lock + FOR NO KEY UPDATE SKIP LOCKED on picked leads |
| Exclusive / shared caps | buyers_count updated in the same transaction; lead flips to sold-out at the pool's max_buyers |
| Suppression | Client keys checked at pick time; global DNC flags leads as suppressed on upload and import |
| Money | Hold at submit (conditional wallet update), refund of any shortfall on completion, idempotency keys, append-only ledger |
packages/db/src/fulfillment.test.ts proves these under concurrent load.
Mixed orders. Every order has one or more order_lines (tier, snapshot, quantity, price); a mix
has several, orders.tier_id is NULL and its snapshot label describes the mix. Fulfillment fills the
lines freshest first, each from its own age window, inside the same per-client lock — keys delivered
for one line are already in client_delivered_keys when the next line is picked, so a mix never
repeats a person. deliveries.line_no records which line each lead was for (the file gets an
age_tier column), orders.charged_micros accumulates line quantity × line price, and the refund is
held − charged. All-or-nothing mixes roll back entirely if any line is short. In the portal:
product → Mix ages (or ?mode=mix); "Order again" on a mixed order rebuilds it (?mix=tierId:qty,…).
Deploy on your own server
One Linux VM with Docker is enough to start (4 vCPU / 8 GB RAM handles tens of millions of leads).
cp deploy/.env.production.example deploy/.env.production # set hosts, POSTGRES_PASSWORD, IDENTITY_PEPPER docker compose -f docker-compose.prod.yml --env-file deploy/.env.production up -d --build docker compose -f docker-compose.prod.yml --env-file deploy/.env.production run --rm -w /app/apps/api migrate \ node --import tsx scripts/create-admin.ts --email you@company.com --name "Your Name" --password "a-long-passphrase"
Try it on one machine first: set ADMIN_HOST=:8080, PORTAL_HOST=:8081, COOKIE_SECURE=false (and the
matching ADMIN_URL / PORTAL_URL) in deploy/.env.production, add -p leadpool-trial to each command,
then open http://localhost:8080 and :8081. Demo data: … run --rm migrate node --import tsx src/scripts/seed.ts;
throw the trial away with … down -v. The dev database (docker compose up -d) is a separate project and
isn't touched.
This runs Postgres 18, a one-shot migration, the API, the worker, and Caddy (automatic HTTPS for
ADMIN_HOST / PORTAL_HOST, security headers, SPA hosting, /api reverse proxy). Uploads and
exports live in the storage volume; back up pgdata + storage + your .env.production.
Operations notes
-
IDENTITY_PEPPERmust never change once data exists — dedup and suppression depend on it, and it also derives the keys that encrypt two-factor secrets and delivery-channel credentials. Back it up with your database backups. -
Two-factor: users turn it on under Settings → Security (password re-check → QR code → code → recovery codes). A user who loses their phone and recovery codes is reset by an admin (Staff / client Users → edit → Reset 2FA) or, for client users, by their org owner. Settings → General → "Require two-factor for all staff" can only be switched on by an admin who already uses 2FA.
-
Production: set
NODE_ENV=production,COOKIE_SECURE=true, serve admin/portal/API behind HTTPS on one origin per app (the SPAs call/apion their own origin), and enable Postgres backups with point-in-time recovery before real money flows. -
Exports live in
STORAGE_DIR/exportsand are purged after the configured lifetime (Settings → General). An Excel copy expires with its CSV; "Rebuild export" drops both. -
Archiving a pool column or age tier never deletes anything. Archived ones are listed under the live ones (pool → Columns / Age tiers → Archived …) with Restore. A restored column goes back to the end of the list with the values leads already carry — filtering and the lead-date role stay off until switched on; leads imported while it was archived simply don't have it. A restored tier comes back switched off and is refused while it would overlap a live tier. An archived column's key can't be reused for a new column (the values are stored under it) — restore it instead.
-
Email: connect your mail server in Settings → Delivery channels → Email (host, port, security, sign-in, from address — presets for Microsoft 365, Google Workspace, SendGrid, Mailgun, Amazon SES) and click Test; it is used for every email the platform sends. Alternatively set
SMTP_URL+MAIL_FROMin the environment (the channel wins when both exist). Also setPORTAL_URLandADMIN_URL(seedeploy/.env.production.example). With neither, every email is written toSTORAGE_DIR/mail/<day>/*.eml— Settings → Notifications shows which mode is active, lets you send a test and lists the last 50 emails with any errors. Failed sends retry with backoff (2 → 32 minutes) before being marked failed. -
Both apps have a light and a dark theme (sun/moon button in the sidebar header), remembered per browser. The inline pre-paint script in
apps/admin/index.htmlandapps/portal/index.htmlis identical, so onesha256-…indeploy/Caddyfile's CSP covers both — if you edit it, change both files and update that hash. -
Sidebar — the panel button at the top of the sidebar hides it (remembered per browser); the small tab on the left edge of the page brings it back (in Classic, a blue tab with an arrow). Below 1024px wide the sidebar is a slide-in menu behind the ☰ button.
-
Appearance — both the admin and the client portal (palette button in the sidebar header / top bar, or Settings → Appearance), applied instantly and remembered per browser:
- Style — Modern (the frosted-clay look) or Classic, a flat, dense enterprise-CRM skin (Tahoma 12px, blue applet bars, tab strips, yellow row selection; light only, no blur, shadows or animations).
- Menu — icon rail + sidebar on the side, or tabs across the top with a row of contextual shortcuts.
- Width — full width or a centred compact frame (1320px).
They are
data-kit/data-nav/data-widthattributes on<html>. The classic skin ispackages/ui/src/styles/classic.css: unlayered rules scoped to[data-kit="classic"]that restyle the shared components through theirlp-*hook classes — a new screen built from@leadpool/uicomponents gets both looks for free.
Repeat orders
A client saves any order as a repeat order from the product page (step 4, Repeat) — or from
/repeat in the portal — and it's placed for them automatically:
- When — every day, weekdays, chosen weekdays or monthly (day 1–28), at a time in their own time zone (daylight saving handled; a time skipped by the clock change runs just after it). Optional end date.
- How — the worker's
schedule-runjob sweeps every minute and places each due run through the normal order path: balance hold, approval queue for approval-mode clients, fulfilment, file, emails. A slot runs at most once (row claim + an idempotency key per slot), and missed slots aren't made up. - Where — each run's file goes to the channels picked on the repeat order (the client's destinations under Settings → Delivery), on top of any auto-send destinations.
- Money — a run the balance can't cover is skipped (nothing charged, the client is emailed); after 3 skips in a row it pauses itself. A product, tier or filter that no longer works pauses it too.
- Controls — the client can edit quantities / schedule / destinations, pause, resume, "Run now" (an extra order) or end it; staff see every client's repeat orders on the client → Repeat orders tab and can pause, resume or end them. Every order a repeat order places links back to it.
Tables: order_schedules, order_schedule_runs, orders.schedule_id (packages/db/src/schedules.ts;
the cadence maths is packages/shared/src/schedule.ts).
Lead returns
- Policy — Settings → Lead returns: the on/off switch for every client, the return window (days after delivery), the return limit (% of an order, at least one lead) and which reasons clients can pick. The same tab shows returns waiting, the last 30 days (credited, returned, return rate) and the clients with their own rules. Turning it off hides returns from every portal at once and drops them from the staff menus once nothing is left to review; returns already sent stay in the queue.
- Per client — client → Settings → Lead returns for this client: switch it off for one client, or give them their own window / limit (empty = the default).
- Client — order page → Returns → Return leads: paste lead IDs from the file, phones (any format) or emails, or load the edited CSV; the portal shows what matched, what didn't and what's over the limit. Pick a reason (disconnected, wrong person, duplicate, bad email, fake, no consent, other).
- Staff — Lead returns (rail badge) → each return shows the leads with their details, the share of
the order and the client's 90-day return rate. Untick leads that aren't bad and Approve — approved
leads are credited at the price paid for them (per line for mixed orders,
return_creditin the ledger) and set to archived so nobody else buys them; the rest are rejected. The client gets an email. - A delivered lead can be returned once; the allowance is enforced under the order's row lock, and rejected leads free their share of it again.
Tables: lead_returns, lead_return_items, orders.returned_qty / returned_micros,
client_orgs.returns_enabled (packages/db/src/returns.ts).
Tables: sort and filter
Every data table uses the shared DataTable (packages/ui/src/components/table.tsx). A column with a value
(accessorFn / accessorKey) sorts when its heading is clicked; meta.filter adds the filter button —
text, select (with meta.filterOptions), number, money (micro-dollars) or date.
- Small lists (clients, staff, imports, suppression lists, reports, …) sort and filter in the browser;
stateKeyremembers the view for the tab. - Big lists (orders, returns, wallet history, audit log, a pool's leads) pass
useTableState()to the table and its query: the API getssort,dirandfilter(JSON) andplanTableQuery(packages/db/src/table-query.ts) turns them into SQL. Each list declares its sortable / filterable columns up front, so a column id can only pick a known expression and every value is a bound parameter; paging stays keyset-based (the cursor carries the sort it was made for). Sorting a pool's leads by a lead field has a 10-second statement limit.
Guided tours
Both apps explain themselves with driver.js tours, themed per kit (Modern light / dark: frosted card, soft entrance, mint ring around the highlighted part; Classic: flat applet window, no motion).
- Welcome tour — on first sign-in: the menu, status counts, account, appearance and the compass button. When it ends with Done, the current page's tour follows.
- Page tours — every page (and every tab of a pool, client or Settings) has one, shown once on the first visit. Replay any time with the compass button in the page header or the account menu (Take the tour of this page / Welcome tour).
- Preferences — Settings → Appearance → Guided tours: turn first-visit tours off, restart the welcome tour, or Show every tour again.
- Never in the way — a tour waits while a dialog, menu or drawer is open, closes when you navigate, skips steps whose part isn't on screen (a hidden column, a button your role doesn't have, the rail on a phone) and only starts automatically once the page's data has loaded. Esc, the × or Skip tour close it; a skipped tour counts as seen.
- Where it's stored — per user on the server (
staff_users/client_users.tours_seen,.tours_auto;GET/PATCH /api/{admin,portal}/auth/tours), so it follows people across browsers.
Writing a tour: add data-tour="name" to the element (or the tour prop on Tabs, PillTabs, Segmented,
DataTable), describe the steps in apps/<app>/src/tours/<page>.ts (TourDefinition: id admin.<page> /
portal.<page>, **bold** in bodies) and call usePageTour(tour, { ready }) in the page — or in each tab, so
every tab gets its own. Engine and styles: packages/ui/src/tour/tour.tsx, packages/ui/src/styles/tour.css.
Delivery channels
Besides the portal download, delivered orders can be pushed to clients through five channels. Staff connect them once; each client then gets its own destination.
| Channel | What the client receives | You need |
|---|---|---|
| The file attached (or a portal link above the size limit you set) | SMTP host + sign-in, or SMTP_URL |
|
| The file as a document message from your business number | Meta for Developers → WhatsApp: Phone Number ID + a permanent access token. Free-form messages only reach people who wrote to you in the last 24 h — outside that window pick Approved template and enter its name, language and variables. Without a number on file, the send dialog offers a wa.me link instead | |
| Microsoft Teams | An Adaptive Card in their channel (order, product, rows, portal button, cloud links) | In Teams: channel → Connectors (or Workflows) → Incoming Webhook → copy the URL. A test posts a card titled "Lead Portal test" |
| OneDrive | The file uploaded to your drive under a folder per client, plus a share link | Token mode: a Graph Explorer token with Files.ReadWrite (expires after ~1 h). App mode (recommended): an Azure app with Files.ReadWrite.All application permission — tenant ID, client ID, secret and the drive owner — which never expires |
| Dropbox | The file uploaded to your Dropbox under a folder per client, plus a shared link | An app at Dropbox Developers with files.content.write + sharing.write; paste its key + secret and click Connect (refresh token — never expires), or paste a generated access token (expires after ~4 h) |
- Connect — Settings → Delivery channels (admins with
settings.manage). Each tab has step-by-step setup, a Test button (sends a real test message or uploads a small test file) and 30-day stats. Credentials are write-only: they are encrypted at rest (AES-256-GCM) and the UI only ever shows "Saved …last 4". Changing anything resets the channel to untested. - Per client — client → Delivery tab: email addresses, WhatsApp number, Teams webhook, cloud folder name, file format (CSV / Excel) and Auto-send. Each card has its own Send test.
- Send — order page → Send: tick channels, choose the format, add an optional note. The history under
the delivery file shows every send with status, link and Retry. Needs the
orders.sendpermission (managers and admins). - Auto-send — when an order's file is ready, every destination with Auto-send on is sent once.
- Client portal — turn on Clients can use … in a channel's "Client portal" section and it appears under the client's Settings → Delivery: the owner edits their addresses / number / webhook and auto-send, and any user can Send to… from an order. Cloud folders stay under staff control.
- How sends run — rows in
delivery_sends, processed by the worker (channel-sendqueue, swept every minute). Cloud uploads go first so the email / WhatsApp / Teams messages carry their links. Failures retry up to 5 times (1 → 3 → 10 → 30 min); an authentication error marks the channel needs attention. Every send and failure is in the audit log. - Testing without real accounts —
CHANNEL_DEV_WEBHOOK_HOSTS(non-production only) allows extra Teams webhook hosts, andWHATSAPP_API_BASE,GRAPH_API_BASE,MS_LOGIN_BASE,DROPBOX_API_BASEandDROPBOX_CONTENT_BASEpoint the adapters at a mock server.packages/db/src/channels/channels.test.tscovers each provider's request shapes and error handling.