APIWORX.esb
APIWORX.esb gives APIWORX admins a read-only view of the legacy ESB platform: volume, failures, flows, connectors and connection health per customer, with history that builds up day by day.
What APIWORX.esb is
A nightly job on a DigitalOcean droplet reads the production ESB database and pushes a daily snapshot into APIWORX. Every ESB page reads only those snapshot tables — the app never connects to the ESB database and stores no ESB database credentials. It is admin-only and does not re-run or edit ESB flows.
Pages
The overall dashboard includes an admin-only ESB view with All systems, AppFlows and ESB flows filters. Record syncs and failed records cover the last seven complete UTC days; ESB totals use login daily aggregates. Flow rows carry a system label and open the corresponding system's detail page. ESB snapshots remain global during impersonation; AppFlow activity follows the existing customer scope. AppFlow execution counts are shown separately from ESB record counts.
| Page | Where | What it shows |
|---|---|---|
| Dashboard | /esb/dashboard | Combined ESB and AppFlow volume, KPI sparklines, daily trends, customer-system donut, top routes, searchable customer/connector/flow tabs, matching and last 14 nightly loads. |
| Customers | /esb/customers | ESB logins grouped under Portal companies, with daily trend and connection errors on the detail page. |
| Connectors | /esb/connectors | Platforms by flow count, volume, failure rate and erroring connections. |
| Flows | /esb/flows | Every flow with route, status, volume; detail shows daily syncs and status history. |
| Connections | /esb/connections | Connected accounts, API errors first, with Erroring since and errors per day. |
| Customer Matching | /esb/matching | How each ESB login maps to a company, with manual overrides. |
Dashboard weekly comparisons use the last seven complete UTC days versus the prior seven, independently of the selected window. Today is partial; the first AppFlow window day is also partial. Seven-day trends exclude partial days and require at least three complete days. ESB totals come from per-login daily totals, not flow counts; connector totals count records at each endpoint and should not be added together as a global total. Portal migration status remains unavailable when it has not been supplied.
Nightly snapshot pipeline
At 2 AM CT the loader sends the last 3 complete UTC days of daily totals plus today's flow, login and connection snapshots. The first run backfills at least 90 days in daily batches. History is kept indefinitely.
Ingest endpoint
POST /api/esb/ingest (also /api/public/esb/ingest) with header Authorization: Bearer <ESB_INGEST_KEY>. Each run sends start, any number of batch requests (≤ 5 MB each) and finish under one run_id.
| Phase | Sends | Effect |
|---|---|---|
| start | run_id, window_from, window_to, snapshot_date, loader_version | Creates/resets the run and clears daily rows for days in window_from…window_to. |
| batch | login_daily, flow_daily, flows, logins, connections | Upserts daily rows and snapshot rows (keyed by snapshot_date). |
| finish | error (null on success) | Marks the run succeeded (or failed when error is set) and records row counts. |
Loader queries (run on the droplet)
- login_daily — per day per login, ok and failed record syncs from the sync logs. Source of all totals.
- flow_daily — per day per flow via the workflow rule's integration, with the objects synced.
- flows — each integration with its source/destination platform names and status.
- logins — each user with email domain and active/inactive/draft flow counts.
- connections — each connected platform account and its API error state; never token, secret, key or password columns.
How ESB data maps to the APIWORX Portal
The APIWORX Portal is the source of truth for who a customer is. A nightly sync (3 AM CT, plus the "Refresh from Portal" button on Customer Matching) copies the Portal's customer list, migration records and contract summaries into a local read-only cache. ESB pages only ever read that cache, so they keep working from the last successful sync even if the Portal is unreachable — the Matching page shows "Portal data as of …" and any sync failure.
| ESB concept | Maps to | How |
|---|---|---|
| Login (ESB user) | Portal customer | Each login carries an email domain; the login is matched to the Portal customer with that primary domain. Manual overrides always win. |
| Flow (integration) | Customer's integration | Flows belong to a login, so they roll up under the matched Portal customer. Source/destination platform names become the route (e.g. Shopify → NetSuite). |
| Connector | Platform | Connectors are the distinct platform names seen across flows and connections — they are not Portal records, but every flow and connection under a connector rolls up to its customers. |
| Connected account | Customer evidence | A connection's account name can match a Portal customer name when the email domain does not. |
| Contract & migration | Portal cache | ARR, next renewal date, migration status, wave, owner and target cutover shown on ESB pages come from the Portal contract and migration cache — never from the ESB database. |
AppFlow company tags are matched against Portal customer names the same way, so a customer with both ESB and AppFlow activity appears as one row. HubSpot company ids come from the Portal and link straight to the HubSpot record.
Customer matching
Rules, in order: manual override, login email domain (generic mail domains skipped), connected account name (platform names skipped), then login name. Names are normalized (lowercase, & → and, legal suffixes removed), and ambiguous names or domains are never auto-matched. Matched logins use the Portal customer id as their key; unmatched logins use esb-<user_id>. Manage overrides on Customer Matching.
Freshness & states
- "Data as of" shows the last successful load — amber after 30 hours, red if the latest load failed.
- If the latest load failed, pages keep showing the last good data under a red banner.
- "No ESB snapshot yet" appears until the first load; "No ESB activity in this window" for an empty range.
- Failure rate is amber at 1% and red at 5%.
Permissions & security
Admin-only on every page and server call. Snapshot tables are read-only for every signed-in role; only the ingest endpoint writes them. Ingest logs record only run id, phase, row counts and duration — never payloads.