Alerts

Alerts tell people that something happened in a project: a new issue, a regression, or a drop in the Real Experience Score. alerts() is a plugin in the deployment's config, and each way to send one (mail, webhook, Discord) is a channel inside it. Nothing is on unless it is listed.

The deployment configCopy Anchor Link

apps/api/analytics.config.ts lists the plugins and is read at startup. Secrets stay in the environment; the config only reads them.

import { alerts, discord, mail, smtp, webhook } from "@spoar/engine/alerts";
import { defineConfig } from "@spoar/engine/config";

export default defineConfig({
  plugins: [
    alerts({
      channels: [
        mail({ transport: smtp(process.env.MAIL_URL), from: "Analytics <remco@gmail.com>" }),
        webhook(),
        discord(),
      ],
    }),
  ],
});

• Leave alerts() out and there are no alert routes, and POST /v2/admin/jobs/alerts answers 503 with "Alerts are off".
• Leave a channel out and a target on it answers VALIDATION_FAILED: "mail is not enabled on this deployment".
• An empty or malformed MAIL_URL never stops the API: mail targets turn paused with the problem as the reason, and GET /v2/admin/alerts/status shows it.

Mail transportsCopy Anchor Link

smtp() speaks TLS from the start on smtps:// and upgrades with STARTTLS on smtp://; a server without STARTTLS is refused, so mail is never sent unencrypted. @ and : inside a user or password are written %40 and %3A. Mail credentials never reach the database.

How alerts are sentCopy Anchor Link

POST /v2/admin/jobs/alerts runs every 10 minutes from the jobs workflow:

1. Queue. New issues and regressions become alert events, up to 100 per run. Each gets one delivery per enabled target of its project that is subscribed to it, and a retried run never queues twice. Every regression alerts once.
2. Dispatch. Due deliveries are grouped per target into a batch, and the target's channel sends the batch as one mail or one request. A failure keeps them pending with the error and the next try from the retry policy, until the policy runs out and they turn failed. One failing target never holds back another.

Mail arrives as one message per target per run, newest alert first, in plain text and HTML. Discord gets one message with an embed per alert. A webhook gets a signed WebhookBody; see alert webhooks for checking it.

Speed dropsCopy Anchor Link

speed.drop is opt-in per target. Each alerts run compares yesterday's Real Experience Score (UTC day, production traffic, all devices, p75, metrics with at least 20 samples) with the 7 days before it, for every project with a target subscribed to it, and fires when the score fell 10 points or more to under 90. A drop alerts once per project and day. Change the numbers in the config:

alerts({ channels, speedDrop: { points: 15, below: 85, baselineDays: 14 } });

A project's targetsCopy Anchor Link

Targets are set per project by an admin, over HTTP or with the admin client:

curl -X PUT https://api.analytics.remcostoeten.nl/v2/projects/remcostoeten.nl/alerts/targets \
  -H "authorization: Bearer $RA_ADMIN_TOKEN" \
  -H "content-type: application/json" \
  -d '{ "targets": [ { "channel": "mail", "to": ["remco@gmail.com"] } ] }'

A wrong field answers VALIDATION_FAILED with the path of each field in details.fields, such as /targets/0/to/0.

All of them need a project admin. A webhook target's secret is generated by the API (whsec_ and 32 random bytes), shown only in the answer that created or rotated it, and stored per target to sign its requests.