Outcome analytics
Evaluation counts tell you how often a flag was read. Outcome analytics tells you what people did after they saw each variant, so you can compare conversion between variants.
How it works#
Your app reports two things. When a person is shown a flag, it sends an exposure. When they do something that matters (a purchase, a sign-up), it sends an outcome. FlagSwift credits each outcome to the variants that person had already been shown, then compares each variant’s conversion rate with the control’s on the flag’s Analytics tab.
- An outcome is only credited to variants a person was shown, so send the exposure first.
- Each person counts once per flag, variant and event, however often they repeat it. How often, and the values you send, are summed separately.
- The control is the variant served when the flag is off (the environment’s off variant), or
offfor a flag without variants. - A comparison appears once each side has at least 100 people. It is a two-proportion z-test with a 95% interval. Treat it as a signal: it does not correct for checking many events, or checking repeatedly while data comes in.
This is outcome analysis, not an experimentation platform
Privacy and switching it on#
Nothing is kept unless you run the server with ANALYTICS_ENABLED=true. What is kept per person is a keyed hash of the key you send (scoped to the project, using your JWT_SECRET), never the key itself. Rows are deleted after 90 days. Changing JWT_SECRET starts everyone over, because the hashes change.
Reporting from your app#
Both endpoints take a server or client key and an env query parameter (not needed for a key limited to one environment). Both reply 202.
Report exposures#
/api/v1/analytics/exposures?env=production| Field | Type | Description |
|---|---|---|
| userKey | string | Who was shown the flag. Up to 255 characters. Use the same key for outcomes. |
| exposures | array | Up to 50 of { flag, variant } for a flag with variants, or { flag, enabled } for one without. |
curl -X POST "$FLAGSWIFT/api/v1/analytics/exposures?env=production" \
-H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
-d '{"userKey":"user-42","exposures":[{"flag":"checkout-v2","enabled":true},{"flag":"button-colour","variant":"green"}]}'The reply says how many were recorded and how many were ignored (an unknown flag or variant). With analytics off it replies { "enabled": false } and keeps nothing.
Authorization
evaluation:read
Report an outcome#
/api/v1/analytics/outcomes?env=production| Field | Type | Description |
|---|---|---|
| userKey | string | The same key used for exposures. |
| event | string | A name for the outcome, such as purchase. Letters, numbers and . _ / + - only, up to 64 characters. |
| value | number | Optional, such as an order total. Summed per variant. |
recorded in the reply is how many flags the outcome was credited to, up to the 20 the person was shown most recently.
Authorization
evaluation:read
From an SDK#
Every FlagSwift SDK has helpers for this, so you do not need to call the endpoints yourself. Nothing is sent unless you call them or turn on autoExposures (default off). Only the person's identifier, the flag, the variant (or on/off), the event name and an optional number are sent. If analytics is off on the server, the SDK sees { "enabled": false } and stops sending for an hour.
// Browser, React, Vue, React Native, Next.js: the current user
flags.trackExposure("checkout-v2");
await flags.track("purchase", 49.9);
// Node, Nest, Next.js server: you pass the person
flags.trackExposure("checkout-v2", { identifier: user.id });
await flags.track("purchase", { identifier: user.id }, 49.9);
await flags.flushAnalytics(); // before a serverless function returnsclient.TrackExposure("checkout-v2", user)
client.Track("purchase", user, 49.9)
client.FlushAnalytics(ctx)client.track_exposure("checkout-v2", user)
client.track("purchase", user, 49.9)
client.flush_analytics()PHP, Laravel, .NET, Flutter and Swift have the same three calls in their own style (trackExposure, track, flushAnalytics). The person is the first identifier of the user, else the first device id; without either the call is ignored.
- Exposures are batched per person and sent about two seconds later. An outcome is sent straight away, after that person's exposures, because an outcome is credited only to variants the person was already shown.
autoExposuresreports an exposure the first time each person gets each outcome from a single-flag read (isEnabled,variant,value). Reading every flag at once does not report.- Calls never throw and never slow evaluation. A failed request is retried once, then dropped. Counters (pending, sent, dropped, enabled) are in the SDK's health or status object.
- Flags that a server SDK evaluates locally are not counted in the evaluation metrics, so a flag used only by server SDKs can look unused on the Metrics page.
Reading the results#
Outcomes for one flag#
/api/flags/{id}/analytics?environment=production&days=14Returns, per variant, how many people were shown it, and for each event the people who did it, the rate, the value per person, and the comparison with the control (verdict, difference, 95% interval, p-value). days is 1 to 90, within the retention period.
Authorization
metrics.read