Skip to content
FlagSwiftDocs

Measure a feature's impact with exposures and outcomes

Switch on outcome analytics, report what people were shown and what they did next, and read the result on the flag.

Time
About 10 min
Who can do it
An operator turns it on; anyone with metrics access can read
Where
Server settings, your app, and a flag's Analytics tab
You finish with
Conversion by variant on the flag

What and why#

Evaluation counts tell you how often a flag was read. They cannot tell you whether the people who saw the green button bought more than the people who saw the blue one. Outcome analytics answers that. Your app reports two things, and FlagSwift joins them for you.

Exposures and outcomes#

  • An exposure says a person was shown a flag: the flag key and the variant (or on and off for a flag without variants).
  • An outcome says a person did something that matters: an event name such as purchase and, optionally, a number such as the order total.

How an outcome is credited#

An outcome is credited to the variants that person was already shown in that environment. FlagSwift looks up the person's exposures, keeps the most recent variant for each flag, and credits the outcome to the 20 flags they were shown most recently. A person who was never shown a flag counts for no flag, so nobody can be credited to a variant they did not see. Each person counts once per flag, variant and event however often they repeat it. The number of repeats and the summed values are kept separately.

What is stored#

Never the key you send. FlagSwift keeps a keyed hash of it (HMAC-SHA-256 with your JWT_SECRET, scoped to the project), so the same person in two projects has two hashes. Only counts are kept, and rows are deleted after the retention period.

What this is and is not

This is outcome analysis, not an experimentation platform. There is no sample-size planning, no stopping rules, no automatic winner, no funnels and no guardrails.

Before you start#

  • Access to the server settings. Nothing is recorded unless ANALYTICS_ENABLED=true is set.
  • A server or client key, and a flag with variants if you want to compare more than on and off. Reporting uses the evaluation:read scope, which every SDK key has.
  • A stable key for each person that your app can use for both exposures and outcomes, up to 255 characters.
  • To read the result you need the metrics.read permission. Every built-in role has it.

Walk through it#

  1. Turn it on

    Set ANALYTICS_ENABLED=true and restart the server. It is off by default. Rows are kept for 90 days. Old rows are removed in the background, at most once every six hours per instance.

    Do not rotate JWT_SECRET casually

    The hashes are made with JWT_SECRET. Changing it changes every hash, so everyone starts over and earlier exposures can no longer be matched to new outcomes.
    You should see. With the setting off, reporting calls reply { "enabled": false, "recorded": 0, "ignored": 0 } and keep nothing. With it on, enabled is true.
  2. Report an exposure

    When a person is shown the flag, send an exposure with the same key you will use for outcomes. For a flag with variants, name the variant. For a flag without variants, send enabled instead.

    Exposure body
    FieldTypeDescription
    userKey*stringWho was shown the flag. Up to 255 characters, trimmed. Also accepted as user.
    exposures*arrayBetween 1 and 50 entries per request. Duplicate entries in one request are collapsed.
    exposures[].flag*stringThe flag key.
    exposures[].variantstringA variant key that exists on the flag, for a flag with variants.
    exposures[].enabledbooleanWhat was served, for a flag without variants. Recorded as on or off.

    Each entry needs a variant or enabled. An entry that names an unknown flag or variant is ignored, not rejected, so a client cannot invent rows.

  3. Report an outcome

    When the person does something that matters, send an outcome.

    Outcome body
    FieldTypeDescription
    userKey*stringThe same key used for the exposure.
    event*stringLetters, numbers, spaces and . _ / + - only, up to 64 characters. Other characters are removed.
    valuenumberOptional, such as an order total. It must be finite and between -1,000,000,000 and 1,000,000,000. Values are summed per variant.

    Send the exposure first

    An outcome is credited only to variants the person was already shown. An outcome with no earlier exposure is credited to nothing, and the reply says recorded: 0.
  4. Or use an SDK

    Every SDK has trackExposure, track and flushAnalytics helpers, so you do not call the endpoints yourself. Exposures are batched per person and sent about two seconds later. An outcome is sent straight away, after that person's exposures. Automatic exposures are off unless you turn on autoExposures. Call flushAnalytics before a serverless function returns.

  5. Look at the result

    Open Flags, choose the flag and open the Analytics tab. Scroll to Outcomes by variant, choose the environment and a range of 24 hours, 7 days or 30 days. It lists how many people were shown each variant, then one table per event.

    What the section looks like once exposures are in but no outcome has been reported yet. After the first outcome, an event table appears under the variant line.
    You should see. The line above the tables reads like green: 412 shown, with the control marked. After a test purchase the event appears as its own table.

Do it with the API

Both endpoints answer 202. env is required for a project-wide key and optional for a key limited to one environment (naming a different one is a 403). For an exposure, recorded counts rows written and ignored counts unknown flags or variants. For an outcome, recorded is the number of flags it was credited to.

Request
# Report that user-42 was shown the green variant
curl -X POST "https://flags.example.com/api/v1/analytics/exposures?env=production" \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"userKey":"user-42","exposures":[{"flag":"button-colour","variant":"green"},{"flag":"checkout-v2","enabled":true}]}'

# Later, report that they bought something
curl -X POST "https://flags.example.com/api/v1/analytics/outcomes?env=production" \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"userKey":"user-42","event":"purchase","value":49.9}'
Response (202)
{ "enabled": true, "recorded": 2, "ignored": 0 }

Or from an SDK

Go, Python, PHP, .NET, Flutter and Swift have the same three calls in their own style. A call with no identifier for the person is ignored.

Request
// Browser, React, Vue, React Native, Next.js client: the current user
flags.trackExposure("button-colour");
await flags.track("purchase", 49.9);

// Node and other server SDKs: you say who
flags.trackExposure("button-colour", { identifier: user.id });
await flags.track("purchase", { identifier: user.id }, 49.9);
await flags.flushAnalytics(); // before a serverless function returns

Good to know#

Use one key for exposures and outcomes

The most common reason for an empty result is a mismatch: an exposure sent for an anonymous id and an outcome sent for the signed-in user id. Pick one identifier, and keep it the same on both calls.
Do I have to send an exposure for every request?
No. An exposure is stored once per person, flag and variant, and later calls only raise a count. Sending it once per session is enough.
What if a person is shown two variants of one flag?
That happens if the configuration changes while they are active. The outcome is credited to the variant they were shown most recently.
Does this slow my app down?
The SDK helpers never throw and never slow evaluation. A failed request is retried once, then dropped. If the server has analytics off, the SDK stops sending for an hour.
Can a browser report outcomes?
Yes, with a client key. The endpoints accept both server and client keys, and the flag and variant must exist in the environment, so a client cannot create new ones.

What to check#

  • The exposure reply says how many were recorded and how many were ignored. Ignored means an unknown flag or variant.
  • With analytics off, the reply is { "enabled": false } and nothing is kept.
  • The Analytics tab shows a message that outcome analysis is switched off if the server setting is missing.
  • After a test purchase, the event appears under the flag. If it does not, check that the exposure used the same key and was sent first.

If something goes wrong#

The reply is { enabled: false } and the tab says outcomes are switched off

Why: The server is not running with ANALYTICS_ENABLED=true.

Fix: Set it, restart, and send the exposures again. Nothing was kept while it was off.

An exposure reply says ignored: 1

Why: The flag key does not exist in that environment, or the variant is not one of the flag's variants.

Fix: Check the exact key and variant spelling. For a flag without variants, send enabled instead of variant.

An outcome reply says recorded: 0

Why: That person has no exposure in this environment, or the exposure used a different key.

Fix: Send the exposure first, with the same userKey, and use the same env on both calls.

400 INVALID_EVENT or INVALID_VALUE

Why: The event name was empty after cleaning, or value was not a finite number within one billion.

Fix: Use letters, numbers and . _ / + - up to 64 characters, and send value as a number, not text.

The tab says This period has more outcomes than one report reads

Why: The period holds more outcome rows than a single report reads, so the figures cover the most recent ones. A second note says Only the most common events are listed when there are more than 25 events.

Fix: Choose a shorter range, or report fewer distinct event names.

400 ENV_REQUIRED

Why: A project-wide key was used without ?env=.

Fix: Add ?env=production (or the environment name), or use a key limited to one environment.