Testing in production

Let QA try a feature in production, with real infrastructure and data, while it stays off for everyone else. When they are happy, start the rollout.

This page follows a new wishlist feature. Testers can be people checking by hand or an automated suite running against production. Anyone who should see a launch before customers do, such as a product manager signing it off or a marketing team preparing its announcement, can use the same test accounts.

Hide the feature from everyone#

Start with the feature off for every user:

description: Add products to a wishlist
bucketBy: userId
rules:
  - key: everyone
    conditions: '*'
    percentage: 0

It buckets by userId, since only signed in users have wishlists. The default project already declares userId and deviceId.

Let QA in#

force turns the feature on for matching users, whatever the rules say. List the test accounts:

# ...
force:
  - conditions:
      - attribute: userId
        operator: in
        value:
          - qa-user-1
          - qa-user-2
          - qa-user-3
    enabled: true

Once the datafile is published, those accounts see the wishlist in production, and nobody else does.

Use dedicated test accounts rather than employees' own. A datafile in a browser can be read by anyone, IDs included. On a server, a boolean attribute such as qaTester, set from trusted account data, keeps the list out of the datafile altogether.

Keep testers in a segment#

When several features need the same testers, keep them in one segment:

description: QA test accounts
conditions:
  - attribute: userId
    operator: in
    value:
      - qa-user-1
      - qa-user-2
      - qa-user-3

# ...
force:
  - conditions: qa
    enabled: true

Each feature under test forces the same segment, and adding a tester is one change.

Include test devices#

A feature that anonymous visitors also see, such as a new landing page, needs testers who are not signed in. Give each device a random deviceId stored on it, and list the test devices beside the accounts:

description: QA test accounts and devices
conditions:
  - or:
      - attribute: userId
        operator: in
        value:
          - qa-user-1
          - qa-user-2
          - qa-user-3
      - attribute: deviceId
        operator: in
        value:
          - qa-device-1
          - qa-device-2

A feature rolled out to anonymous visitors too should bucket by { or: [userId, deviceId] }, as Bucketing explains. The qa segment decides who sees an unreleased feature, not who may use it, so keep authorization on the server.

Evaluate as a tester#

The application passes the tester's IDs like any user's:

const context = {
  userId: 'qa-user-1',
  deviceId: 'qa-device-1',
}
if (f.isEnabled('wishlist', context)) {
  // show the wishlist
}

Roll out after testing#

When QA signs off, remove the force entry and start the rollout:

# ...
rules:
  - key: everyone
    conditions: '*'
    percentage: 5

Progressive delivery covers widening it from there.

Related#

• Features: force
• Segments
• Testing before deployment
• Early access, for customers who opt in