Skip to content
Orbit guides

Workload author guide

How to write and keep a Playwright test repository that Orbit can run: the layout, the manifest, the tags and owners, locks, how Orbit lists your tests, variables, secrets and test accounts, and how to check your work locally, on the Station and in CI.

Who this is for

Section titled “Who this is for”

People who write Playwright tests in a workload repository. You know Playwright, Git and Node.js. You do not need to know how Orbit works inside.

Before you start

Section titled “Before you start”
  • The repository lives in your organisation’s Orbit GitHub organisation (written <your-org> on this page). The quickest start is a new repository from the template orbit-workload-template: it already has the manifest, a Playwright config, fixtures, a data folder and the CI check.
  • Node.js and Git on your PC, and read access to <your-org>.
  • From your Project Admin: the team slug for the manifest (for example example-customer-team) and the environment your repository tests (for example dev). The repository is registered in the Console by a Project Admin or System Admin; see Onboard a customer project.
  • To run your tests on a Station you need the QA Engineer role (or higher) on the project.

Steps

Section titled “Steps”

1. Lay out the repository

Section titled “1. Lay out the repository”
orbit.workload.yaml          the manifest (required, in the root, on the default branch)
playwright.config.ts         reads every address from environment variables
package.json
package-lock.json            or pnpm-lock.yaml: exactly one lockfile, matching the manifest
suites/
  AUTO_MM01_ASSIGNMENT_RULE/ one folder per suite
    assignment-rule.spec.ts
  AUTO_MM02_MATERIAL/
    create-material.spec.ts
fixtures/                    test.extend fixtures and page objects
data/                        test data: no addresses, no credentials
.env.example                 variable names only (KEY=), never values
.gitignore                   covers .env*, test-results/, playwright-report/
CODEOWNERS                   optional: test owners per path
.github/workflows/validate.yml   the Orbit check on pull requests
  • A suite is the first folder under suites/ that holds the test file. A repository without suites/ uses the top folder of each file. Suites matter for parallel runs, reports and locks (step 6).
  • One repository tests one environment. For another environment (for example qas), the Project Admin registers another repository or asks you for one.
  • Never commit a .env file. Only .env.example is allowed, with names and no values.

2. Write the manifest

Section titled “2. Write the manifest”

The manifest is orbit.workload.yaml in the repository root, on the default branch. A complete example:

schemaVersion: 1
id: example-customer-dev          # the workload id: unique across Orbit, cannot change later
name: Example Customer Dev        # display name; you can change it
team: example-customer-team       # the team slug from your Project Admin
runner:
  type: playwright
  config: playwright.config.ts    # default
install:
  packageManager: npm             # npm or pnpm; the matching lockfile must exist
  browsers: [chromium]            # chromium, firefox, webkit
environment:
  slug: dev                       # the environment this repository tests (optional but recommended)
  required: [BASE_URL]            # names of plain variables the tests read
  secrets: [APP_USER, APP_PASS]   # names of secrets the tests read
  dataDir: data                   # optional; checked for addresses and credentials
profiles:
  smoke:
    grep: "@smoke"
    workers: 2
    retries: 1
  regression:
    workers: 4
    retries: 2
    timeoutMinutes: 180
  default: regression             # optional; else the first profile
locks:
  - { tag: "@mutates-config", scope: suite }
artifacts:                        # optional; these are the defaults
  traces: on-failure
  videos: on-failure
  screenshots: on-failure
metadata:
  taxonomy: orbit-v1
  requireTestId: false
  vocabulary:
    module: [mm, sd]
    role: [requestor, approver]
Key Rules
id Lowercase letters, digits and dashes, 3 to 50 characters. It becomes the workload’s name in Orbit and cannot change after registration.
team The team slug your Project Admin gives you. Orbit refuses a repository whose team has no role on the project.
environment.required, environment.secrets Names only, never values. Capital letters, digits and _, starting with a letter. Names that start with ORBIT_ are reserved. A run stops before it starts if one of them has no value.
environment.slug The one environment of the project this repository tests.
profiles At least one. A profile is a named set of defaults for a run: project (Playwright projects), grep and grepInvert, workers (1 to 16), retries (0 to 5), timeoutMinutes (1 to 720, default 120) and headed. default: names one of them.
locks Tags of tests that change shared settings. See step 6.
metadata.vocabulary The allowed values of the tags @module:, @feature:, @process:, @scenario: and @role:.
Other top-level keys Not allowed, except your own keys starting with x-, which Orbit ignores.

Every run reads the manifest of the commit it runs. A broken manifest on a branch makes runs of that branch fail with “The workload’s manifest has errors”; the other branches keep working.

3. Tag your tests

Section titled “3. Tag your tests”

Tags go in the test’s details. Orbit reads them when it lists your tests.

import { test, expect } from '../../fixtures';


test('creates a material', {
  tag: ['@id:MM-001', '@priority:P1', '@req:MM-123', '@module:mm', '@smoke'],
}, async ({ page }) => {
  // …
});
Tag Format Required
@priority:P1 … @priority:P4 P1 to P4 (p1 is accepted and read as P1) Yes. A test without a priority fails the check. Priorities drive the quality gates.
@id:<ID> Capital letters and digits in dash-separated parts, ending in a number: MM-001, BP-CR-001. Unique in the repository, at most one per test. A warning when missing; an error when the manifest says requireTestId: true.
@req:<KEY> A requirement key such as ABC-123. A test may carry several. No. It links the test to a requirement for requirement coverage. A key the project does not have shows as uncovered there; Orbit does not reject it.
@module:, @feature:, @process:, @scenario:, @role: A value from metadata.vocabulary No. A value outside the vocabulary is an error.
Flags such as @smoke @ and lowercase words with dashes No. Use them in profile grep.
Lock tags such as @mutates-config Declared under locks No. See step 6.

Every tag must look like @key:value or @flag. Anything else is an error.

4. Name the test owners

Section titled “4. Name the test owners”

Orbit shows an owner for every test. It takes the first of:

  1. The orbit.owner annotation of the test or of its test.describe:

    test('creates a material', {
      tag: ['@id:MM-001', '@priority:P1'],
      annotation: { type: 'orbit.owner', description: '@anna.tran' },
    }, async ({ page }) => { /* … */ });

  2. The CODEOWNERS entry for the spec file (.github/CODEOWNERS, CODEOWNERS or docs/CODEOWNERS).

  3. The manifest team.

The annotation must be static, in the test’s details as above. An annotation added while the test runs is not seen. Tests whose owner is only the manifest team count as No explicit owner in the Console’s test debt. The full rules: Onboard a customer project.

5. Keep the config safe to load

Section titled “5. Keep the config safe to load”

The Station lists your tests (for the Explorer, and before each run) by loading your Playwright config. While it lists them:

  • Only plain variables are set. Secrets and test accounts are not. Never read a secret in playwright.config.ts, in a global setup that runs at load time, or at the top level of a spec file.
  • Read secrets inside tests and fixtures, where they are set.
  • A required variable that is missing during a check gets a placeholder value, so the config must not call the system under test while it loads.
  • Annotations and tags must be static (see steps 3 and 4).

If the config needs something that is not there, the Station shows “The test list could not be refreshed” with catalog-refresh-failed, and the reason.

A config that follows these rules:

import { defineConfig } from '@playwright/test';


export default defineConfig({
  testDir: './suites',
  use: { baseURL: process.env.BASE_URL },   // a plain variable: fine at load time
});

6. Use locks for tests that change shared settings

Section titled “6. Use locks for tests that change shared settings”

Some tests change configuration that other tests depend on. Tag them with a lock tag and declare it:

locks:
  - { tag: "@mutates-config", scope: suite }
  • scope: suite: runs of the same suite wait for each other; different suites still run at the same time.
  • scope: environment: while one run holds the lock, no other run on that environment runs those tests.
  • Inside a suite that has lock-tagged tests, tests always run one at a time, whatever the number of workers.

More, including how a waiting run looks: Station run features.

7. Read variables, secrets and test accounts in the tests

Section titled “7. Read variables, secrets and test accounts in the tests”

Everything reaches your tests as environment variables. You read them with process.env.

What Where the value comes from How you read it
Plain variables (environment.required) The Console, set by a Project Admin per environment. Readable by project members. process.env.BASE_URL
Secrets (environment.secrets) The Console, write-only. The Station gets them for the run only. process.env.APP_PASS, inside a test or fixture
Test accounts The project’s pool of test accounts. The Station leases them for one run, so two runs never share an account. ORBIT_ACCOUNT_<ROLE>_USERNAME and ORBIT_ACCOUNT_<ROLE>_PASSWORD
Run facts Set by the Station ORBIT_PROJECT, ORBIT_ENVIRONMENT, ORBIT_WORKLOAD, ORBIT_STATION_ID

About test accounts:

  • A Project Admin sets which roles your workload needs on each environment, and how many accounts of each (Project settings, Test accounts, Workload needs). Without needs, runs use the environment secrets.
  • The role is upper case with _ for -: the role gl-admin gives ORBIT_ACCOUNT_GL_ADMIN_USERNAME. The second account of a role ends in _2, the third in _3.
  • For a role with one account, the Project Admin can add aliases, so existing tests that read, say, APP_USER and APP_PASS keep working.
const role = 'APPROVER';
const user = process.env[`ORBIT_ACCOUNT_${role}_USERNAME`];
const password = process.env[`ORBIT_ACCOUNT_${role}_PASSWORD`];

Orbit hides secret values and leased passwords in the run log and the report. Still, never print a secret yourself, and never put addresses or credentials in data/ or in the code.

8. Run the tests on your PC

Section titled “8. Run the tests on your PC”
  1. Install the dependencies: npm ci (or pnpm install --frozen-lockfile), then npx playwright install chromium.
  2. Put the values you need in a local .env file. It is in .gitignore; never commit it. The template loads it with process.loadEnvFile('.env') for local runs.
  3. Run Playwright as usual, for example npx playwright test --grep @smoke.

Local runs are for writing and debugging. Their results do not reach Orbit.

9. Run the tests on the Station

Section titled “9. Run the tests on the Station”
  1. Push your branch.
  2. On the Station, open Run. Choose the environment, the workload and the profile, and type your branch in Branch, tag or commit. Choose Start run.
  3. The Explorer shows the last test list this Station made. After a merge to the default branch, choose Refresh from main to see the new tests. See Station run features.
  4. Read the result like any run: the run page, Open report, and the run in the Console. The Tester guide shows the daily flow.

When the lockfile changes, the first run on each Station installs the new dependencies, so it is slower. If your lockfile gains a package from another private repository, tell your Project Admin: the workload needs Refresh dependencies once in the Console.

10. Check your work locally and in CI

Section titled “10. Check your work locally and in CI”

The Orbit workload check validates the manifest, the tags and the repository rules. Install the dependencies first, then run it in the repository folder:

Terminal window
npx github:<your-org>/orbit-workload-tools validate
npx github:<your-org>/orbit-workload-tools validate --no-list   # skip the test list (faster)

It prints each finding with its file and line, then OK or FAILED with the number of tests, profiles, errors and warnings.

In CI, every workload repository runs the same check on pull requests, through the reusable workflow of your organisation (the template already has it in .github/workflows/validate.yml):

name: Validate
on: [pull_request]
jobs:
  orbit:
    uses: <your-org>/.github/.github/workflows/validate-workload.yml@v1

On GitHub the findings appear as annotations on the pull request and as a job summary. The check is usually required before a merge to main.

What the check refuses:

Rule Error or warning
The manifest does not match the schema, or has an unknown top-level key Error
The lockfile is missing, or another package manager’s lockfile is there Error
runner.config does not exist, or playwright test --list fails Error
A test has no @priority, a malformed @id, a duplicate @id, or a tag value outside the vocabulary Error
A committed .env file, or a .env.example with values Error
A file under dataDir contains an address, a credential-like key or a secret-like value Error
A test has no @id (and requireTestId is false) Warning
A profile selects no tests Warning

What can go wrong

Section titled “What can go wrong”
You see Why What to do
priority-missing from the check A test has no @priority:P1…P4. Add the tag.
id-duplicate Two tests carry the same @id. Give each test its own id.
vocabulary A @module: (or similar) value is not in metadata.vocabulary. Add the value to the vocabulary, or fix the tag.
env-file A .env file is committed, or .env.example has values. Remove the file from Git; keep names only in .env.example.
playwright-list Playwright cannot list the tests, often because dependencies are not installed. Run npm ci (or pnpm install --frozen-lockfile) and try again.
“The test list could not be refreshed” (catalog-refresh-failed) on the Station The config failed to load with plain variables only, for example because it reads a secret. Move the secret read into a test or fixture (step 5).
“Variables are missing” on a run A name in environment.required or environment.secrets has no value for that environment. Ask a Project Admin to set it in the Console.
“The workload’s manifest has errors” on a run The manifest of that branch is invalid. Run the check on the branch and fix it.
The Console refuses to register the repository Wrong team, an id already used, or no manifest on the default branch. See Onboard a customer project.
Tests show No explicit owner No orbit.owner annotation and no CODEOWNERS entry. Add one (step 4).
Two runs change the same settings and break each other The tests have no lock tag. Add a lock (step 6).

Values to record

Section titled “Values to record”
Value Where you find it Kind
Repository <your-org>/<repo> GitHub Not secret
Workload id The manifest Not secret; cannot change after registration
Team slug From your Project Admin; the manifest team Not secret
Environment slug From your Project Admin; the manifest environment.slug Not secret
Variable and secret names The manifest environment.required and environment.secrets Names are not secret. Values are kept in the Console only, never in the repository.
Test account roles From your Project Admin (Workload needs) Not secret
Section titled “Related guides”