Start with playbooks

What is a Playbook?

A playbook is the way to run multiple groups of tests in sequence. You can think of it as an overall script that consists of one or more playlists.

• The playbook controls the order in which playlists run.
• Each playlist automatically collects test cases and scenarios based on tags.

When you start a playbook, JOSF executes playlist 1, then playlist 2, and so on.
The execution within a playlist is not ordered: tests run in parallel and in an undetermined sequence. Every test must therefore be fully independent.

Playbooks are especially useful when you have predefined flows—like smoke → regression → end-to-end—but still want flexibility in which specific tests belong to each phase. With proper tagging, your playlists and playbook stay up to date automatically.

---

How a Playbook Works

1. Create a playbook.
2. Add one or more playlists.
   Each playlist has its own tag filters and parallelization settings.
3. When the playbook starts, JOSF automatically collects all test cases and scenarios that match the playlist’s tag rules.
4. Playlists run in the order you configured.
   The tests within a playlist run in parallel with no guaranteed order.

Need test A to run before test B?
→ Put them in separate playlists and control the playlist order.

---

Viewing Results

After execution, you can review everything in the playbook results.
There you’ll find, per playbook:

• success/failure statistics
• timing information
• per-playlist results

For more background:

• See the dashboard tutorial for analysing results.
• See the playlist tutorial for filtering, tag logic, and parallel execution.