How you structure your Test Sets affects how smoothly your team executes tests. Should you create one big Test Set or break it into smaller, nested ones? The answer depends on your workflow.
Flat structure (one Test Set, many instances of tests):
Use this when tests belong together and run as a unit: like a full regression cycle or a complete feature validation. A flat structure keeps related tests in one place, making it easier to track overall status and assign the whole batch to a team or person.
Nested structure (multiple Test Sets, organized by category):
Break tests into separate Test Sets when they’re run independently or by different team members. For example, you might have a “Mobile QA” Test Set that contains “sub-Test Sets” for “Login Flow,” “Payment,” and “Checkout”, each run separately or assigned to specialists.
Even with nested Test Sets, you can organize and group them together using custom fields and filters, so you still get a unified view when you need one.
When to nest:
- Tests are run by different people or at different times
- You need to track performance of specific test categories separately
- Tests apply to different platforms, components, or features
- You want granular assignment and notifications
When to flatten:
- Tests must run in a specific order (use Enforce Run Order if needed)
- You’re tracking end-to-end workflows that fail as a unit
- Your team runs everything together in one cycle
Pro tip: Test Sets are limited to 2,000 instances, but even a lower number of instances in a Test Set may become confusing for execution and assignment. If you find yourself creating Test Sets with too many instances, nested structure is your friend.
What about your team? Do you prefer flat or nested Test Sets, and what drove that choice?