Experiments (A/B Testing)
Test changes to your game on a share of your players or servers before rolling them out to everyone. Each group changes specific keys in a remote configuration, Gamebeast assigns players to groups and serves them the right values, and you compare the groups in analytics.
An experiment tests one or more changes against your live game. You pick a configuration, create up to 10 groups, and give each group its own changes to that configuration. Gamebeast decides which group each player (or server) is in, and serves them that group's version of the configuration. Your game code doesn't change: it reads configuration values the same way it always does.
Legacy experiments are being sunset on October 21, 2026
This page covers the current experiments system. Experiments created before it are listed under Experiments → Legacy in the dashboard, and stop being supported on October 21, 2026. Move any legacy experiment you still rely on before then. See Legacy experiments.
On Roblox, experiments need SDK version 1.0.0 or newer. See the Roblox installation guide.
How it works
- You create an experiment against one configuration. Each group records only the keys it changes, for example
economy.xpMultiplierfrom1to2. - A player's game server fetches the configuration. Gamebeast checks whether the player is eligible, puts them in a group, and applies that group's changes before sending the configuration back. The SDK never sees the other groups.
- The player keeps their group. The next time they join, they get the same group and the same values.
- Markers are tagged with the group. Every marker the player sends while in the experiment records which group they were in, so you can compare groups in the Query Builder.
Because each group changes only the keys it names, the rest of the configuration stays live. If you edit an unrelated key while an experiment is running, every group gets the edit.
Create an experiment
Go to Experiments in the project sidebar and click New experiment. The composer has four steps.
Drafts are only kept in your browser tab. An experiment can't be edited after you create it. To change one, terminate it and create a new one.
1. Definition
- Name. Must be unique in the project.
- Description. What you're testing and why. Your teammates will thank you.
- Schedule. When the experiment runs, in your project's timezone. Leave the range empty to start immediately and run until you stop it, or set only a start to schedule it and run until stopped.
2. Audience
What's the target? Choose the unit that gets assigned to a group:
| Unit | Assigned per | Use when |
|---|---|---|
| User | Player | The change is about one player's experience: prices, rewards, tutorial flow. Most experiments. |
| Server | Server instance | The change affects everyone in a server at once: map rotation, server-wide events, matchmaking. Players in one server share a group. |
Who is eligible? By default every player (or server) that fetches the configuration is eligible. Turn on Targeted to add conditions, and only matching units enter the experiment. Everyone else sees the base configuration. You can combine conditions with and / or and nest them in groups.
You can target:
- Roblox fields the SDK sends automatically: device, device sub-type, input type, country, language, place ID, and place version.
- Player history: whether they're a new player, when they were first seen, and days since first seen.
- Unit ID, to include or exclude specific players or servers.
- Custom properties your game sends (on Unity, set them in
UserProperties; in JavaScript, passproperties). The picker suggests properties Gamebeast has recently seen your game send.
Server experiments can only target fields that exist without a player, so player history isn't available for them.
Sticky assignments (on by default, only shown when targeting is on) decides how often targeting is checked:
- On: targeting is checked once. A player keeps their group even if they later stop matching, for example if they switch from phone to PC.
- Off: targeting is checked on every request. A player who stops matching stops getting the experiment, and gets their original group back if they match again.
Targeting decides who sees an experiment. It's not a security control, because values like device and custom properties come from the game. Never use it to grant purchases, entitlements, or access.
3. Allocation
Choose how many groups there are and how traffic is split between them. Each group has a name and a color and icon, which are used throughout the dashboard and in analytics.
This is the same control as in the dashboard. Try dragging the dividers:
- Control33.34%
- Double XP33.33%
- Triple XP33.33%
- Every group gets at least 1%, and the split always adds up to 100%.
- Type an exact share into a group's percentage field, or press Rebalance to split evenly.
With a single group, the handle sets how much of the eligible audience enters the experiment. The rest are held out: they stay on the base configuration and aren't measured against the group. This is the experiment's starting rollout, so you can raise it later from the experiment page, or turn on Gradual rollout under Advanced to ramp it up on a schedule. Use this to try a risky change on a small share of players first:
20% of eligible players enter New shop layout; the other 80% see the base configuration.
How should we assign players? With two or more groups, choose how players are placed:
| Mode | How it works | Choose it when |
|---|---|---|
| Round robin (default) | Players fill groups in arrival order, following the split exactly. | You want group sizes to match your split, even at low traffic. |
| Weighted | A player's group is decided from their ID, so the split can be uneven. | You want assignment to depend only on the player, not on arrival order. |
With weighted assignment, the real split gets closer to your percentages as more players enroll. Either way, a player keeps the group they were first given.
Advanced
Automatic assignment is on by default: eligible players enroll the first time they fetch the configuration. Turn it off to run a manual-only experiment, where only players you add by hand take part. This is useful for QA and playtests.
Gradual rollout releases the experiment to more players over time. Turn it on and choose a starting share, such as 5%, then add steps like "25% after 1 day" and "100% after 3 days", or pick a preset. This is a separate control from the split above: the rollout decides how many eligible players can enroll, and the split decides which group they go into. At 10% with a 50/50 split, 5% of eligible players land in each group. Every group grows together, so the comparison stays fair the whole way up.
The Advanced section opens by itself when a draft already has a rollout or manual-only assignment, or when a rollout stage needs fixing before you can continue.
Try the presets and drag through time:
1% of eligible players can enroll. The rest see the base configuration until the rollout reaches them.
- Control (50%)0.5%
- Double XP (50%)0.5%
The ramp is a smooth curve through each step rather than a staircase, so the share rises gradually between steps instead of jumping on the hour. Players who enroll early keep their group as the rollout grows.
4. Changesets
Pick the configuration the experiment changes. The project's primary configuration is selected by default.
Each group edits its own copy of that configuration in the same editor you use for configurations. Whatever you change becomes the group's changeset. A group you leave unchanged is served the base configuration, which is how you make a control group. The group list shows how many changes each group has.
- Switching to a different configuration discards every group's changes.
- Keys used by another experiment are locked. They're shown read-only, so two experiments can never change the same key. Experiments on different keys, even in the same configuration, can run at the same time, and a player can be in several of them.
- A lock covers a key and everything inside it: an experiment on
economy.rewardsalso lockseconomy.rewards.daily. - Scheduled experiments lock their keys as soon as they're created, so the base can't change before they start.
Click Create experiment. It starts straight away, or at the scheduled time.
If a group changes a private key, the experiment is marked Server-side only and is only served to requests made with a server key. Games using an SDK key, including the Roblox SDK, won't receive it. To test in-game, change only public keys.
While it runs
Experiment states
The experiments list groups experiments by state:
| State | Meaning | Listed under |
|---|---|---|
| Scheduled | Created, waiting for its start time. Its keys are already locked. | Active Experiments |
| Active | Running and enrolling players. | Active Experiments |
| Ended | Reached its end date. | Completed Experiments |
| Terminated | Stopped by someone before its end date. | Terminated Experiments |
You can filter the list by configuration and search by name.
The experiment page
Click an experiment to see its settings, targeting, and each group's changes as a table of base value against experiment value. Each group shows its share of traffic and how many players are enrolled.
The Rollout card shows how much of the audience can enroll right now, a chart of the ramp, and when the next step happens. You can:
- Advance to the next step now instead of waiting for it.
- Pause to hold at the current share. Scheduled steps wait, and keep their spacing when you Resume.
- Set % to choose a share yourself.
- Edit curve to reshape the stages still ahead. The curve picks up from the current share.
Lowering or pausing a rollout only stops new players from enrolling. Players already in the experiment keep their group. Each change asks for a reason, which shows up in the rollout history.
If an experiment's targeting uses custom properties, the page lists the properties your SDK must send for players to be eligible.
Manage members
The Members table lists every player or server in the experiment, with their group, how they were assigned (round robin, weighted, or manual), their status, and when they were assigned. Filter it by group, status, or source, or search for an ID.
For each member you can:
- Override to group… to move them to a different group. Use this to QA a specific group on your own account.
- Remove override to send them back to the group they were originally assigned.
- Exclude to take them out of the experiment. They get the base configuration.
- Re-activate to put an excluded member back.
Add member puts a specific player or server into a group, which is how you fill a manual-only experiment. Each change asks for a reason, which is kept for the audit trail.
Measure results
Experiment groups are available in the Query Builder as filters and breakdowns:
- Break down any metric by an experiment's groups to compare them side by side.
- Filter to players in an experiment, in specific groups, or in no experiments at all.
Only markers sent after a player was assigned count toward their group.
If some markers in your time range couldn't be matched to experiment data, the Query Builder shows a note with the share that was left out. Those markers are excluded from experiment filters and breakdowns rather than being counted as the control group, which would skew the comparison.
The AI assistant can also measure an experiment for you: ask it how an experiment is doing. It compares each group with the control, and reports the lift, a 95% confidence interval, whether the difference is significant, and a warning if the groups are unevenly sized, which can mean something went wrong with assignment.
End an experiment
An experiment ends by itself at its end date. To stop it early, open it and click Terminate.
- Every player goes back to the base configuration immediately.
- Its keys are unlocked, so other experiments and configuration edits can use them.
- It can't be restarted. Its definition, members, and results stay available to view and query.
Only scheduled and active experiments can be terminated.
Permissions
| Action | Needs permission to |
|---|---|
| View experiments and their members | View experiments |
| Create an experiment | Create experiments |
| Terminate an experiment | Terminate experiments |
| Advance, pause, resume, or set a rollout | Update experiments |
| Override, exclude, re-activate, or add members | Assign experiment members |
Without permission to create, you can still open the composer to review it, but you can't create the experiment.
Legacy experiments
Experiments made with the previous system are listed under Experiments → Legacy. They keep running as they are until October 21, 2026, when legacy experiments stop being supported.
The current system differs in a few important ways:
| Legacy experiments | Current experiments |
|---|---|
| Each group is a full copy of a configuration, frozen at creation | Each group stores only the keys it changes, so the rest stays live |
| A player can be in at most one experiment | A player can be in several experiments, as long as they change different keys |
| The game applies the group's configuration and tags markers | Gamebeast applies the group's changes and tags markers for you |
| Up to 8 groups | Up to 10 groups, or 1 group with a holdout |
| Assignment state could be lost and players re-drawn | Assignment is durable, and each player keeps their group |
To move a legacy experiment, update to Roblox SDK 1.0.0 or newer, then create a new experiment with the same groups and changes. Legacy experiments can't be converted, and their assignment history isn't carried over, so players will be assigned again. Terminate the legacy experiment when the new one starts, so players aren't in both.
Current experiments can't target cohorts yet. If a legacy experiment relies on a cohort filter, target the same players with conditions where you can.
Configurations
Change your game's values from the dashboard without shipping an update. A configuration is a JSON document your game reads through the Gamebeast SDK. Edit it, deploy it, and every running server picks up the change.
Engagement Markers
Markers are used to track player interactions and events in your experience. This data can be used to create heatmaps and other visualizations to help you understand how players are interacting with your experience.