Creating standard experiments

Standard experiments require developer changes to internal services. Before creating a standard experiment in the UI, necessary configuration details will be provided to you (e.g. experiment and variant names). You will need to use your standard escalation process to gather the information needed to complete the experiment creation form.

To create a new experiment, select ‘Create’ from the sidebar.

Form Fields Overview

Below is a quick overview of the main fields in the form:

Experiment name

A textbox where you can enter the experiment name in. You cannot leave this empty. Use your standard escalation process to get the experiment name.

Please find the requirements for naming convention as below.

1. No spaces allowed, instead, underscore (“_”) should be used.

2. You can use alphabetical characters (a-z, A-Z) and numerical digits (0-9)

3. Please suffix with wave_01, if it’s the first wave of this experiment, or wave_02, wave_03, etc, for sequences of experiments improving on the same feature. This is not enforced, but is advised for clarity.

4. Example:

   Suitable experiment name: search_auto_boost_wave_01

   Suitable experiment name: search auto boost wave 01

   If an experiment is created for testing/QA purposes, please prefix with \test_. Example: \test_search_auto_boost

Sites

A dropdown that lets you select the website you want to run your experiment. You can click multiple websites in this section.

Type

Select “Standard” for standard experiment.

Variants

You can write your variant names and traffic split in this section. Use your standard escalation process to get the variant names.

Please find the requirements for naming convention as below.

1. No spaces allowed, instead, underscore (“_”) should be used.

2. You can use alphabetical characters (a-z, A-Z) and numerical digits (0-9).

3. Prefix with c| for control, v1| for the first variant, v2| for the second etc. - The Experimentation UI does this by default, so you only have to type the names without prefix.

4. For control, the standard naming is c|control when it reflects the default version of the site.

5. Example:

   Control: c|control

   Variant 1: v1|auto_rotate_banner

   Variant 2: v2|manual_rotate_banner

The traffic split allows you to decide how much traffic to allocate to each variant. The traffic split must add up to 100%. Variants section can support up to 8 variants.

Experiment start and end date

You cannot select or enter a start or end date earlier than the current date.
The end date must be later than the start date.
The minimum duration for running an experiment is 4 weeks. If you wish to run an experiment less than 4 weeks, please consult with the Experimentation team.

Base events and metrics

You can select the section or page of the website where your experiment will take place. Choose this based on the changes made in your experiment. For instance, if your experiment is on the homepage, select “Homepage” in this field. You can select more than one base event if your experiment impacts multiple sections or pages, such as the homepage and the product page.

The default metrics are the key metrics we have pre-populated based on your base event. You can select additional metrics in the optional metrics section if necessary.

Set live tick box

A checkbox that, if selected, will automatically set the experiment to go live at the selected start date and time. If left unchecked, the experiment will be paused if the current time is later than the start time. You can schedule the experiment and set it live later on after getting approval from the Experimentation team.

You can now view the details of your experiment. Please refer to the managing your experiments section on how to view experiment details.