How to Setup Firebase for Applications? Firebase A/B testing Guide

With Firebase, you can perform A/B testing in applications with minimal effort and better results. It’s a mighty easy platform.  

This is the second part of the series to guide you through set and Testing! Part 1- Ways to implement A/B tests in Firebase.

How to configure the A/B test?

We are starting with Firebase A/B testing using the Remote Config feature.

1. Set up the A/B test in the Firebase console. 

2. Click the A/B testing item from the left side menu and click the Create Experiment button.

3. The Popup box will ask to choose from- Remote Config, Notifications, or In-App Messaging to configure an A/B test. Since we will be configuring the test via config parameters select the Remote Config option.

4. Create an experiment via Remote config:  

In the A/B test configuration screen, allow users to define and set up the A/B test with a couple of conditions and parameters to be satisfied. 

Here we’ll use the same parameters that were created in the “Set up Remote Config in Firebase console” section.

How to configure the A/B test in the Firebase console using Remote Config?

To configure the Firebase A/B Testing using Remote Configuration, follow the below step-by-step process:

Step 1: Provide the Experiment name and details: 

Step 2: Condition to target users

In this section, we need to select the app in which we need to activate this experiment.

Here, for targeting, take All users to be eligible for an experiment based on the conditions we use.

Activation Event(Optional): 

An event that activates the experiment for a user. All targeted experiment users will receive an experiment variant treatment from the start of the experiment. Only those users who trigger the activation event will be included in the experiment measurement.

Here, select Continue_Shopping(custom event) as an activation event to track how many users are clicking the shopping button, then click Next.

Step 3: Define Goals

By utilizing goal metrics (such as Crash-free users and user retention) plus maximum occurrences of events (such as daily user engagement, app_remove, first_open, and custom events). 

The goal will be deemed a metric that can be chosen for the variant that best achieves the experiment leader as the primary goal. Click Next.

• Types of metrics to track 

Same as goals, there is a list of metrics available in the above screencap, at max 5 metrics can be used per A/B test. Distinct metrics as additional metrics (secondary goals) can be configured as compared to goals as secondary goals.

E.g. for the test experiment, we have set Purchase revenue as the primary metric and in additional metrics, we’ve selected other metrics

Step 4: Select variants

By using the parameters that are defined in Set up Remote Config in the Firebase console section, multiple parameters are applied for variant setup with values for Control Group and Variants. Also, the A/B test has been configured as an alternative text and color shade for the shopping button.

• Types of variants

The baseline indicates the App’s original view, whereas variants define the different variations to accomplish the goal. At max 7 variants are allowed for configuration.
Multivariate A/B tests can be achievable by adding parameters/clicking Choose or creating new.

• Adjust variant weights

Variant weight refers to the distribution of variants to users. It will be in the form of a ratio among all variants.

Step 5: Click the Review button.

4.  Save the draft review it and test it in the testing environment

Test A/B test in Testing Environment

1. It’s the cherry on the cake.

Firebase leverages users to verify A/B tests in the device before making it live. Click AB test tab -> Select your AB test from the drafted section -> Open settings using 3 dots  -> Click

Manage test devices.

2. Provide the device instance ID and select the variant that you want to test. To get Instance IDs for Android and iOS:

     a. Android snippet to fetch Instance ID as below:

    b. iOS snippet to fetch Instance ID as below:

3.  Add ID in the token box, and select the variant that you want to verify. Click on Save.

4.  Below are our example screens with changes visualized:

Deploy Experiment in Live Environment

Yeah! It’s time to make experiments live across all users to provide easy access and engagement. It’ll help us in deciding whether to roll out the variation in the live environment or not.

1. In the Firebase console, on the left side menu clicking the A/B Testing option will show a list of running experiments, drafted versions that are in review/testing environment, and completed A/B tests in case any.
2. Our experiment will be visible in the Draft section as it is not yet published. Click the experiment title and open it. Experiments can start in two ways directly: one is from the Drafted section. Then, click the start menu.

3. 

4. With this, we come to an end of our PART-2 of the Firebase: A/B testing series. In the next part, we will be covering Notification set-up, In-App messaging and exploring Multivariate A/B testing. Meanwhile, you can start implementing and reach out to us in case of additional queries at [email protected]. And we would be, as always, happy to help.