Commanders Act X
Platform XDocumentationWelcome to Success
  • Welcome !
  • Platform updates
    • Announcements
    • Documentation updates
    • Release notes
  • Getting started
    • How the platform works
      • Glossary
        • Condensed platform concepts
    • Platform interface
      • Prod and Testing environments
      • Productivity tools
        • Commander's AI
    • Integrating your data
      • GTM Tutorial
      • OneTag Tutorial
      • Migration guides to the Platform X
        • Migrate from SSv1 to SSv2
          • Send data to serverside v2
            • Purchase event example (ssv1 to ssv2)
        • Migrate from old mobile sdk
  • Features
    • Sources
      • Sources Overview
      • Sources Catalog
        • Web
          • Web container
            • User guides for browser-side platform
              • Container
                • Hosting
                • Creation and modification
                • Generation
                • Testing
                • Deployment and roll back
                • Deletion
                • Statistics
                • Modification history
                • Javascript block
                • Branches
                • Plugin Commanders Act Assistant
              • Tags
                • Add tags
                • Configure tags
                • Rules
                  • Basic actions
                  • Triggers
                  • Perimeters & constraints
              • Data layer and data types
                • External variables
                • Internal variables
                • Event variables
                • Data storage
              • Deduplication
                • Setup guide
                • Setup example
                • Deduplication reports
              • TagPerformance
                • Setup guide
                • Report analysis
                • Troubleshooting
            • Setup guides for developers
              • Web container setup
              • Datalayer setup
              • Browser-side events setup
              • AMP
              • Angular
              • AngularJS
              • React
              • SPA implementation guide
              • VueJS
              • IOT & TV Apps
            • Best Practices
              • FAQ
              • Common Container Strategies
              • Common Trigger Strategies
              • Performance Optimization
              • tC.* attributes and methods
              • APIs
                • Onsite API
              • TMS & Consent banners IDs
          • Javascript SDK
            • Next.js serverside rendering
          • Pixel Tracking API
          • Google Tag Manager (GTM)
          • Shopify
        • Mobile APP
          • Android
          • iOS
          • Flutter
          • React native
        • Advertising
          • Bing Ads (cost import)
          • Facebook Ads (cost import)
          • Criteo (cost import)
          • Realytics
        • Import CRM users
          • API users
          • Users file importer
        • Import conversions
          • API Conversions and Product catalog
          • Conversions files importer
        • Product catalog
          • Product catalog files importer (FTP)
        • Server
          • HTTP tracking API source
            • (deprecated) HTTP tracking API source 1.0
          • Node.JS
          • Python
          • PHP
      • Source Live Event Inspector
      • Source data quality
    • Destinations
      • Destinations overview
        • Automatic Audience replay
      • Destinations catalog
        • AbTasty
        • Actito
        • Adform
        • Adobe
          • Adobe Analytics
          • Adobe Campaign
        • AdRoll
        • Adition
        • Adventori
        • Affilae
        • Alphalyr Marketing Studio
        • Amazon
          • Amazon Ads Conversions API
          • Amazon S3
        • Attraqt
        • Awin
        • Batch Audience
        • Branch Events
        • Button
        • Campaign Analysis Legacy
        • Commission Junction
        • Criteo
          • Criteo - Events
          • Criteo (audiences)
          • Criteo (offline conversions)
        • Data Activation Legacy
        • Dataventure
        • Destination Logs Exporter (closed beta)
        • Dialog-Mail
        • Dynamic Yield
        • Easyence
        • Effinity
        • Eloqua
        • Emarsys
        • Email export
        • Equativ Audience
        • Experian
        • Facebook
          • Facebook Conversions API
            • Facebook CAPI through GTM
            • Performance tab (Event Match Quality)
          • Facebook Custom Audiences
          • Facebook Lead Ads
        • FTP
        • Gamned
        • Google
          • Google Analytics 4
            • Google Analytics 4 - Proxy Mode
          • Google BigQuery
          • Google Conversion Adjustments
          • Google Customer Match
          • Google Display & Video 360
          • Google Enhanced Conversions
          • Google Enhanced Conversions for Leads
          • Google Floodlight Mobile App Conversion
          • Google Search Ads 360 Enhanced Conversions
          • Google Store Sales Direct
        • IBM
        • Inxmail
        • IntelliAd
        • Jellyfish
        • Kameleoon
          • Kameleoon Audience
          • Kameleoon Events
        • Kelkoo
        • Kwanko
        • LinkedIn Conversions API
        • Liveramp
        • Magento
        • Marin Software
        • Mapp
        • Matomo
        • MediaMath
        • Microsoft Advertising Universal Event Tracking
        • Mindlytix
        • Moebel
        • Nextdoor Conversion API
        • OXID
        • Optimizely
        • Outbrain
        • Partnerize
        • Piano Analytics
          • Piano Analytics Collection API
          • Piano Analytics Enrichment API
        • Pinterest
        • Piwik PRO
        • Prediggo
        • Qlik
        • Quantcast
        • Quora Ads Conversion API
        • Rakuten
          • Rakuten Audience
          • Rakuten Events
        • Realytics
        • Reddit Conversions API
        • Responsys
        • RhythmOne
        • Rich Relevance
        • RTB House Audience
        • Salesforce
          • Salesforce Audience Studio
          • Salesforce Marketing Cloud
          • Salesforce Commerce Cloud
        • SAP Commerce Cloud
        • Selligent
        • Skai
        • Smart Adserver
        • SmartFocus
        • Snapchat Conversions API
        • Splio
        • Syte
        • Tableau Online
        • Taboola
          • Taboola Audience
          • Taboola Events
        • Target2sell
        • Temelio
        • Teradata
        • The Trade Desk
          • The Trade Desk Conversions API
          • The Trade Desk Audience
        • TikTok
          • TikTok Events API
          • TikTok Offline Events
        • TimeOne
        • Tradedoubler
        • TradeTracker
        • X (Twitter) Conversion API
        • Xandr
        • Webhook
        • Webtrends
        • ZBO Media
        • Zeta
      • Destination builder
        • Javascript destination builder
          • Tutorial - How to build a server destination with the JS sandbox
          • Serverside javascript helpers
      • Destination filters
      • Mapping and Properties transformation
      • Event delivery
      • Destination event inspector
      • Dry mode (lab)
    • Enrichments
      • Augmented User Attributes
        • Business case
      • Events enrichment
      • Storage Settings
    • Data Quality
      • Event Specification
      • Sources data quality
      • Data cleansing
        • Supported transformation functions - Data cleansing
          • Format a date
    • Identity resolution
      • Migrate from Fuse v1 to Fuse v2
    • Customers
      • Segment
        • Segment overlap
        • Segment stats
    • Explore
      • Campaign analysis
        • Attribution
        • Control Group (Closed Beta )
      • User analysis
        • Dashboards
      • Consent Analysis
    • Consent management
      • Responsability of actors
      • Setup Guides
        • Tag Manager
          • Commanders Act TMS
          • Google Tag Manager (GTM)
          • Google Tag Manager (GTM) - Consent Mode
          • Google Consent Mode in Commanders Act CMP
          • Adobe Launch
        • Websites (Hardcoded)
        • FR : Suppression des cookies lors du retrait du consentement
        • Mobile apps
          • iOS
            • ATT - App Tracking Transparency (iOS 14.5+)
          • Android
      • User Guides
        • Categories & Tags
          • Manage Categories
          • Manage Vendors
          • Assign Categories
        • Privacy Banners
          • Banner Templates
            • Accessibility Template
          • Manage Banner
          • Deploy Banner
          • Copy Banner
        • Consent Analysis
        • Exports
        • Settings
      • Extensions
        • Cookie Scanner
        • Piggybacking
        • Tag Firewall
      • Marketing Preferences Center (additional module)
      • Knowledge Base
        • Consent Object
        • Consent cookies exemption
          • Implementation guide for exempted consent statistics FR market
        • Consent Cookie
        • IAB TCF V2.2 Release details
          • IAB TCF v2.2 CMP requirements
          • IAB TCF v2.2 Migration guide Web
          • IAB TCF v2.2 Migration guide App
        • IAB TCF V2.2 Consent
        • IAB TCF V2.2 and Google FAQ
        • Google ACM requires IAB TCF
        • CCPA & Global Privacy control
      • Rest Data API
        • GET/PUT Consents / preferences
      • OnSite API
        • Getting Started
        • consent.get
        • consent.update
        • consent.revoke
        • consent.onUpdate
        • consent.onReady
        • consentBanner.show
        • consentBanner.hide
        • consentCenter.show
        • consentCenter.hide
      • Platform API
        • Get statistics
  • Use cases
    • Data activation
      • Engage new customers
        • Welcome banner for new customers
        • Real-time promotion for hesitant customers
        • Discount banner for installing the application
        • Personalized ads
        • Engage similar audience (lookalike)
      • Increase loyalty
        • Drive to favourite store
        • Increase Customer Lifetime Value with a loyalty program
        • Notification about order delay
      • Increase revenue
        • Abandoned cart
        • Products recommendations
        • Complementary product offer
      • Retain customers
        • Identify a churn risk with RFM segmentation
        • Identify customers’ preferred channel
        • Contact with the customer support
      • Advocacy
        • Incentive to share customers' experience and rating
        • Sponsorship Program
        • Social Media Hashtag
    • Website performance
    • Consent banners A/B testing
    • Customer analysis
    • Campaigns performance analysis
  • Developers
    • Tracking & Integrations
      • Tracking
        • About events
          • E-commerce/retail events
          • Web event specificity
          • Mobile App event specificity
        • Events reference
          • Common events
          • E-commerce events
          • Video events
          • Campaign Tracking events
        • Properties reference
          • Global properties
          • Video properties
          • Permanent properties
        • Data API
          • HTTP API
          • Segment API
          • User API
          • Product catalog and conversion API
      • Server IP Whitelisting
    • Config API
    • Changelogs
      • Measure.js changelog
      • Web container generator
    • Content Security Policy
  • CONFIGURE
    • Data Management
      • Events collection
      • Data retention duration
      • Data Governance
    • Administration
      • User management
      • Domain Management
        • WAF Proxy (CloudFlare,...)
        • A record
        • CNAME record
        • On-Premise Proxy
        • Cookie CAID
        • First party hosting
      • Single Sign-On
      • Two-factor authentication (2FA)
      • Copy Management
    • Cookies
      • Cookie 1st
      • Cookie sync partners
      • First domain tracking (Phoenix)
    • Disclaimer
Powered by GitBook
On this page
  • What is a control group?
  • How it works?
  • **** Limitations
  • How can I read the results?
  • Incremental revenue
  • Conversion rate
  • Average Revenue Per User (ARPU)
  • Average basket amount
  • Conversion rate evolution
  • Radar
  • Metrics
  • Can I export the control group?

Was this helpful?

Edit on GitHub
Export as PDF
  1. Features
  2. Explore
  3. Campaign analysis

Control Group (Closed Beta )

PreviousAttributionNextUser analysis

Last updated 2 years ago

Was this helpful?

What is a control group?

A control group is very helpful to measure the performance of a campaign. A control group is a small group of users (a percentage of users from associated segments) which is willingly not exposed to an ad or an email in order to measure the difference in terms of engagement or conversion between the population not exposed (control group) and the population exposed (users in the associated segments).

In our case, we take into consideration only conversions for now (online and offline purchases).

How it works?

A control group is setup on a stream (share menu). Check the box ‘Set control group’ to activate the possibility to configure a control group.

Specify the % of users you want to set for your control group: these users will be not pushed on the stream (and so not exposed to the campaign).

10% is generally enough to have statistically significant metrics, however for very small segments it is recommended to double this %.

As soon as you start the stream, the control group will be launched also, and results will be recorded.

  • be careful with the modification of streams with an active control group as it impacts the results on the campaign performance dashboard. Same with segments used in a stream with an active control group, if you modify a condition it impacts all the results.

  • When a stream is over you cannot relaunch it, you should duplicate it

How can I read the results?

When a stream with a control group is activated, on the stream list page an icon will appear on the right of the stream’s name.

Click on this icon to access to the analytics part.

First, select the period of analysis: you can choose for example to read the results on the last 3 days or last week.

All the figures will be related to this analysis period.

Definition of each metrics:

Total population: number of users in segments which were selected on the stream. It is equals to Audience activated + Control Group.

Audience activated: number of users that were pushed on the stream (and exposed to the campaign). It is equals to Associated segments – control group.

Control Group: number of users in the control group, it represents the % of users selected to be in the control group and not exposed to the campaign. It is equals to Associated segments – audience activated.


Cards

You have here some metrics related to conversions.

We compare the performance between the users exposed and not exposed in terms of conversions (online and offline purchases).

In details:

Incremental revenue

Incremental revenue represents the campaign impact on your revenue.

All starts with a supposition: if we consider the full population in the control group what are the performances, and what is the incremental revenue generated by the campaign?

This incremental revenue could be positive or negative, and it represents the campaign performance and impact.

In details, we consider the full population of users (= Audience activated + users in control group) and we apply control group results for Conversion Rate and Average Basket Amount to all these users: \

Then, we can easily compare the revenue generated by the Audience activated and the simulated revenue generated by the total population as control group and see if the incremental revenue is negative or positive.

Formula:

Incremental revenue = (Revenue generated by the Audience activated + Revenue generated by the Control group) - (simulated Revenue generated by the total population as control group)

Conversion rate

For each population, we calculate here the conversion rate over the period selected.

Formula:

Conversion rate = number of conversions ÷ number of users

(we compare the conversion rate calculated for the population ‘audience activated’ and for the population ‘control group’)

Uplift (% difference for the conversion rate)

Represents the impact of the campaign. As we compare users in the activated audience and users in the control group, we have to determine the impact of the campaign in terms of conversions.

If the uplift is low or negative, that means users in control group have good results without the exposure to the campaign and so the campaign is not as efficient as it should be.

On contrary, if the uplift is high, that means the campaign is over performing because users exposed to the campaign convert more than users in the control group, the impact is very positive on purchases.

Formula:

Uplift = ((conversion rate ‘Audience activated’ - conversion rate ‘Control group’) ÷ conversion rate ‘Audience activated’) x 100

Average Revenue Per User (ARPU)

We compare here the revenue per user for each population (audience activated and control group).

Formula:

Average Revenue Per User (ARPU) = Total revenue ÷ number of users

(we compare the ARPU calculated for the population ‘audience activated’ and for the population ‘control group’)

Average basket amount

We compare here the average basket amount for each population.

Formula:

Average basket amount = Total revenue ÷ number of conversions

(we compare the average basket amount calculated for the population ‘audience activated’ and for the population ‘control group’)

Charts

Conversion rate evolution

You can see here the conversion rate evolution day after day for the two populations: the audience activated and the control group.

This chart is cumulative in order to reflect more the impact in terms of users, exposure and conversions, that means for each day conversions are added.

Radar

You can visualize directly here the comparison between the two populations (audience activated and control group) for the following metrics: ARPU (Average Revenue Per User); Average Basket Amount; Conversion Rate. It allows you to analyze quickly which metrics are very impactful and relevant to compare the populations.

Metrics

You have here a summary of all metrics needed to compare the populations, such as the number of users and number of conversions (needed to calculate the conversion rate for example). The turnover is based on total_final_conversion_amount, meaning it could evolve depending on products returns, for example.

Can I export the control group?

Yes, you can select an export (email, S3 or FTP for example), and select the universe 'campaign':

You can select 1 segment or choose 'all users' and you can export the following variables:

  • Control Group: does the user belong to the control group? True / False

  • Exposure count: number of exposure to campaign for a user = number of times a user entered a segment and was pushed through the connector

  • First exposure: date of first exposure

  • Last exposure: date of last exposure

  • Stream ID: ID of the stream for which the control group was activated

**** Limitations

⚠️