Swarm Acquisition Success with Branch Journeys

journeys winners.png

In June/July, the growth PM and I discovered the most cost-effective acquisition strategy for Swarm we’ve ever seen: a cross promotion with Branch Journeys pop-up modals on Foursquare’s mobile web.

We ran 2 experiments, one testing value propositions on the home page, and one testing different types of pop ups (pill, half page, full page) across venue pages. (See what they looked like in the image above.)

Here’s the experiment breakdown:

Homepage Multivariate Experiment

The goal of this experiment is to see which of the two value props are the most effective at encouraging users to download Swarm.

Audience: Mobile Web only, US only

Duration: 30 Days

Control: No Upsell (10%)

Test Group 1: Check-in Value Prop (45%)

Test Group 2: Travel Value Prop (45%)

Venue Page Multivariate Experiment

The goal of this experiment is to see which of the three cross promo styles prompts are the most effective at encouraging users to download Swarm.

Audience: Mobile Web only, US only

Duration: 30 Days

Control: No Upsell (10%)

Test Group 1: Full Page (30%)

Test Group 2: Pill (30%)

Test Group 3: Half Page (30%)

Winners:

Homepage: Test Group 1 - check in (45%)

Venue Page: Test Group 1 - full page (30%)

Branch Stats.png
Branch Installs by Experiment.png

Forecasting ROI

Experiment 1

Total Installs from Winner: 144 (45% of viewers)

If we get 144 installs from 45% of mobile web viewers, then assuming equal performance, we could get 320 installs from 100% of viewers.

Experiment 2

Total Installs from Winner: 461 (30% of viewers)

If we get 461 installs from 30% of mobile web viewers, then assuming equal performance, we could get 1,536 installs from 100% of viewers.

Total, if Journeys runs on both home page and venue pages: 1,856 monthly installs.

1856 x 12 months = 22,272

Total, from the experiment as is: 1,073 installs.

1703 x 12 months = 12,876

Total Cost for Journeys: $24,000 (12 months, paid quarterly - 4 installments of $6000)

Projected CPIs

Based on Winners’ Performance: $1.07

Based on Experiment Totals: $1.86

Conclusion: Journeys could result in a CPI of $1-$2, the most cost-effective acquisition method we’ve seen at this scale so far. Also, if performance of the winning version remains consistent through a whole year, this method will account for a 48% lift in annual installs, benchmarking against the average amount of installs we see from paid acquisition monthly, and projecting that over the course of a year.

The Next Phase

In October, I created a new venue page experiment testing different value props with the full-page modal and one of our best-performing photos on social media.

Screen Recording 2018-10-19 at 04.35 PM.gif

The tests ran for two weeks: Oct 19-Nov 2.

Here were the results:

Journeys 2.png

With the highest click to open rate, and click to install rate, and nearly tied for bringing in the most installs, “Keep Track” is the best version. If we got 114 installs from 30% of viewers, assuming equal performance, we could see 380 installs over 2 weeks from 100% of viewers.

Projected over time, this version could yield 760 installs monthly, and 9,120 annually.

However, this is not as efficient as the original Venue Page experiment run, which would yield 1,356 installs monthly.

Conclusion: “Looking to check in?” is the still the most effective at acquiring new users.

Retention

In summary, Venue Page Modal v2 performed the best, having the best average active days since signup across the users enrolled, followed by Home Page Value Props v2 having the 2nd highest average active days since signup.

Across the three campaigns, Home Page Value Props v2 10/12, Swarm 6.0 Value Props - Venue Page, and Venue Page Modal v2 10/12, there really was not a significant difference in day 0 retention. Around 47%-50% of users who installed the app would drop off on the day of installation for all of these campaigns.

However, Venue Page Modal v2 campaign (avg 4.5 days active) showed the greatest success in average user retention followed by Home Page Value Props v2 (avg 4.1 days active) and then Swarm 6.0 Value Props (avg 3.2 days active).

Journeys 3.png

Examining the figure above, Home Page Value Props v2 and Swarm 6.0 Value Props performed similarly across the first 5 days. Around 75% of users would drop off after the 5th day. On the other hand, Venue Page Modal v2 churned around 70% of its users on the 5th day and performed better than the two campaigns in user retention.

However, Home Page Value Props had a better retention rate where 92.5% of users would drop off on the 17th day compared to Venue Page Modal v2 at 94.4%.

Erika Finnimore