You have an attribution problem, and how machine learning can help solve it

Do people who "convert" on your website, however you define that, view more than one page before they take action? If so, you have an attribution problem — that is, you don't know which page to credit for the conversion. 

If it's just two pages, and everyone gets to the first page the same way, you can do a funnel analysis, which is probably good enough. But it's very likely the reality for your business is far more complicated than that. Traffic into your funnel probably comes from many different sources.

Or maybe you have a paywall, and you allow five free views before you present the buy or registration option. If so, to which of those five views do you give the credit for the conversion?

Consider this — what if the criteria most closely associated with conversion has little or nothing to do with which pages the user viewed? Maybe it has more to do with velocity — how many pages the person viewed in a day. 

This isn't an idle question. Let's say, just for fun, that you got 100 conversions last month, and all of them viewed five pages before paying. Ten people viewed a particular fifth page, which was head and shoulders above any other "last page viewed." Your marketing team sees that data and gets on a call with the editorial team. "We want more pages like that! That's a converting page!"

Is it? What they missed was that 35 of the conversions viewed another page somewhere in their journey — just not last. Maybe you want more pages like that one!

The more you think about it, the harder it gets. How do you know where to place the credit for the conversion, and therefore where to concentrate your efforts going forward?

It's a sticky problem because there are so many factors that might influence the visitor's decision. For example …

• Do your conversions happen more often (as a percentage of traffic) on mobile or on desktop?
• Are you counting all your online influencers — e.g., page views, emails, advertising campaigns.
• Are you distinguishing between mobile and desktop with all these influencers?
• Should you break out tablets as a third group?
• Are visitors converting on the same device type they used to view your free content, or are they switching? In which direction are they switching?
• Do you have online and offline influencers? Do you have a way to get data on your offline influencers (direct mail, newsstand, kiosks) and combine that with your online customer data? E.g., can you use promotion codes?
• Are conversions more likely when someone uses up all their free views in a shorter or a longer time frame?
• Are conversions more likely on a day when someone views three or more pages? (Maybe, rather than presenting the paywall after five views, you should present it after three page views on the same day.)
• Does time on the page matter?
• How about how they found your content in the first instance?

That's just off the top of my head. If you and I sat down with a couple cocktails and thought about this for an hour, we could easily double that list. And once we had a page full of possibilities, we might go crazy trying to figure out a method to untie this knot and decide which factor is the most important. And we'd still wonder if there isn't some sneaky correlation we never thought of.

How machine learning can help

I put this issue to Jonathan Harris, the founder of Sub(x), which provides machine learning as a service. Here's what he said. (As you can tell from the spelling, he's a Brit.) 

Most teams would accept that they are capturing only a fraction of the potential value from online customer acquisition data and analytics. People I speak with agree that the processes for managing and preparing data, analysing it, interpreting results, telling stories and actioning insights are largely manual and remain prone to bias. Alongside this, organisational challenges make it hard to extract value. Businesses struggle to incorporate data-driven insights into day-to-day processes whilst also demanding faster time to insight and response.

Importantly, there is a recognition that the problem can be tackled using Machine Learning, and as businesses like ours mask the complexity of using this technology and remove the processes that usually create friction for people, businesses can reduce the cost of unchecked hunches, wasted time and effort on marketing campaigns, and blind spots in strategy. By making the use of machine learning no more complex than the familiar tools teams use every day, it can reduce the accumulated cost of human bias in relation to data and the negative impact this has on a company's bottom line!

Q: Machine learning isn't magic. You still have to collect enough data to find which factors correlate with success. How do you make sure you're getting the right bits of information?

We've collected billions of data points from a wide range of media businesses and through this developed a single score that today is so meaningful that it can correlate back to and predict marketing indicators like click, conversion, churn, retention or customer lifetime value (CLV). Finding these otherwise hidden signals is part of the process of building effective models. It takes time.

Q: Does this solve the problem of requiring a site owner to collect all the data from their own users before they can find the right correlation? And is there a single answer, or might it vary from site to site?

As I mentored previously, building a model takes time. Our secret source is that we have engineered a universal machine learning model for media. We know that our model works and can deliver outcomes based on the unique footprint of each brand. This means that from launch to insight can take as little as a few weeks with no data wrangling or model adaptation required.

Q: What's an average lift in response rate from implementing this sort of solution?

The model is only as good as the strategy that goes into it, but broadly anywhere from 50% to 300% uplift within the first 3 months.

Q: What question have I foolishly neglected to ask you?

The question of explainability. ML should not be a black box. Importantly, any decision makers should ask, does this approach inform and help action decision-making, and how does it bring visibility to how our strategy is performing? ML can drive rapid change, but without visibility on the underlying decisions it delivers much less value to the business in the long term.

So there you have it. You're probably not going to solve your attribution problems with a better spreadsheet, but machine learning has a good track record of helping. Substantially.