A/B seeing ya! Is AI the end of split testing? This episode might just blow your mind. It's all about the future of marketing and how AI is going to revolutionise conversion rate optimisation (CRO). In the conversation, host ...
This episode might just blow your mind. It's all about the future of marketing and how AI is going to revolutionise conversion rate optimisation (CRO).
In the conversation, host Selbey Anderson's Dom Hawes meets Julian Thorne, Chief Operating Officer of Sub(x), an AI-enabled CRO business. Julian shares his journey in subscription marketing and how he became a Chief Commercial Officer in the UK's best-known publishing companies.
Julian then introduces Sub(x), a start-up that uses AI and first-party data to deliver personalised messages on client websites. Dom and Julian discuss the evolution of marketing, the limitations of traditional split testing, and the accessibility of AI in CRO for smaller businesses.
Julian emphasises the importance of AI in optimising conversion rates and delivering the right offers to the right people. He also highlights the integration of AI into existing marketing tools and the need for transparency and explainability in AI decision-making.
Julian Thorne is the Chief Operations Officer of Sub(x), a marketing technology provider that uses AI automation to drive revenue, growth and customer acquisition for digital subscription businesses.
Prior to joining Sub(x) Julian was the Chief Customer Officer of Future plc responsible for all direct customer revenues worldwide including subscription revenues. Julian has previously held various senior executive positions including, CCO of Dennis Publishing, CEO of Dovetail Services and Marketing Director at Saga. Julian also founded The Big Wheel Consultancy which specialised in providing marketing consultancy and data insight services to businesses seeking to maximise their membership and/or subscription revenues.
He is a Fellow of the Institute of Direct and Digital Marketing.
Full show notes: Unicorny.co.uk
LinkedIn: Julian Thorne| Dom Hawes
Websites: Sub(x)| Selbey Anderson
Other items referenced in this episode:
Moore’s Law (29:42)
00:00:03 - Introduction to Conversion Rate Optimization (CRO)
The episode introduces the concept of Conversion Rate Optimization (CRO) and its importance for businesses. CRO involves increasing conversion rates in marketing and on digital platforms. It is essential for businesses to improve the effectiveness of their existing assets and do more with less.
00:01:31 - Julian Thorne's Background in Subscriptions
Julian Thorne shares his career background, starting with United Newspapers and his love for subscriptions. He worked for various publishing companies and eventually became the Chief Operating Officer of Sub(x), an AI-enabled CRO business.
00:02:21 - Introduction to Sub(x)
Julian explains that Sub(x) is a startup that helps marketing teams design and deliver on-site marketing messages using a drag and drop interface. They utilize first-party data and AI to deliver personalized messages to the right audience at the right time.
00:03:18 - Julian's Learning Journey and the Evolution of Marketing
Julian shares his experience of learning on the job and obtaining a diploma in Direct Marketing. He emphasizes the importance of data in marketing and how it has transformed businesses to become more customer-centric. The relationship between marketing and finance has also improved over the years, aligning on metrics and ROI.
00:05:59 - The Evolution of Split Testing and the Role of AI
Julian discusses the history of split testing and its limitations, including human bias and the difficulty of setting up statistically valid tests. He highlights the importance of understanding audience signals and using
00:13:18 - Understanding AI and Data Points
The guest explains that their AI models analyze around 50 different data points, such as scroll depth and time of day, to determine relevant actions. The AI then serves personalized content and offers based on user preferences.
00:14:31 - Importance of Data Sets and Audience Size
The guest emphasizes the correlation between the size of the data set and audience interactions. Higher click-through rates require less audience size, while lower click-through rates necessitate a larger audience. A minimum of 500,000 unique visitors per month is recommended for effective AI implementation.
00:15:18 - Transitioning from A/B Testing to AI
Many businesses still rely on A/B testing and may be hesitant to cede control to AI. The guest suggests starting with A/B testing and gradually introducing AI to build trust and confidence. Delegating responsibility to AI requires trust in its capabilities.
00:16:41 - Transparency in AI Decision-Making
The guest emphasizes the importance of making AI decisions transparent and understandable. AI can be used to explain decisions by creating cohorts and providing data insights into audience behavior. This understanding can inform marketing strategies and content creation.
00:19:21 - Optimizing Marketing Budgets with AI
AI optimization can help businesses do more with less by reallocating budgets based on AI-driven insights. By closing the inbound traffic loop, businesses can measure outcomes and focus on ROI rather than just traffic volume. AI insights can also guide content creation and audience segmentation.
00:27:45 - Exploring Price Elasticity
The conversation delves into the concept of price elasticity and its testing in subscription businesses. Different contract lengths are tested based on customer engagement levels, highlighting the correlation between engagement and willingness to exchange a longer contract for a discount.
00:29:39 - Democratizing AI for Smaller Businesses
The discussion turns to the future of AI and its accessibility to smaller businesses. The decreasing cost of computing power and the effectiveness of AI in targeting and decision-making make it a valuable tool for marketing teams. The guest predicts the democratization of AI in the marketing field.
00:31:33 - Strengthening the Marketing-Finance Relationship
The importance of a strong relationship between marketing and finance is emphasized. The guest shares an example of working closely with a CFO to make immediate decisions based on metrics like the lifetime value to customer acquisition cost ratio. The investor community's understanding of subscription value is also highlighted.
00:33:59 - Focusing on Value Creation
The need for marketers to focus on value creation rather than activities is discussed. Lead and lag indicators are explained, with lead indicators being crucial in correlating with lag indicators, such as EBITDA. The guest shares the importance of first-time retention rate as a lead indicator in subscription businesses.
This podcast uses the following third-party services for analysis:
Chartable - https://chartable.com/privacy
PLEASE NOTE: This transcript has been created using fireflies.ai – a transcription service. It has not been edited by a human and therefore may contain mistakes.
00:03
Dom Hawes
In this episode, we're going to dive deep into a subject that should be close to the heart of every business that uses digital channels. Which means your business. I guess it also means my business. It basically means pretty much every business. Today we're talking about conversion rate optimization, or CRO. Now, you don't need a marketing degree to know what CRO is because it's pretty obvious from the name. Conversion rate optimization is the process of increasing conversion rates in your marketing and in and on your digital products and websites that could be subscribing to a newsletter or mailing list, buying a product or service, or signing up for an event or filling out a form, that kind of stuff. Now, if you think that's overly technical or maybe a little tactical, or maybe something that doesn't matter to you, I'd ask you to think again. We are all in a position right now where we're having to do more with less and improving the effectiveness of things that you already own, things you've already built.
01:00
Dom Hawes
Well, that's a pretty essential first step. Now, like all other areas of our daily work, AI is going to shake up the world of CRO. So today we're going to dive headlong into the future to give you the heads up on what your business needs to be thinking about. And we're doing that with the help of Julian Thorne, chief Operating Officer of an AI enabled conversion rate optimization business, Subx. And here's Julian explaining his career to date.
01:27
Julian Thorne
So I started life professionally with United Newspapers. I was a graduate trainee at United Newspapers. Part of my placements was to work on a magazine called Auto Express, and the publisher there, I think, had a graduate trainee and said, what are we going to give him to do? And gave me the subscription file to look after, thinking it wasn't going to be that important. And I managed to screw it up, to be honest. But nevertheless, I learned a lot and I love looking after subscriptions. I absolutely fell in love with subscriptions straight away, and that's been my career since then. I've worked for Dennis Publishing as a subscriptions manager, ended up on the board there. I then went to Saga as a marketing director. I ran a subscription fulfillment business for BBC and Denis as a joint venture. And then I set up a consultancy specializing in recurring revenues and eventually became CCO of Dennis Publishing again, and then Future Publishing when Dennis was sold to Future.
02:21
Julian Thorne
So subscriptions has been my background and subscription marketing specifically.
02:25
Dom Hawes
Just for context, why don't you give me a quick elevator pitch for your business, Subx?
02:29
Julian Thorne
Subx is a startup business. We've grown pretty rapidly in the last year or so. We enable marketing teams to design on site marketing messages on their own website in one design platform. So a pop up, a slide in a fixed position, an exit intent, all designed using a drag and drop interface, which is very easy to use. But the clever bit is that we use first party data underneath and integrated into that design platform to get the right message to the right person at the right time. Marketeers create messages, and then our AI framework delivers the right message to the right person at the right time, based on the understanding of what that user engagement is on the site.
03:11
Dom Hawes
When you started marketing, how did you get to learn the basics? Because not everyone gets, like, career development tools or assistance.
03:18
Julian Thorne
No, too true. So, learning on the job at the beginning, but then I did a diploma in Direct Marketing. It was then called, run by the Institute of Direct Marketing, now called the Institute of Data and Marketing, which says a lot about how marketing's progressed. And that was I'm giving away my age now, Dom. That was pre internet age. So I had to go to lectures, real people in real rooms after work. And I have to say that was really hard work, doing a job and doing a postgraduate diploma, but it was well worth it. So having that core basic training, academic training, I've relied on that ever since. Still use it to this day.
03:59
Dom Hawes
Fantastic. And there's a link there. Back in the day, optimization really meant Direct, because that was the only I mean, the clue is in the name, right? That was the only way you had of going Direct. And the world seemed a lot simpler, I guess, in your experience across your career, and I suspect mine started around the same time, how do you see businesses approach to marketing has changed over those years?
04:20
Julian Thorne
I think a lot of it's down to data. When you and I maybe first started, the amount of data that was available to marketeers was relatively little free Internet. Now there's a vast amount of data, which means when you're a marketeer, you can tell a lot more about your audience, and that ultimately has led to better ROI. It also means that you understand your customers better. So as a result of that, I think marketeers are becoming more and more the voice of the customer in businesses. And businesses have, over the years, moved from maybe product centric to customer centric, which means marketing has come more to the forefront. The other big change that I've noticed is that marketing and finance are working together these days. They're working on the same metrics. When I first started, finance were the enemy, and I think it's probably fair to say finance thought marketing were just wasting money.
05:12
Julian Thorne
That's been a big change. So we're often talking about the same metrics, the same ROI, same LTVs, and we're talking to finance people that are well trained in marketing disciplines as well, at least understanding their Ro, why that's a big shift. And I've noticed that in the last five to ten years, boom.
05:29
Dom Hawes
God, you don't know how happy I am that you've actually just said that. I'm going to come back and explore that sentiment a little bit more right at the tail end of today's show, because I'm really interested in the relationship between marketing and finance, and I don't meet many people who have the same opinion as you. But before we do that, want to cast your mind back to when we very first met, and I was talking to you about split testing, and were talking about how AI was going to be the future. It was going to revolutionize conversion rate optimization. I suppose before we start digging into CRO and AI enabled CRO in more detail, we probably just need to look at split testing and the history of it a little bit. So, Julian, maybe you could talk to us about that a little bit.
06:12
Julian Thorne
So split testing or a B testing, as it's often referred to. I first came across that in the world of direct mail, where you would have an audience or a list and you would split the audience and send one pack to 150 percent and the other to the other 50%. To work out what offer was the best offer. There's nothing wrong with that. But that is dependent upon the fact that you don't know much about your audience. You buy a list, you know where they live, you know their name, but other than that, you know relatively little when it comes to presenting offers on websites where you've got a whole load of data, then you can start to target that based on the individual user behavior. The issue with A B testing is that what do you test? So who decides what you're going to test?
06:55
Julian Thorne
It could be the highest paid person in the room. It could be human bias. Almost certainly is going to be human bias when you set the test up. It's actually quite complicated to set up an A B test that's statistically valid. It can take time. The result can take time. And I've been in numerous meetings where the result is questioned at the end. So you're going to go, I've done an A B test, and at the end someone said, I don't believe it. And there might be just some justification in that because there's human bias attached. I've also seen, and this is probably a good point just to mention the 40 20 rule that I first came across in the diploma, which is 40% of the response you might get to a particular message is dependent on the audience. 40% is dependent on the product and the price, the customer value proposition and the price, and only 20% is dependent on the creative.
07:45
Julian Thorne
Designers hate me for reminding them of this, but it's true. It's just borne out by the stats. More often than not, an A B test will feature a creative test, because that's human bias. I want to test red against yellow. I don't know if you remember. But every button now is green because for some time ago he did a test, but ultimately an A B test. Human bias, but it's inefficient. I'll give you an example. One of our clients, and I'm simplifying this, but one of our clients did an AB test on a website testing a three month subscription offer that was 30 pounds against a one year subscription offer, which gave a 10% discount. And they did an AB test and they calculated the lifetime value and they rolled out with the three month offer, which did have the best result on the OB test.
08:35
Julian Thorne
But the result of the OB test suggested there was plenty of people there prepared to pay for a year, just not as many. If you could find the people prepared to pay for a year, as well as the people prepared to only accept a three month offer, then you're going to increase your total return from that audience. But you need to understand the signals that suggest that those are the people that are going to take the year offer and the signals of those people that are going to accept the three month offer. If you find those signals and AI helps you do that, then you get the best of both worlds. You can offer both offers to the right person at the right time and increase the overall CRO from that audience.
09:08
Dom Hawes
It's tempting to think, because AI is the thing that everyone is talking about, that AI is brand new. And of course it's not. It's been around for a very long time and it's already built into many of the products that we use. But I think what you're doing well, certainly to me it seems very new and it seems to evoke some of the challenges that any new technology has that you need to appeal to maybe an early adopter or an innovator type audience. Is that what you're experiencing?
09:35
Julian Thorne
I'm probably just going to question the question a little bit there. So when I was running marketing teams not so long ago, less than a year ago, I joined Subx a year ago for large publishing companies. Typically, I had the team split with top of the funnel teams and teams marketing on our own websites. The top of the funnel teams, those principally spending money with the social networks, spent most of their time thinking about customer value proposition and pricing and creating a lot of different messages and then pushing them out onto Facebook and Instagram, particularly Instagram. There was no A B testing going on there. Those messages were being delivered to the right audience at the right time by AI instagram. We know that. And that's been going on for many years. It's not new. And I don't think there's any coincidence that the innovators in large language models that have recently come to the fore principally come out of the social networks, out of their laboratories because they've been using AI for a long time, so marketers have been using that.
10:34
Julian Thorne
But I was very struck by the time we sent people to our own website. All of a sudden we're back to A B testing, we've got someone coming in from Facebook, let's do an A B test on offer. So the AI's got a lot of publicity recently for all sorts of reasons, but it's been around a long time in terms of CRO. It just hasn't been available to smaller companies and that's where Subx is coming in.
10:56
Dom Hawes
But still, you must be, I guess, like, everyone's got to have a beachhead. So you're probably targeting a compelling need in a particular sector. Where are you targeting?
11:05
Julian Thorne
We're targeting media businesses at the moment, but our ambition is to move into anyone selling digital subscription products. And the reason for that is that the challenge is quite a tricky one. So the challenge is a lot of audience engaged with a lot of media content, but the amount of interactions are relatively sparse. So the number of subscriptions you might sell, the number of email newsletters you might capture, the number of people who register to your site or pay a paywall. It's a relatively small number compared to the vast audience. So AI is perfectly suited to that complex problem of saying how do we find the signals for that relatively sparse piece of data?
11:44
Dom Hawes
When we met, you talked about two things which AI is really good at. I guess it's how the large language model works. In large language models, it's next best word, I guess, like, how are we going to string a sentence together? What's the most likely next word? But in your case, it's next best action.
12:00
Julian Thorne
Yes. Next best offer and next best action.
12:02
Dom Hawes
Can you just explain those to me a little bit?
12:04
Julian Thorne
Sure. So if someone comes to your website, typically, and I'm sure you've witnessed this yourself, you arrive on a website and within a matter of seconds, you've been asked to buy something where you haven't been nurtured in any way. You've just been presented straight away with, do you want to buy something that might not be the next best action for you? It might be the right next best action for you, depend on your history with the site or maybe where you've come from. But AI helps marketeers to deliver the right message to the right person at right time based on that person's behavior and interaction with the brand and their behavior and interaction with the brand compared to other people that look similar or have the same sort of behaviors against that particular offer.
12:45
Dom Hawes
Is the technology like aggregating behaviors to create cohorts or does it work at the individual level?
12:51
Julian Thorne
It's done at an individual level and it's done in real time and also historical data. For some brands, we find that historical data is a very good predictor of conversion rate, and for others, what they're doing right now on the site has a slightly higher weighting. Every brand's different as you would expect.
13:10
Dom Hawes
Yeah. Okay. And I guess also things like the journey they take and how quickly they move between pages and how quickly they move between activities on a site may be good indicators.
13:18
Julian Thorne
Exactly. So we only use first party data. We look at typically around 50 different data points. So there'd be ones such as scroll depth, as you've mentioned, time of day, whether we've seen them before, lots of stuff that make perfect sense, nothing magical about it. And looking at all those data points. And then the AR models we use, and we use a variety of different models depending on the challenge. The AR models we use work out which of those data points are most relevant to an action, to taking an action and not taking an action.
13:53
Dom Hawes
Okay. And then serve unique content to that user based on what you think or what the AI thinks they want to do next.
13:58
Julian Thorne
Serve the right offer. Not unique, but the right offer from the offers that the marketeers are prepared to make to their audience.
14:04
Dom Hawes
All AI needs to be trained on data. And I'm guessing that in your case, the data sets need to be quite large in order to spot the correlations between behavior. Is that the case?
14:15
Julian Thorne
Yeah, I mean, it's the correlation between visitor numbers, so the amount of user data you have and the amount of interaction with the offer. So clicks, if you've got a very high number of clicks, you need less audience. If you've got a low number of clicks, you need a bigger audience. So it's back to the old statistically valid calculation we used to use in direct mail, but as a very broad ballpoint. 500,000 plus uniques a month gives us enough data to work with.
14:46
Dom Hawes
But I guess not everyone's going to have that amount of traffic. So those that don't have it or don't have it yet probably are going to be relying still on split testing, I guess, or other more traditional tools.
14:58
Julian Thorne
I would agree. The traditional stuff, because you haven't got statistically valid data for the AI to effectively beat random. So then, yeah, A B testing still has a role there.
15:09
Dom Hawes
What about more traditional risk averse businesses? Do you think that some businesses may worry about effectively ceding control of their conversions to technology? Do you feel or do you meet people who still prefer to have that kind of hands on, intuitive approach?
15:23
Julian Thorne
Yeah, people have been trained in A B testing coming back to Finance. Finance understands a b testing. Everyone understands a b testing. It's not a difficult concept to grasp, this idea that you test two things and there's a winner and a loser. As I've already talked about with AI, there's not a winner and a loser. There's just the right offer to the right person. But there's a slightly more sophisticated story that needs to be told for people to understand that. But no, you're absolutely right, Don. There is your expression, seeding control is spot on. We talk about delegating responsibility to AI, which I think is quite a neat way of saying because you don't delegate something unless you trust the person you're delegating to. So what we typically see is that with our clients, they start with they use our platform to conduct a traditional A B test and then ironically, they test AI against the A B test.
16:15
Julian Thorne
So they're running an A B test with the AI and then as they see that performance improvement from AI, then they start to trust, which means you can then delegate, which, using your terminology, they can seed control, but they do that willingly. But asking someone to go straight from having years and years of A B testing to go, trust us, this box is going to work, doesn't work. The other thing that we're really keen on is making it not a black box. So being able to explain the decisions, the AI is working and at that point we have to talk about cohorts. We deliver at an individual level, but when we explain what's going on, we put people into cohorts so we can explain what the AI is doing. That's particularly useful because it gives great data insight into what audiences are doing and how they're behaving.
17:03
Julian Thorne
But it also enables the marketers to understand a broad level what the AI is doing.
17:08
Dom Hawes
And I guess to some extent, if that's helping them segment and create cohorts, that's going to help them nudge their outbound marketing in the right direction too.
17:17
Julian Thorne
Yeah, spot on. So I'll give you an example. So we might be looking at those 50 data points I talked about earlier, and we might be saying to a client, right, let's have a look at the top five data points that are most correlated with whether or not someone takes an action. So you might be looking at those top five and you might say, actually looking at this is quite interesting here. People that have come to the site based on click through from your email newsletter are showing a far higher propensity to engage with you further and maybe pay you some money. And as a marketeer, you take that away and say, might be a good idea to build our email newsletter list up. Actually, it might be a good idea to recruit an editor for that and build that product out. So that's an example of where the data insights that you can glean from the AI can then practically be used to develop your business.
18:06
Julian Thorne
And to your point, you might well look at it and say, actually look at our Instagram visitors, they're doing very well, we should spend a bit more money.
18:15
Dom Hawes
This is on site optimization, obviously, so it's not going to help you with your paid social or your PPC at all.
18:21
Julian Thorne
We can take data around our audiences on the customer's website and use that to inform PPC as a data point. So you've got you close the loop, as it were saying, okay, these are the people that are showing particular high levels of engagement and then feed that back into the way that you buy your audiences.
18:41
Dom Hawes
Okay, I guess that's going to optimize over time also significantly, presumably. And actually, given that your media spend is probably one of the larger parts of your budget, it's going to help you save quite a lot of that as well, I would guess.
18:53
Julian Thorne
Exactly. I mean that's back to our previous conversation about finance. So why are you spending more money with a third party when you can't convert that audience when they land on your website? Because you're relying on a B testing. If you can turn around and say, look at the increase that AI is delivering when that audience arrives, then that frees up more budget from the finance director to then grow your business.
19:14
Dom Hawes
I keep hearing this phrase and I therefore keep saying it a lot, but everyone I've meet at the moment is being told to do more with less. So that sort of optimization is kind of bang on trend. But what I hadn't figured until we started talking was that onsite optimization and the cohorts you're able to create, particularly if you've got decent attributes, you know where the traffic is coming from and you can then spot themes in how they behave. If you then can change your outbound activity, this could be quite a quick route to doing more with less.
19:43
Julian Thorne
Exactly. It's often around content as well. Where do we invest our content creation on the site? What content is really engaging with people? There's usually a correlation between engagement and cash, but not always. We do see on some sites, people highly engaged with content but never prepared to pay for it. And other sites highly engaged with particular content and very willing to pay. So it's those sort of insights that AI uncovers and brings to the fore which are very valuable for getting that ROI that we talked about before.
20:17
Dom Hawes
Okay, let's take a quick breather there and recap because we have covered one hell of a lot of ground. Now, my main feeling from what we've just been talking about is that while traditional methods of conversion rate optimization like A B testing have their place, and they probably will do in the future too, the future of marketing, especially in digital spaces, is increasingly going to be intertwined with the capabilities of AI. I mean, why wouldn't it be? And when we come to optimization, I mean, firstly, AI moves optimization into real time and that represents a benefit leap over more traditional methods. It offers the ability to both analyze and predict behavior by combining vast volumes of historical and live data points simultaneously. And that is something that was, I mean, just previously inconceivable and it allows for hyper personalization tailoring content or offers to individual users instantaneously based on their interactions.
21:18
Dom Hawes
AI is also eradicating the limitations of human subjectivity. It minimizes bias that often cloud judgment with like, a more traditional approach to split testing. And as an added bonus, AI driven insights can inform and elevate other facets of your marketing strategy, from content creation to audience segmentation. Yada yada. Which kind of leads me neatly on to my next point, which is all about closing the inbound traffic loop. Julian talked to us about how, as well as helping to optimize user behavior on a website, that kind of technology can also help with third party strategies like pay per click campaigns and content creation, as I've already mentioned. And he gave us two very specific examples where AI was able to correlate and improve performance from two different traffic sources. And in one of those examples, he spoke about increased engagement and increased spend potential with traffic that was coming from an email newsletter and how that was enabling a business to commit more resource to developing that channel and guiding its content creation.
22:28
Dom Hawes
Now, if you can do the same across all of your inbound channels, measure, and test conversion methods, then you can reallocate your media budgets based on outcomes, not inputs. I-E-I mean, how much money are you making as opposed to how much traffic are you getting? That's huge. That closes the loop on your media spend and it gives you a link, and we're going to talk about this a bit later, I think in part two, it gives you a link to be able to go back to your CFO and talk about value. Now, there's loads more I could comment on, but I'm keen to get back to Julian because there is more to learn. Let's dig into a little bit more detail the shift from sort of traditional, more traditional split testing towards AI tools. Is it affecting the way businesses understand user engagement?
23:16
Julian Thorne
Absolutely, yeah. Because as we talked about in the first part of the show, dom AI will uncover, looking at those 50 or so data points that I talked about before, how audiences are behaving and be able to surface that probably more importantly, whether or not that behavior is valuable. I'm using an example from banking. There's plenty of people who've got a bank account and the banks don't make any money from them, and there's plenty of people that visit websites where you don't make any money from them. So being able to understand the audience behaviors and then being able to make decisions that maybe promote those behaviors more, AI will give you that understanding because it understands the correlation between customer behavior and engagement and whether or not they interact with your marketing messages.
24:04
Dom Hawes
When someone integrates these kind of tools, are they having to get rid of a lot of other tools, or are they integrating? Are you seeing a blended approach from people?
24:12
Julian Thorne
Yeah, that's a good question. I'll answer that in a minute. But it's probably worthwhile talking about AI and how it's sat and data scientists and those data science teams and where the future might be going here a little bit. Typically the larger clients that we work with have data science teams already in house. They unearth some really fascinating stuff, looking at data that's available to that business. The problem typically is how do you actually use that insight? How do you connect it to anything? How's it turn from being a sort of academic discussion in a boardroom going that's fascinating, to actually doing something? AI is inbuilt into our platform, which is a design platform to create marketing messages, but AI is built into it to deliver the right message to the right person at the right time. And we see that with other businesses as well.
25:00
Julian Thorne
We've seen it over the years where AI is built into the tools, it's not sort of defined as a separate thing. It's built in. So when you buy instagram campaigns, you are buying instagram's AI, but it's not flagged up, you're just buying media. And that's how we perceive that's going to continue to happen. So AI will be built into the tools that marketeers use, rather than a separate sort of data science function that sits there going, well, there's just some fascinating insights here, but how do we actually use it?
25:34
Dom Hawes
So someone taking Sub X in are basically putting a product in and the product's got output. They don't need to worry about what's happening under the bonnet, I guess.
25:42
Julian Thorne
No, but they're interested in what's happening under.
25:44
Dom Hawes
I bet they are. I mean, if for no other reason than an understanding of people and data and an understanding of the correlation points between allows you to get some predictive capability too.
25:54
Julian Thorne
That's right.
25:54
Dom Hawes
How might a business that does have enough traffic, that is using those kind of tools, be taking the data that you've got and making scenarios that give them a view of the future?
26:04
Julian Thorne
What our platform is built to do is essentially predict whether or not someone's going to interact with a marketing message and it uses historical real time data to make that prediction. As you and I both know, your current behavior is a predictor of your future behavior. Sometimes I'm sure we all wish it didn't, but it is. So we've increasingly developed our models to be able to predict future behavior with a reasonable degree of accuracy. Obviously, we go back and back test it to see whether or not our prediction was correct. On top of that, we're able to see the impact on that prediction of presenting a message. So if we present a message, do we diminish future engagement or do we increase future engagement? Now, as a subscription marketeer, that is gold dust. Because if you can predict future engagement with a reasonable degree of accuracy, then.
26:54
Julian Thorne
That in itself is a lead indicator of retention and retention is core to the whole subscription business. More so being able to talk to our clients and say, you can target people on your website or our AI will target the right message to people on the website where if they do go on to subscribe their predictive future engagement is high, then that gives the marketeers the confidence to go to the finance department and say, I think I can predict the LTV on this spend. Lifetime value on this spend. So I can talk to the finance department and say I want to spend this amount of money acquiring a customer with a degree of confidence about how much money you're going to make back from that customer in the future. So what we do all the time is predict behaviors when we present a message. But increasingly we can predict behaviors post that.
27:45
Dom Hawes
Okay, that's really cool. And what about things like price elasticity? I guess we talked earlier about putting the next best offer in front of someone. I guess that's probably how you test.
27:57
Julian Thorne
Price because we work with subscription businesses we test contract lengths. So as I explained in the earlier example, a three month contract length versus a one year contract length versus a two year contract length you'll be familiar with those SaaS based software as a service based pricing mechanics. Moving pricing around for the same product is dangerous. For one customer it's five pounds. The same product is ten pounds. That's not fair on customers and leads to problems in terms of price discipline and control. But testing contract length is very useful because it's correlated to engagement. But really bluntly, if someone's quite engaged with your product they'll be prepared to exchange a longer contract length for a small discount. Whereas if someone's trying it out not so engaged they probably want a shorter contract length with a higher price per month, week, day usage, whatever it might be.
28:56
Julian Thorne
And finding those differences in the audience preparedness I suppose to pay for a longer contract length or a shorter contract length finding that has a significant impact on the bottom line performance from that audience as opposed to an A B test where you just offer the winner.
29:12
Dom Hawes
The cases we're talking about, they're not necessarily niche because actually the market you're addressing is big and I assumed at the start of this show that were earlier in the lifecycle of this technology than of course we are, but we're still not mass market are we, on this kind of technology? Certainly on tools that enable real time decision making and then optimize a user experience in real time. How long do you think that sort of capability is away from SME type websites? Yeah.
29:39
Julian Thorne
You're asking me to predict the future.
29:40
Dom Hawes
Yeah, I know I said I wouldn't.
29:42
Julian Thorne
Do that but I lied. So with all the caveats about not being able to have a crystal ball and all that sort of stuff. So we've thought about this. I think what we see in our more grandiose moments is democratizing AI, giving AI to marketing teams in smaller businesses, not just a domain of big tech. And I think that might be a function of Moore's Law, which I'm not sure if you're familiar with, but the concept of the cost of computing power diminishing every two years or halving every two years. And the rate of it doubling every two years. All of a sudden, being able to have the processing power that AI tools demand and the speed the cost of that has diminished. And that gives access to smaller businesses and makes it cost effective. On top of that, as with general discussions around AI, and we talked about the cost of a B testing.
30:37
Julian Thorne
A B testing takes people in order to target stuff on a site, people need to make manual decisions and people cost money. AI is better doing that and costs less. So you end up with this rather nice combination of reduced cost where your marketers can concentrate on doing stuff that they're good at customer value proposition, product development, marketing, as opposed to targeting, which is a very technical element, and allowing AI to do that job for you. So your costs come down, but your return on investment goes up because AI is more effective at doing it. You mentioned Chatbots earlier that's exactly the same business proposition, which is we can answer this question better using AI than using a person. Therefore we're delivering a better customer service, less cost. It's the same broad principle. Where is AI operating as an agent, as it were, in your team to increase the productivity of that team?
31:33
Dom Hawes
I wanted to come back to the relationship between marketing and finance because for a long time, particularly in B, two B, and actually it's still there. I still get it. You go to a conference these days and they'll talk about they're still talking about siloed departments like sales and marketing being siloed. They're still talking about marketers not talking the same language as the rest of the business. I don't know if that's true. Like you mentioned earlier, I think that really proficient, savvy marketers already speak finance, and I think the CFO and the CMO probably ought to besties.
32:05
Julian Thorne
I completely agree. When I worked for Dennis Publishing, it was owned by private equity before it was sold successfully. The private equity guys obviously understood the relationship between marketing spend and creating future value. And I worked very closely with the CFO as the CCO. We were besties, actually, not personally, but in business terms. We got to the situation and all credit to the CFO, he's a great guy where we could ring him up on a Friday afternoon and say, we're seeing some great results here. We want to increase spend over the weekend. And he would literally talk our language and say, what's the LTV CAC ratio, lifetime value to customer acquisition cost ratio. We'd say it's above three and he'd say, just spend it and that'd be great. I mean, that is just brilliant because you can then respond immediately to what's going on in the market.
32:55
Julian Thorne
A lot of that was during COVID when stuff was changing all the time, behaviors were changing all the time. So yeah, to your point, the reason why the CFOs are so interested in marketing and there's an obvious reason why marketing is interested in the CFO because source the money. But the reason why the CFOs are interested is because the investors increasingly understand the return on marketing spend. So us as a startup, when we talk to investors, they talk that language as well. They say, what's your sales motion? What's your ABM strategy? What's your scalability? What's your LTVCAC ratio? Where do you start counting the lifetime value? Do you start counting it during the trial period or know detailed stuff? And this is coming out of, I think increasingly the investor community understanding subscription value because that's driven by Netflix and Spotify. Amazon didn't make a lot of money for a long time because building an LTV model is creating future value.
33:50
Julian Thorne
The CFOs are starting to understand that rather than typically looking at a PNL in a year, they're thinking, what value am I creating? Because marketing doesn't deliver in the month, does it? You spend it in that month, you don't get it back in that month. Yeah, that's called sales. Whereas marketing creates future value in order to justify that spend. The CFOs need to understand that. And they do because the investor community understands that.
34:14
Dom Hawes
But I think also with subscriptions, certainly the conversation we've had, I guess it covers B, two B SaaS as well, because that's all just another type of subscription business. But where I've seen there be a disconnect, I think it's where the marketing teams are too focused on the metrics around their outputs rather than the outcomes the business cares about, like lifetime customer value or cost of customer acquisition. They're too obsessed about engagement or the success of a particular like an individual activity rather than the success of the business. And I think that's probably part of the disconnect. And in those circumstances, I've seen very clearly the CFO becomes the person who's accountable for value delivery. And what we're trying to say through this podcast and our other channels is that marketers need to think about value creation, not activities. No one cares about activities. We only do them because we want to create value.
35:05
Dom Hawes
And I think there is a breed of market that focuses too much on the outputs and not the outcome.
35:10
Julian Thorne
I'll challenge that a little bit.
35:11
Dom Hawes
Go on then.
35:12
Julian Thorne
So it's the concept of lead and Lag indicators. So Lag indicator is how much money did I get? And the CFO is always interested in that. The score. In other words, lead indicators are how do you get to that score? So the example I always use is one from Amazon, which is they concentrated, so they say primarily one lead indicator, how quickly can we get a product to someone when they brought it? And their whole business was focused on that to the point where they sort of promoted drones and know but the whole business is focused on how quickly can we get it there. Because they knew that there was a very strong correlation between that lead indicator and making money. Just turning around to someone and say, can you make more money?
35:49
Dom Hawes
How yeah.
35:50
Julian Thorne
So understanding the lead indicator, so what you're talking about the outcomes, as long as they're very closely correlated to the.
35:57
Dom Hawes
Lag indicator and there's the key, it just has to be correlated. And I think in many cases they're probably not. So you might be interested in this, Julie. We spoke to Margaret Franco, who when we spoke to her was CMO At, a financial technology giant. Now, sadly, by the time were ready to go to air, she had changed companies. So actually her interview never saw the light of day. But we are hoping to get her back in the studio because she was amazing. You spoke with such great clarity. Anyway, look, she told me that she only has two KPIs. She uses EBITDA and engagement. And what she has managed to do is correlate the engagement in her pipeline to future EBITDA. And she's got loads of data, by the way. So those are really the only two things that she talks about when she goes to talk to her CFO.
36:41
Dom Hawes
And so when her CFO comes to her and goes, okay, where are we on engagement numbers? He or she will then be able to use that as a confidence metric against future EBITDA. And I just thought that's kind of really cool. But if you don't have enough big data yourselves, maybe I suppose you probably.
36:59
Julian Thorne
Can'T correlate in a subscription business, the equivalent of that lead indicator is first time retention rate because that reflects the health of the entire marketing stack. If you're acquiring customers who don't stick, you're acquiring the wrong customers. If you're acquiring customers that should stick but don't, then you've got the wrong product. So that's a very strong lead indicator as a single KPI, which informs how much money you're going to make. And it focuses people's attention on if you look after that number and that number is going the right way, you'll make more money.
37:29
Dom Hawes
Wow, what a great interview. If you've listened to any of my recent episodes, you will know that I'm obsessed at the moment with value creation and mainly, I guess, because of where we are in the economic cycle optimization or doing more with less. Now, today we got into the weeds of a particular product that's serving a particular niche, that being subscriptions, but understanding the power of the technology that's being applied to this market and then asking like, what can you do? Or who is there that can help you optimize your website traffic in that way? Who is there that can help you optimize and create value for yourself? That is all of our challenge right now. At the halfway point, I mentioned the benefits the real time data analysis could bring and how technology like this can also close the Attribution loops with your inbound traffic sources.
38:26
Dom Hawes
And I'm going to highlight a few things that I'm going to take away from today's chat. And as usual, we'll list a timeline and summary on the show notes at Unicorny Co. UK. And by the way, you can also find a little bit more analysis in my weekly newsletter on LinkedIn. It's called marketing difference. I'm going to link that as well on the show notes. But today look, let's look at today. I was really captivated by Julian's point about AI being built into the tools. I think at the moment we're all a little bit obsessed with the power of generative AI because we're able to interact with it directly through tools like Chat, GPT, Copy, AI, Fireflies, Jasper and the rest. Now, I think optimization AI is more mature than generative AI, and that's kind of why it's being built into products and that simplifies its usability and that's what's starting to open it up to more mainstream markets.
39:16
Dom Hawes
And that plays into Julian's observation that this kind of AI is democratizing AI for small business. Like, up till now, AI tools and technologies have been the domain of big tech and large corporations. But the changing cost dynamics and the rapid advancement of technology are enabling even small businesses to utilize AI's power. And this democratization has the potential to revolutionize the way we work. It could level the playing field. It could allow small businesses to compete much more effectively. And I absolutely love that. Now, I also really enjoyed listening today to Julian talk about the relationship between CMOS and CFOs. We discussed the Time Horizons marketing works to and when realistic returns might be anticipated by a business. Julian talked to me about lifetime customer values and cost of customer acquisition, specifically what the jargon calls the LTVCAC ratio, or lifetime value to customer acquisition cost ratio.
40:18
Dom Hawes
Now, of course, in your business, if you're not already doing it, you too should be able to calculate the lifetime value of your customers. If you can then measure the cost of your acquisitions, book a meeting room, sit down with your CFO, and work out what ratio works for your business, then you have a shared metric, a shared understanding of what it takes and what it costs to create value in your business. Now, you may have a lower volume business than publishing, but the same rules should apply. I'm no great fan of the term ROI or return on investment. When it comes to marketing, there are loads of reasons for this. I could do a whole show on it if you're interested, but one of my main reasons is because it's not time bound and therefore leads to short termism, but lifetime cost to value ratio.
41:07
Dom Hawes
I think that hits the nail on the head perfectly for me. And I think I'm going to go away and do some work with our own CFO and senior finance people on just this. Remember, in the messy world of metrics and marketing, it's seldom about having the answers, it's about asking the right questions. And with that, I'm going back to the unicorny forest to forage for more insight. See ya.
COO
Julian Thorne is the Chief Operations Officer of Sub(x), a marketing technology provider that uses AI automation to drive revenue, growth and customer acquisition for digital subscription businesses.
Prior to joining Sub(x) Julian was the Chief Customer Officer of Future plc responsible for all direct customer revenues worldwide including subscription revenues. Julian has previously held various senior executive positions including, CCO of Dennis Publishing, CEO of Dovetail Services and Marketing Director at Saga. Julian also founded The Big Wheel Consultancy which specialised in providing marketing consultancy and data insight services to businesses seeking to maximise their membership and/or subscription revenues .
He is a Fellow of the Institute of Direct and Digital Marketing.