[MUSIC PLAYING] Hi, everyone.

I'm Anna Seza.

I'm the founder and CEO of CreativeX.

This is Josh.

He's the VP of data at Diageo.

He's going to tell you more about himself.

We are probably one of the only folks not talking about GenAI.

We're going to talk about data and what you could do with fun data applications.

I looked up at ChatGPT this morning as to what are the factors that predict whether or not people will think you've given a good presentation.

And you got a bunch of the same banalities.

One of the things I never heard is it said you should tell the audience three things you're going to share with them.

And then you should share those things and everyone will think you've done a good job.

So I've never done this before.

We're going to try it.

So what you're going to get is you're going to get a grammar lesson about the difference between intra and intradata.

I'm going to reveal to you what Noah actually thinks about all of you, because he told me when we were preparing for this talk.

And Josh is going to share some cool data that puts some more insight into the wear-in-wear-out challenge.

We're also going to pace like crazy New York cats, because we are caffeinated and didn't want to sit.

So over to you, sir.

OK, hi, everybody.

So Josh from Diageo, one of our goals is something called creativity with precision.

And what that means is how are we using data and insights to-- I would say paired with creative flair to actually drive the business forward.

And we have some of the most iconic brands on the planet.

Hopefully all of you enjoy them later during happy hour.

Ultimately, there's some challenges to being one of the largest alcohol companies on the planet.

One of them is pace and scale.

Everybody needs a lot of creative yesterday.

That is one major challenge.

Number two is big global orgs, terrible at communicating to each other, lots and lots of silos.

So learning across what works and what doesn't, damn near impossible.

And then finally, best practices.

There's a best practice for Meta, YouTube, Facebook, TikTok, websites, every single one of them.

And they're all slightly nuanced.

And those algorithms like to mess with you if you do not follow their rules.

So there's a lot of challenges that happen along the way.

Ultimately, the first question that we started with was, how do we ensure creative fit?

And you're probably going, what the hell does creative fit mean?

Number one would be fit for platform.

We know for a fact that if you don't play by an algorithm's rules, it is going to penalize you.

If you play by its rules, it's going to show it to more people.

That's just kind of how it works.

So one is platform fit.

Are you within spec?

Do you actually have a logo?

Do you have way too many words within an image?

Number two would be fit for purpose.

Is the creative actually, I would say, driving to some place where you want it to drive to?

Is it supposed to drive awareness?

Is it supposed to drive conversion?

Some type of purpose for the creative rather than just creative.

And then the last piece would be fit for the person.

We talk about it as reach and consumers and all this other stuff.

At the end of the day, is it relevant?

Is it in some way, shape, or form personalized?

And are we actually talking to a person at scale?

Or are we talking to lots of people and just talking to ourselves ultimately?

There we go.

Why does this matter?

It's good for business.

So 49% seems a little bit low, but ultimately, creative is responsible for about 49 probably higher percentage of sales.

You can be an expert at a lot of parts of execution, but creative matters a lot.

Number two is, again, the algorithms.

They will penalize you or they will reward you in terms of more reach for less.

About 10% depends on the platform.

It can go higher.

So there you have that.

Ultimately, Diageo's goal is maximize every dollar spent, or in my case, pounds, and leave nothing on the table is ultimately what we like to get to.

So we partnered with CreativeX.

And the purpose behind that was, how can we get a better understanding of our creative fit?

And ultimately, there's a lot of different ways that you can do that.

Spoke a little bit about them of taking a look at our creative and going, hey, what is within an aspect ratio?

What is having enough of the-- I'd say too much copy, brand, things like that.

And ultimately, produce a creative quality score.

Now, the really interesting part about all this is it produced about a database of a little over 400,000 ads and counting.

And within there, it allowed me to start a little war within my company.

So it was, for instance, Ireland versus Brazil.

What is your creative quality scores?

Johnny Walker versus Guinness.

Bottles were broken, mayhem everywhere.

Ultimately, it started asking more questions of, what else could we do with the data besides just creative quality?

So just the first thing to think about it is, first, we did something from a pre-flight standpoint.

So we upload it and we go, hey, how is this doing with regards to creative quality score?

Then we'd finally go to Final Asset, publish it, and we'd go to in-flight and get the similar ones.

Now, the interesting bit about this in the two words in red is, what do you do with a combo of those things?

You know what you tested, and you know what you ran.

What happens if maybe it doesn't run, as an example?

So it allowed us to do an analysis of, what other questions could we ask?

And ultimately, this all ladders up into something that Diageo lovingly and hatingly calls Conscious Create, which is our goal to shape our, I would say, marketing investment overall.

How do we reach or provide the experience that is appropriate for our consumers, that exceeds all of the expectations?

How do we maximize that, rather than just spending a lot of time talking to ourselves or setting money on fire?

So now I'm going to pass it over to Anastasia.

There you go.

All right.

So don't you love it when one of your partners comes to you and says, hey, you know this thing your product does?

We wanted to do something else.

So can you do that instead?

But Josh usually has good ideas.

So we're like, cool.

Let's entertain this.

And so his idea was, look, historically, you focused on telling us what's inside the ad.

But really, what we want to do, given that you can track the content creative agencies are making for us, and what's actually ending up in flight, is help us think about how we measure activation and usage.

Again, are we making the kind of content that's actually getting used in market?

But I'm a little bit of an overachiever.

So I thought, what else could we do with this data?

And so we started actually thinking about, well, if you can match the stuff that your agencies are making with the stuff that's running, can you get too much richer layer of what we broadly think of as creative attribution?

And this is where that grammar lesson from the sixth grade finally sunk in.

And I started to understand the difference between intra and inter.

Historically, when we think about creative data-- and this probably comes to every computer vision demo you've ever seen-- you talk about the stuff in the ad.

This is a bottle.

There is a grapefruit.

There is a hand, et cetera.

And that data is useful for understanding what's in the content.

When we started to get really interesting is when we started to think about inter-creative data and looking at different images and how they're similar or different.

And it started to allow us to get new types of data that we never had before.

It started to enable us to automatically understand, hey, who's the agency that made this ad, without having to ask people for an advance?

It started to help us understand, is this ad an original or an adaptation?

And if it's an adaptation, what kind of adaptation type is it?

Is it a translation?

Did someone tweak the size?

Did someone add some new copy to it, et cetera, et cetera?

So all of a sudden, we had access to this new type of creative data.

And so we started thinking about, well, what do we do with this data?

So we took-- historically, when we think about asset measurement and creative measurement, because of the way that digital works, we think about creative measurement at an asset level.

Like, hey, this ad did well or it didn't do well.

But that's inherently very limiting, because most ads are just permutations of a core concept.

So we started to think about, well, what if we could take an ad and pair it against everything that was being run in market, and actually start to understand, of everything that's being run, what are the activations and adaptations of this concept, creating what Noah lovingly called, the first time I showed him this, a creative family tree, where you can really start to understand the parent and all of its permutation children.

And this is where I asked Noah, hey, Noah, tell me about the audience.

And he's like-- his one advice was, overestimate the audience.

They are much smarter than the normal people who go to these conferences.

So this gives me permission to geek out for a little bit.

So basically, what's happening in the background here is, for every single image or video that we get, we run it through neural networks to generate something-- the easiest way to think about is a fingerprint, right?

And we do this for hundreds of thousands of images that our brand partners are running.

So you get, basically, these fingerprints for every single ad in the system, whether or not they're tested pre-flight, and your creative agency's made them, or whether or not they're running in flight.

And then what you're doing is you're basically creating a similarity calculator, right?

And you're looking at, hey, what is the similarity between these two different assets, and where does it score?

The easiest way to think about it is a giant database, which has got infinite rows that are running, and looking at the different similarities between all the various images you have and grouping them together.

And then where we had to get creative is think about, OK, what is the threshold at which we consider things similar or dissimilar?

And one of the product decisions we've made is we actually gave brands control over the similarity threshold and said, you can decide what you consider an appropriate similarity threshold.

And here, of course, each dot represents a similarity between a pair of images or videos.

The red dots are the false matches.

The blue dots are the true matches.

Here you can see we've set the similarity threshold 0.5.

You've got two errors that sneak in there.

And no AI system is perfect, but obviously the job here is to get as much as possible while minimizing the errors.

And so here you can see that it's recognizing images that are different in color, that have different text, different languages, logos in different places, et cetera, but it understands it's part of the same core concept.

And this is where we think it gets really interesting from a usage and activation point of view.

Video is obviously a lot more complex.

You're comparing every single still across millions and millions of videos.

But this also allows you to see whether or not your static image assets are being used inside videos to give you an even broader family tree of how your creative is being used.

So a lot of different comparisons, a lot of work.

Humans can do this or would not want to.

It's a very boring job.

And so Josh's prompt about a year ago led to the development of this new data application that we call Creative Lifecycle.

And really, what Creative Lifecycle allows us to do is figure out how much of the content that you're creating is actually being used as market.

But it also allows us to calculate essentially the shelf life and the footprint of that content.

How often is the content being taken and adapted, permutated?

Is it having global reach, or is it just being used by one market?

It's allowing us to answer things like the global local problem to figure out whether content from some markets is being used by others, et cetera, and getting to this much richer area of creative attribution in an area where we're treading very carefully.

But because we can understand who made the core concept, we can start to measure creative effectiveness beyond an ad level at a conceptual level.

So what is the outcome of what we're talking about here?

And ultimately, it's a conversation around where in and where out and whether or not an asset was actually used.

We actually found that through this, most of the assets are never used or activated.

So again, if global is producing a lot of assets or a local market's producing, that's a lot of duplication.

That's a lot of money that isn't going towards actually reaching consumer and providing them with an experience.

And the reason why more and more content was being produced or is produced is because there's the assumption that it is worn out.

Consumers need to see something new.

There needs to be a new communication out there.

We found the opposite.

It's not even worn in.

So it's a near constant conversation around what does conscious creation actually look like going forward so that we don't create an entire content landfill within our organization.

And that allows me to reinvest and, as of right now, project it to be about $100 million back towards marketing being able to do more things rather than just producing content.

Thank you.

[APPLAUSE] [MUSIC PLAYING] (upbeat music)

‍