Look for measures underpinned by good, not PR
Our persistence to understand measurement is real and it takes time. We’re not like some others, where rapid commercialisation of advertising impact measures is more important than the stability of the construct on which they are based. Take attention measurement, for example. If you have read our work in the past two years you will have seen our commentary on the attention economy (commonly referred to as the new black of advertising measurement). Research sellers (and media owners) are now scrambling to integrate attention measurement into their commercial bag of tricks. And rightly so, attention is a vital pre-cursor to advertising impact. However, many of these measures fail to tick all (or any) of the golden rules of measurement. Meanwhile unsuspecting advertisers are reeled in by the fancy PR.
The golden rules of measurement for advertisers to consider are:
The quality of the construct: Quality measurement of marketing data needs to begin with the measurement of the right constructs. They need to be grounded in theory and demonstrated by way of empirical evidence to be linked to how marketing affects buyer and consumer behaviour. Such constructs also need to be measurable. Ideally, empirical evidence supporting their use should be generalised across many contexts (eg. time, countries, product categories) so that it can be applied by marketers across the range of conditions in which they are marketing.
The quality of construct measurement: Choosing the right construct is one thing, but how that construct is measured is another. For example, in the assessment of advertising creative executions, researchers have shown that the emotion triggered by an advertisement is important for its effectiveness. Emotions as a construct can be measured in many different ways and there are a number of commercially available emotion software development kits that use Ekman’s 6 emotions as their basis. But what if, after all this time Ekman’s list of emotions was shown to be wrong or needing modification? All of a sudden, the quality of the measurement construct is in doubt, as well as is its capacity to be used to evaluate effective advertisements.
The quality of the collection: Marketing data, like any data, can be affected by poor collection methods. Issues include: one-off findings that cannot be generalised, confounding factors that lay at the heart of claimed results not being controlled for, and the poor management of human bias. These issues have been part of the fabric of the market research industry for many decades. Perhaps the most classic example of bias in traditional market research practices is the reliance on human memory. It’s hard to remember what you did yesterday, let alone the advertising you saw. And guessing what behaviour you might express if you were in a buying situation is even further away from the reality of a 5pm stop at the supermarket with two cranky toddlers. Diary based collections, recall metrics and prompted intention to buy, all carry methodological biases that can skew results and be misinterpreted. The future of measurement is less about asking people what they think they saw, and more about passive data collection techniques with actual observation of behaviours in their natural environment.
We love the idea of advertisers learning more about measurement at a deeper level. If you base fundamental marketing decisions on fundamentally flawed measures, you start the race handicapped. We want advertisers to feel confident enough to ask questions about the rigour behind the measures used and request research papers that openly describe the methodological construct and process. This will determine whether the measure is built to be truly meaningful for brand growth.