Validation
Each model is checked against held-out human data - eye-tracking recordings for attention, people's emotion labels for emotion. These tests measure agreement with typical human responses in research settings, not campaign performance.
What it is
A predictive model is only useful if its predictions match what people actually do. Validation is how we check that: each model is tested on human data it never saw during training, so it can't simply have memorised the answers.
How the models are tested
Attention. The attention model's predicted maps are compared with real eye-tracking recordings for images that were kept out of training. The comparison uses the standard measures from attention research, which check whether the places the model predicts actually match where people looked, and how closely the overall pattern of attention agrees.
Emotion. The emotion model's predictions are compared with the emotions people reported for images that were kept out of training. The tests check both whether the model identifies the emotion people named and whether it gets the overall tone, positive or negative, right.
Complexity. The complexity model's predictions are compared with people's ratings of how visually complex held-out images look.
Results of internal tests
In Emotiva's internal tests on held-out research images:
- Emotion. The model names the same top emotion as people about 8 times in 10, and the right one is among its top two about 98% of the time. It gets the overall tone (positive or negative) right more than 9 times in 10.
- Attention. The predicted maps agree closely with real eye-tracking recordings: on the standard measure for telling looked-at areas from ignored ones, the model scores about 0.90 out of 1.
- Complexity. No single published figure yet.
These are research-setting results on general imagery, not advertising, so they describe agreement with typical human responses rather than campaign performance.
What validation shows, and what it doesn't
These tests measure agreement with typical human responses in research settings. They show that the models reproduce, on new images, the patterns people show in eye-tracking and emotion studies.
They don't measure campaign performance. A good attention or emotion prediction is not, by itself, a prediction of recall, click-through, sales or brand lift.
Validation also reflects the material the models were tested on:
- The models learned mostly from general imagery rather than advertising, so treat results as predictions.
- Results describe a typical viewer, averaged over many people.