Results · feature modelling · SIMULATED

Can stimulus features predict which changes are missed?

Nested feature sets, two grouped cross-validation schemes, three model families. Base rate 37.8% over 1674 trials, 49 participants, 252 stimuli. Semantic terms are a-priori importance and animacy; no embeddings in v1. Splits: GroupKFold(5).

01 Model comparison

Low-level features add the most; semantics add a little more

AUC by feature set and split — **AUC by feature set.** *Logistic regression, GroupKFold(5) by participant and by stimulus.*

Out-of-fold metrics
split	feature set	model	features	AUC	log-loss	Brier	C-index (latency)
held-out participants	baseline	logistic	9	0.801	0.521	0.173	—
held-out participants	baseline	gbm	9	0.787	0.540	0.179	—
held-out participants	baseline	weibull_aft_latency	9	—	—	—	0.803
held-out participants	low_level	logistic	19	0.845	0.469	0.152	—
held-out participants	low_level	gbm	19	0.831	0.495	0.159	—
held-out participants	low_level	weibull_aft_latency	19	—	—	—	0.837
held-out participants	semantic	logistic	11	0.818	0.503	0.165	—
held-out participants	semantic	gbm	11	0.820	0.499	0.163	—
held-out participants	semantic	weibull_aft_latency	11	—	—	—	0.814
held-out participants	combined	logistic	21	0.847	0.467	0.150	—
held-out participants	combined	gbm	21	0.830	0.496	0.160	—
held-out participants	combined	weibull_aft_latency	21	—	—	—	0.838
held-out stimuli	baseline	logistic	9	0.795	0.533	0.176	—
held-out stimuli	baseline	gbm	9	0.766	0.572	0.189	—
held-out stimuli	baseline	weibull_aft_latency	9	—	—	—	0.786
held-out stimuli	low_level	logistic	19	0.842	0.474	0.154	—
held-out stimuli	low_level	gbm	19	0.827	0.498	0.163	—
held-out stimuli	low_level	weibull_aft_latency	19	—	—	—	0.826
held-out stimuli	semantic	logistic	11	0.811	0.514	0.168	—
held-out stimuli	semantic	gbm	11	0.797	0.527	0.173	—
held-out stimuli	semantic	weibull_aft_latency	11	—	—	—	0.795
held-out stimuli	combined	logistic	21	0.844	0.471	0.152	—
held-out stimuli	combined	gbm	21	0.828	0.495	0.162	—
held-out stimuli	combined	weibull_aft_latency	21	—	—	—	0.829

In-sample H4 check: adding semantic importance to the low-level logistic model gives LRT χ²(1) = 16.525, p = p = 4.8e-5, AIC 1568.26 → 1553.73.

02 What carries the signal

Permutation importance and calibration

**Predicted vs observed detection.** *Decile calibration (line) and stimulus-level means (dots) under each split.*

03 Difficulty

Stimulus-level difficulty

Stimulus difficulty — **Every stimulus, by type.** *Detection rate across the participants who saw it; marker size ∝ end magnitude.*