Autocorrelated Sampling in Cognition

Lucas Castillo

University of Warwick

Optimal behaviour is hard

Human behaviour is similar to the Bayesian ideal

yet given existing constraints optimality is impossible.

\[ p(H|D) = \frac{p(D|H)P(H)}{p(D)}\]

pointwise multiplication
consider data under all hyp.

MCMC: The solution from computer science

  • Algorithms that draw directly from the posterior (Brooks et al., 2011)
  • The way they sample avoids need for normalizing constant
  • Trade-off: autocorrelated
  • Metropolis-Hastings (Hastings, 1970):
    1. Start somewhere (set \(x^0\))
    2. Propose a new sample (\(x^t = x^{t-1} + n; n\sim\mathcal{N}(0, \sigma)\))
    3. Move there or not with \(p = max(1, \Phi(x^t) / \Phi(x^{t-1}))\)
    4. Go to 2

Metropolis Hastings

MCMC in cognitive models

MCMC has been a successful explanation of behavioural phenomena across many domains:

Bruckner et al. (2025) associate differences in learning across the lifespan with different number of samples drawn

Lieder et al. (2018) relate the anchoring bias to starting point effects when few samples

In Davis & Rehder (2020), MCMC explains people’s failures to integrate evidence into causal inferences

Zhu et al. (2024) incorporate MCMC into a larger model (Autocorrelated Bayesian Sampler) that can explain choices, reaction times, estimates, and confidence judgements

MCMC appraches to cognition

Sampling Website

We have compiled literature here: https://sampling.warwick.ac.uk/#Literature

Can be a bit daunting…

The samplr package

R package available on CRAN

install.packages("samplr")


sampler_mh(
  start = 1,
  distr_name = "norm",
  distr_params = c(0, 1),
  sigma_prop = .1,
  iterations = 1e4
)
expression min median
MH (1e4) 9.07ms 11.9ms

Incorporating onto your model

Bruckner et al. (2025)

MCMC Algorithms Available

Three qualitative “tricks” to improve MCMC performance:

  • multiple chains
  • gradient-based proposals
  • autocorrelated proposals
Algorithm samplr function M. Chains Gradients Autoc. Proposals
Metropolis Hastings (MH) (Hastings, 1970) sampler_mh() ❌ ❌ ❌
Metropolis-coupled MCMC (MC\(^3\)) (Geyer, 1991) sampler_mc3() ✅ ❌ ❌
Hamiltonian Monte Carlo (HMC) (Neal, 2011) sampler_hmc() ❌ ✅ ❌
RECMH* sampler_mh(alpha=) ❌ ❌ ✅
MCHMC sampler_mchmc() ✅ ✅ ❌
RECMC3* sampler_mc3(alpha=) ✅ ❌ ✅
RECHMC (Horowitz, 1991) sampler_rec() ❌ ✅ ✅
MCREC sampler_mcrec() ✅ ✅ ✅

* Use these algorithms by setting the alpha parameter to a value other than 0 in the sampler_mh() and sampler_mc3() functions respectively.

Time performance

expression min median
MH 9ms 10ms
HMC 92ms 98ms
RECHMC 100ms 104ms
MC3 104ms 110ms
RECMC3 113ms 114ms
RECMH 108ms 117ms
MCHMC 711ms 710ms
MCREC 834ms 834ms

Multiple Chains

The basic idea

Have more than one chain running at the same time (Geyer, 1991)

Other chains have higher temperature (more likely to accept)

Sometimes chains swap positions (if the colder of the two is “happy” with the new place)

Gradient-based proposals

The basic idea

Don’t make “blind” proposals, but base them on the gradient of the target density (Neal, 2011)

This increases acceptance rate without being stuck in a location.

Achieved by simulating a physical system with random momentum in each iteration. (simulate \(L\) steps of size [precision] \(\epsilon\))

Autocorrelated proposals

In MH,

  • \(\theta^{t'} = \theta^{t-1} + n; n \sim \mathcal N (0, \sigma)\)

In Gradient-based proposals:

  • \(\rho \sim \mathcal N (0, 1)\)

This can lead to random-walk behaviour which is inefficient sometimes

Instead, recycle the previous jump to get a sense of trajectory.

Features grounded in data

Castillo et al. (2024) and Castillo et al. (2026) ask participants to produce a sequence of random heights

  • Sequences best explained by an MCMC model with all qualitative features (above a random generation model; Cooper (2016))
  • Asking people to produce items as they “come to mind” shows no difference in performance

Overview

Offerings of the package

  • 8 MCMC algorithms with/without 3 qualitative features
  • Cognitive model that incorporates MCMC sampling (ABS, Zhu et al., 2024)
  • Diagnostic functions that can help distinguish between models

Documentation

  • Package is well documented with several vignettes (link)
  • Paper (Castillo et al., 2025) (just accepted! 🎉) at Behaviour Research Methods (link)
    • Qualitative phenomena MCMC algorithms have fit
    • Introduction to ABS model and how its implemented in the package
    • Introduction to simulation-based inference (to fit these models without likelihoods).
  • Recently we did a workshop on this, materials available online (link)

Thank you

Yun-Xiao Li

Adam N. Sanborn

Thank you!

References

Bramley, N. R., Dayan, P., Griffiths, T. L., & Lagnado, D. A. (2017). Formalizing Neurath’s ship: Approximate algorithms for online causal learning. Psychological Review, 124(3), 301–338. https://doi.org/10.1037/rev0000061
Brooks, S., Andrew Gelman, Jones, G., & Meng, X.-L. (Eds.). (2011). Handbook for Markov chain Monte Carlo. Taylor & Francis.
Bruckner, R., Nassar, M. R., Li, S.-C., & Eppinger, B. (2025). Differences in learning across the lifespan emerge via resource-rational computations. Psychological Review. https://doi.org/10.1037/rev0000526
Castillo, L., León-Villagrá, P., Chater, N., & Sanborn, A. N. (2024). Explaining the flaws in human random generation as local sampling with momentum. PLOS Computational Biology, 20(1), 1–24. https://doi.org/10.1371/journal.pcbi.1011739
Castillo, L., León-Villagrá, P., Falbén, J. K., Chater, N., & Sanborn, A. N. (2026). Random generation is what comes to mind in naturalistic settings. Cognition, 274, 106554. https://doi.org/10.1016/j.cognition.2026.106554
Castillo, L., Li, Y.-X., & Sanborn, A. N. (2025). The samplr package: A tool for modeling human cognition with sampling algorithms. https://doi.org/10.31234/osf.io/ax8hm_v2
Cooper, R. P. (2016). Executive Functions and the Generation of “Random” Sequential Responses: A Computational Account. Journal of Mathematical Psychology, 73, 153–168. https://doi.org/10.1016/j.jmp.2016.06.002
Dasgupta, I., Schulz, E., & Gershman, S. J. (2017). Where do hypotheses come from? Cognitive Psychology, 96, 1–25. https://doi.org/10.1016/j.cogpsych.2017.05.001
Davis, Z. J., & Rehder, B. (2020). A process model of causal reasoning. Cognitive Science, 44(5), e12839.
Gershman, S. J., Vul, E., & Tenenbaum, J. B. (2012). Multistability and Perceptual Inference. Neural Computation, 24(1), 1–24. https://doi.org/10.1162/NECO_a_00226
Geyer, C. J. (1991). Markov Chain Monte Carlo Maximum Likelihood. Proc. 23rd Symposium on the Interface Interface Foundation, Fairfax Station, 1991, 156–163.
Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1), 97–109. https://doi.org/10.1093/biomet/57.1.97
Horowitz, A. M. (1991). A Generalized Guided Monte Carlo Algorithm. Physics Letters B, 268(2), 247–252. https://doi.org/10.1016/0370-2693(91)90812-5
Lieder, F., Griffiths, T. L., M. Huys, Q. J., & Goodman, N. D. (2018). The anchoring bias reflects rational use of cognitive resources. Psychonomic Bulletin & Review, 25(1), 322–349. https://doi.org/10.3758/s13423-017-1286-8
Neal, R. M. (2011). MCMC Using Hamiltonian Dynamics. In S. Brooks, A. Gelman, G. Jones, & X.-L. Meng (Eds.), Handbook of Markov Chain Monte Carlo (1st ed., pp. 113–162). Chapman and Hall/CRC. https://doi.org/10.1201/b10905
Spicer, J., Zhu, J.-Q., Chater, N., & Sanborn, A. N. (2022). Perceptual and Cognitive Judgments Show Both Anchoring and Repulsion. Psychological Science, 33(9), 1395–1407. https://doi.org/10.1177/09567976221089599
Spicer, J., Zhu, J.-Q., Chater, N., & Sanborn, A. N. (2024). How do people predict a random walk? Lessons for models of human cognition. Psychological Review, 131(5), 1069–1113. https://doi.org/10.1037/rev0000493
Xu, F., & Tenenbaum, J. B. (2007). Word learning as Bayesian inference. Psychological Review, 114(2), 245–272. https://doi.org/10.1037/0033-295X.114.2.245
Yuille, A., & Kersten, D. (2006). Vision as Bayesian inference: Analysis by synthesis? Trends in Cognitive Sciences, 10(7), 301–308. https://doi.org/10.1016/j.tics.2006.05.002
Zhu, J.-Q., León-Villagrá, P., Chater, N., & Sanborn, A. N. (2022). Understanding the structure of cognitive noise. PLoS Computational Biology, 18(8), e1010312. https://doi.org/10.1371/journal.pcbi.1010312
Zhu, J.-Q., Sundh, J., Spicer, J., Chater, N., & Sanborn, A. N. (2024). The autocorrelated Bayesian sampler: A rational process for probability judgments, estimates, confidence intervals, choices, confidence judgments, and response times. Psychological Review, 131(2), 456–493. https://doi.org/10.1037/rev0000427