Bayesian Estimator Assessment Methods for Minimizing Costs in Multivariate Driving Performance Studies

Document Type

Presentation Abstract

Presentation Date

3-24-2014

Abstract

Reduced alertness and high levels of cognitive fatigue due to sleep loss bring forth substantial risks in today’s 24/7 society. Biomathematical models can be used to help mitigate such risks by predicting quantitative levels of fatigue under sleep loss. These models help manage risk by providing information on the timing at which high levels of fatigue will occur; countermeasures can then be taken to reduce accident risk at such critical times.

Biomathematical models of fatigue predict cognitive performance based on homeostatic and circadian processes. Such models have typically been fitted to group average data. Due to large individual variation, group-average predictions are often inaccurate for any given individual. However, individual differences are trait-like. Between-subjects variation can therefore be captured by individualizing model parameters. These parameters may be estimated using the technique of Bayesian forecasting to combine new individual data with prior distributions that have been pre-specified using population data. In many cases the amount of data collected on the individual at hand, and consequently, the prediction accuracy, will be limited by factors such as the availability of data and cost of collecting it. However, prediction accuracy may be improved by including information from alternative, correlated performance measures in a multivariate Bayesian forecasting framework. Investigation of this latter technique is the topic of this thesis.

When collecting data from two performance measures, we consider how to minimize the cost of data collection while meeting a desired average level of prediction accuracy. We extend a commonly used measure of prediction accuracy, the mean squared error (MSE), by integrating over observed data values to create a uniquely determinable accuracy measure for specific parameterized Bayesian models with fixed data collection strategies. We call this new measure the marginal MSE. We derive the marginal MSE of the prediction accuracy for a general Bayesian linear model.

To understand how the marginal MSE depends on the number of measurements from primary and secondary tasks in the simplest case, we specify the accuracy for the bivariate Bayesian linear model of subject means. For this simple model, we further assume that observations from each performance measure have a fixed cost per data point, and use this assumption to determine the number of measurements of each variable needed to minimize the cost while still obtaining no less than the desired level of accuracy.

To aid the extension of the findings from the linear case to state-of-the-art nonlinear biomathematical fatigue models, we focus on obtaining our extended measure of accuracy for the nonlinear case. Computing this accuracy analytically is often infeasible without reliance on model approximations. Model simulations can be used to compute the accuracy; however, such simulations can be time consuming, especially for models that lack analytic solutions and require that a system of differential equations be solved to produce model dynamics.

Much of this computational burden in assessing estimator accuracy, however, is produced by using the Bayesian minimum mean squared error (MMSE) estimator, and could be reduced by taking advantage of the more rapidly computable Bayesian maximum a posteriori (MAP) estimator. We show for a nonlinear biomathematical model that the accuracy assessment using repeated simulation with the MAP estimator yields a reasonable estimate of the accuracy obtained using the MMSE estimator.

Still, for any given case, determination of whether the MMSE accuracy can be approximated with the MAP accuracy requires these time-consuming simulations. We begin to analytically identify classes of models where the MMSE accuracy can be approximated by the MAP accuracy. We consider a class of quadratic Bayesian models, and show by analytic approximation that for this class, the MAP has twice the marginal MSE of the MMSE.

Additional Details

Doctoral Dissertation Defense. Link to the presenter's dissertation.

Monday, March 24 at 3:10 p.m. in Math 103

This document is currently not available here.

Share

COinS