Cognitive load and fatigue in low load physical activities such as computer work or e-sport

Author
Afshin Samani

Sport Sciences – Performance and Technology, Department of Health Science and Technology, Aalborg University, Niels Jernes vej 12, 9220 Aalborg East, Denmark. Tel: 00 45 99402411. Email: afsamani@hst.aau.dk

 

Even at force levels of mostly below 10% of the maximum muscle force (e.g. computer work in the context of e-sport), prolonged low load physical activity may contribute to the development of musculoskeletal disorders (MSD). Fatigue is often regarded as a precursor of developing MSD. However, identifying fatigue development and designing effective interventions to impede fatigue in such activities have been under-studied. The problem becomes even more complex when the physical activity is concomitant with cognitive loading as the interaction of cognitive and physical aspects of an activity may alter the course of fatigue development. This article summarizes a set of three interconnected experimental studies which aimed at quantifying cognitive load alterations and fatigue during a standardized computer work and outlining the design of a smart fatigue alarm based on characteristics of oculomotor system.

Background

Prolonged low load physical activity such as computer work may contribute to the development of musculoskeletal disorders [1]. Such activities may develop fatigue [2] and fatigue is known as the precursor of developing MSD [3]. Concurrent physical and cognitive loading may exacerbate the fatigability and further intensify the adverse effects of fatigue [4]. Implementing a pausing regime during physical activities may reduce the adverse effects of fatigue on the human body [5]. To intervene in the pausing regime, fatigue should be monitored and the pause breaks should be implemented with proper timing [6]. However, the typical metrics of fatigue have not been fully successful in monitoring fatigue state in low load physical activities (e.g., computer work) [2]. In a set of three interconnected experimental studies, the potentials of utilizing oculomotor system characteristics (i.e., oculometrics) to sense the altered levels of cognitive loading and fatigue during a standardized computer work were investigated [6–8]. Oculometrics are particularly of interest as they could potentially be recorded unobtrusively during physical activities. Furthermore, in the last study in this set, oculometrics were shown to be effective in implementing a smart fatigue alarm which could potentially be utilized to administer the pausing regime during physical activity [6]. Here, the results of these recent studies are summarized and discussed.

Procedures

In study I [7], the sensitivity of oculometrics (e.g., properties of fixations and saccadic eye movements) to altered cognitive loading and their between-days reliability were investigated. Thirty-eight participants took part in this study and performed an identical experimental protocol over two experimental days. The participants performed a standardized computer mouse task which was composed of a cyclic activity consisting of memorization of a dotted pattern and replication of the dotted pattern after a short wash-out period (Figure 1). At each experimental day, the participants performed the computer task in three sessions of five minutes, representing altered levels of cognitive loading, namely, low, medium and high. The cognitive loading was altered by changing the geometrical complexity of the dotted pattern to be memorized.

In study II [8], the same participant pool as in study I took part and the oculometrics, which were found reliable in study I, were studied during a prolonged session of 40 min of an identical computer work as in study I. After finishing every 20 cycles of the task (approximately 200 s ), the participants scored their level of cognitive fatigue based on the Karolinska sleepiness scale (KSS) [9]. The changes of oculometrics with time-on-task were then compared with the temporal changes of KSS.

In study III [6], a computational model of fatigue state was developed and it allowed prediction of fatigue state based on the captured oculometrics. Twenty participants performed the computer task in two experimental sessions. Each session took 30 min and after every 20 cycles of the task, the participants scored their fatigue based on the KSS. Additionally, they could potentially take a micro-break of 25 s decided based on either their own discretion (manual session) or the alarming feedback generated by the fatigue alarm (automatic session). During the micro-breaks, the participants externally rotated their shoulders while holding an elastic band between the two hands and keeping the elbows in approximately 90° flexed and the shoulders up to 45° abducted. Moreover, the micro-break was accompanied by mindful breathing.

Data recording and analysis

With a sampling rate of 360 Hz, a monocular head-mounted eye tracker (Eye-Trac 7, Applied Science Laboratories, Bedford, MA, USA) was used in all studies to record gaze position, pupil diameter and blink moments (Figure 1). A motion-capture system (Visualeyez II system set up with two VZ4000 trackers, Phoenix Technologies Inc., Canada) tracked head movements and sent its information to the eye tracker to allow accounting for head movements. A data-driven algorithm was used to extract ocular events (i.e., saccades, fixations, or blinks). Based on the yielded output of the algorithm, the oculometrics were computed [10].

In study I, a long list of relevant oculometrics were tested and reliable metrics with sensitivity to altered mental load were chosen to be used in study II. Saccade peak velocity, saccade duration, the slope of line regressing peak velocity and amplitude of saccades (i.e. main sequence), fixation duration, blink duration, blink frequency and the mean of pupil dilation range (i.e. pupillary response) were utilized in study II to reflect fatigue with time-on-task.

The overall performance (OP) of participants carrying out the task was monitored in terms of the speed and accuracy of the pattern replication. The workload of the tasks in all studies was subjectively score by each participant based on a computerized version of the National Aeronautics and Space Administration Task Load Index (NASA-TLX) [11]. Furthermore, KSS was scored on a scale from 1 to 9, corresponding to “very alert” and “very sleepy, fighting sleep,” respectively.

All three studies involved common statistical analysis. A repeated-measures analysis of variance was used to examine the effects of the three levels of mental load as the within-subject factor on oculometrics in study I and the time effect in study II. In study I, the relative and absolute reliability of the oculometrics were assessed using intra-class correlation coefficient ICC and the limits of agreement (LoA) obtained from Bland-Altman plots.

Results

In study I, saccade peak velocity, saccade duration and the main sequence decreased whereas fixation duration increased with increased cognitive load demand. These metrics also exhibited acceptable reliability scores (ICC>0.5 and normalized LOA below 20%). Metrics like pupillary response and blink frequency and duration were also highlighted as they have been shown to be reliable in a number of studies [12,13]. As expected, OP decreased and NASA-TLX increased with cognitive load. In study II, as expected KSS increased with time-on-task and this was associated with decreasing saccade peak velocity, saccade duration and increasing pupillary response and blink frequency and duration as well as fixation duration with time-on-task. Study III revealed that the computational model yielded about 70% accuracy in identifying fatigue state. OP and KSS increased with time in both automatic and manual sessions but the increase in KSS was delayed in automatic session compared with the manual session. The participants reported lower loading demand in automatic session as NASA-TLX was markedly lower in the automatic session.

Conclusion

This set of studies confirmed the feasibility of the quantification of cognitive load and fatigue based on oculometrics during a functional computer task. The most insightful finding of this set of studies for practitioners was related to the contribution of oculometrics to planning the pausing regime during physical activity and showing that such a system could be more effective than self-triggered breaks in impeding fatigue.

Overall, this set of studies may shed light to the complex relationships between oculometrics, mental load, and fatigue and provide a practical approach for using eye tracking in the prevention of associated MSD risks in low load physical activities such as e-sport.

References

 

1         Ijmker S, Huysmans MA, Blatter BM, et al. Should office workers spend fewer hours at their computer? A systematic review of the literature. Occup Env Med 2007;64:211–22.

2         Yung M, Wells RP. Responsive upper limb and cognitive fatigue measures during light precision work: An 8-hour simulated micro-pipetting study. Ergonomics 2017;60:940–56.

3         Côté JN. Adaptations to neck/shoulder fatigue and injuries. In: Progress in motor control. Springer 2014. 205–28.

4         Srinivasan D, Mathiassen SE, Hallman DM, et al. Effects of concurrent physical and cognitive demands on muscle activity and heart rate variability in a repetitive upper-extremity precision task. Eur J Appl Physiol 2016;116:227–39. doi:10.1007/s00421-015-3268-8

5         Juul-Kristensen B, Søgaard K, Stroeyer J, et al. Computer users’ risk factors for developing shoulder, elbow and back symptoms. Scand J Work Environ Health 2004;30:390–8.

6         Marandi RZ, Madeleine P, Omland Ø, et al. An oculometrics-based biofeedback system to impede fatigue development during computer work: A proof-of-concept study. PLoS One 2019;14.

7         Marandi RZ, Madeleine P, Omland Ø, et al. Reliability of Oculometrics during a Mentally Demanding Task in Young and Old Adults. IEEE Access 2018.

8         Marandi RZ, Madeleine P, Omland Ø, et al. Eye movement characteristics reflected fatigue development in both young and elderly individuals. Sci Rep 2018;8:13148.

9         Åkerstedt T, Gillberg M. Subjective and objective sleepiness in the active individual. Int J Neurosci 1990;52:29–37. doi:10.3109/00207459008994241

10      Nyström M, Holmqvist K. An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data. Behav Res Methods 2010;42:188–204. doi:10.3758/BRM.42.1.188

11      Sharek D. A Useable, Online NASA-TLX Tool. Proc Hum Factors Ergon Soc Annu Meet 2011;55:1375–9. doi:10.1177/1071181311551286

12      Borghini G, Astolfi L, Vecchiato G, et al. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci Biobehav Rev 2014;44:58–75. doi:10.1016/j.neubiorev.2012.10.003

13      Martins R, Carvalho J. Eye blinking as an indicator of fatigue and mental load—a systematic review. In: Occupational Safety and Hygiene III. 2015. 231–5. doi:10.1201/b18042-48