REAL-TIME PSYCHOPHYSIOLOGICAL MEASURES
FOR ADAPTIVE AUTOMATION SYSTEMS

Wolfram Boucsein(1), Shinji Miyake(2) and Stephen Fairclough(3)

(1) University of Wuppertal, (2) University of Occupational and Environmental Health,
(3) Liverpool John Moores University

 

Adaptive automation (AA) refers to complex biocybernetic systems where the provision of automation is controlled dynamically. Unlike traditional forms of automation, the system can autonomously activate and deactivate automation. Psychophysiologists have worked with ergonomists to specify real-time psychophysiological measures to adapt automation in complex systems such as flight management or adaptive cruise control in car driving. Nickel and Hockey will specify the system needs for on-line assessment of psychophysiological markers for AA and provide data from a laboratory study with cardiovascular and EEG measures. Fairclough and co-workers will report on an experimental study using AA, a yoked control and a non-automation control group based on cardiorespiratory, electrodermal and ocular measures. Boucsein and co-workers will report on the suitability of electrodermal and cardiovascular measures for AA during simulated flight. Miyake and co-workers will demonstrate that heart rate and blood volume can be reliably used to measure workload during adaptive cruise control in real car driving. Finally, Trimmel and co-workers will report heart rates from pilots during long-haul flights as a basis for introducing AA in these operations. Advantages and disadvantages of various measures within the context of AA of complex systems will be discussed.

BACK

 

ABSTRACTS

ON-LINE ASSESSMENT OF PSYCHOPHYSIOLOGICAL MARKERS OF OPERATOR FUNCTIONAL STATE UNDER DYNAMIC TASK LOAD

Peter Nickel and G. Robert J. Hockey
The University of Sheffield

There is growing concern over the consequences of performance breakdown by operators of complex systems. To ensure high reliability and productivity the system's interfaces should: (1) allow the operator to remain 'in the loop'; and (2) protect the system against operator failures from high risk (e.g. fatigue) states. Compensatory control of performance means that these can occur even in the absence of overt decrement, so need to be detected and predicted. One solution is to use adaptive automation, in which the human-machine interface allows flexibility of dialogue control. A complex decision making and simulated process control task is used to identify valid markers of compromised operator functional state (risk states). A new dynamic loading method forces performance breakdown through stepwise changes in task load, allowing risk states (those immediately preceding the breakdown) to be identified. Highly trained student operators provide data on psychophysiological markers of risk (HRV; and two EEG power ratios - generalized beta/alpha and frontal theta/parietal alpha), providing person-referenced baselines for interpreting risk patterns. Results show evidence of validation of risk state indices in predicting breakdown in new test situations, suggesting that they provide an effective basis for triggering of changes in dialogue control.

Funding source: supported by UK EPSRC grant GR\S66985\01

THE USE OF AUTONOMIC MEASURES FOR BIOCYBERNETIC ADAPTATION

Stephen Fairclough, Louise Venables and Andrew J. Tattersall
Liverpool John Moores University

Biocybernetic adaptation refers to the use of real-time psychophysiology as a control input to a computing system. This study was concerned with the use of autonomic measures (e.g. cardiovascular, respiratory, skin conductance level, ocular) to control adaptive automation within a laboratory setting. Thirty participants (N=30) performed the Multi Attribute Test Battery (MATB) for sixty-five minutes under three conditions: (1) biocybernetic control, i.e. provision of automation was controlled by real-time measurement of autonomic activity, (2) yoked control, i.e. schedule of automation provision was identical to group 1, and (3) control group, i.e. no automation available. All participants performed a pre-test session in order to derive psychophysiological algorithms for biocybernetic control. During the pre-test session, participants were exposed to the MATB under five levels of task demand and estimates of subjective effort were collected. Psychophysiological algorithms and threshold values were derived for each individual participant using a linear regression analysis. Analyses of data were performed on: the pattern of automation, psychophysiological changes during periods of automatic/manual control, and the subjective perceptions from participants. The implications for biocybernetic control are discussed.

THE USABILITY OF CARDIOVASCULAR AND ELECTRODERMAL MEASURES FOR ADAPTIVE AUTOMATION DURING A SIMULATED IFR FLIGHT MISSION

Wolfram Boucsein, Andrea Haarmann and Florian Schaefer
University of Wuppertal

Adaptive automation (AA) refers to the capability of a system to vary the degree of automation dynamically depending on the operator's vigilance. The system may alert the operator during hypovigilance, thus increasing his/her attention, or may automatically take over more responsibility in the case of high workload. It is currently discussed for long-term operations such as intercontinental flights following instrumental flight rules (IFR). Vigilance decrements are reflected in autonomic measures such as cardiovascular and electrodermal activity (EDA). We recorded EDA together with heart rate from 16 student subjects during four flight missions with a professional IFR flight simulator, varying the strength of turbulence. Overall, ANOVA for repeated measurements revealed that the factors "flight mission" and "strength of turbulence" significantly influenced tonic EDA and heart rate variability (HRV). The number of non-specific changes in EDA decreased with progressing flight missions but increased with strength of turbulence, thus being an indicator of vigilance and workload. HRV decreased with both, progressing flight missions and increasing strength of turbulence, indicating an increase of mental demands. We conclude that autonomic measures serve as valid indicators of vigilance decrement and workload during long-term operations, and may be used for an adaptive feedback algorithm within an automated system.

ON-ROAD INVESTIGATION OF EFFECTS OF ADAPTIVE CRUISE CONTROL ON WORKLOAD

Shinji Miyake, Takuro Shoji(1), Yasuhiko Takae(2), Nariaki Etori(2), Takayuki Watanabe(2) and Yasuhiro Shiraishi(2)
(1) University of Occupational and Environmental Health, Japan
(2) Nissan Motor Co., Ltd.

Fifteen test drivers were instructed to follow a lead vehicle on a closed test-track with and without Adaptive Cruise Control (ACC) system. The lead vehicle ran with slow acceleration/deceleration in low level driving condition and with 0.1-0.25G acceleration/deceleration in high level driving condition. ECG from the chest and Tissue Blood Volume (TBV) from the tip of the nose were recorded. Subjective ratings of workload (NASA-TLX and sleepiness) were obtained after each run. Mean TBV and HRV parameters were calculated in each condition including before and after 4-minute resting period. Subjective ratings of workload clearly showed main effects of driving difficulties. Sleepiness scores showed significant interaction of drive mode and driving level. Significant main effect of driving level was shown in TBV, which decreased in ACC-High condition. However, main effect of drive mode was not significant in this index. LF/HF showed its highest value in ACC-Low condition and significant main effect of driving level. Our results suggested that TBV is a sensitive measure of a workload. Driving with ACC following a lead car that ran with strong acceleration/deceleration boosted up drivers' frustration level. Since frustration may induce some specific physiological responses, workload should be considered when we use physiological measures as workload indices.

HEART RATE OF PILOTS DURING LONG HAUL FLIGHTS INDICATE LOW AROUSAL STATES

Michael Trimmel, Christoph Goger, Eva-Maria Vouk, Marlene Kritz, Alexandra Klaus and Elisabeth Groll-Knapp
Medical University Vienna

Level of arousal in pilots was investigated during long haul flights. Data were collected from six outbound and inbound flights from Vienna, where 3 went to Delhi (7 hours flight, crew of 2 pilots) and 3 went to Tokyo (11 hours flight, crew of 3 pilots with always one of them resting for 3 hours during flight - but this period was excluded from analysis). Electrocardiogram was continuously recorded from the time of briefing (about 1.5 hours before take-off) until debriefing, after arriving the hotel of the destination city or until the time of undressing the uniform in the staff wardrobe (being back in Vienna) respectively. Data from 13 pilots from Austrian Airlines (AUA) on Airbuses (A340-300, A340-400) were analysed for intervals of 5 min. Results showed that mean HR was 57-60 bpm that after 4.5 hours. In 132 5-min-epochs the mean HR was in the range from 40-50 bpm. A mean HR below 50 bpm was recorded in 7 from 13 pilots and a minimal HR (in 5 min epochs) in the range from 31-40 bpm was recorded in 6 of 13 pilots on flight. Data suggest time epochs of a relatively low arousal after 4.5 hours on some flights.

BACK