Voice Analysis (since 1988)
Research activities deal with the analysis of the speech signal for the detection and measurement of variability due to psychophysiological factors. These are caused at the laboratory or occur naturally in some work situations. The pilots are a population of speakers subject to important workloads and sometimes for long periods. The aviation context provides real-life situations (in flight or from cockpit voice recorders) and offers the possibility to use fixed or mobile simulators. In all cases, speech disturbances are possible subsequent to the onset of failures but also due to workload or fatigue. Some experiments in the field of psychology are also able to generate vocal expression singular properties.
Aviation Context
– Cockpit Voice Recorder (C.V.R) analysis (with Bureau d’Enquêtes et d’Analyses BEA), 1989 – 1992. 
A record has been selected as the failures observed before the accident caused long discussions between the crew members providing a rich sound material for studying the consequences of vocal emotion.
– Voice analysis in long-haul flight with incidents voluntary caused (with Services Technique de la Navigation Aérienne S.T.N.A – Airbus). For certification in flight with a reduced crew of two pilots instead of three, the real long-haul flights have taken place with the creation of failures by the engineers on board. The crew conversations were recorded to study the correlation between the stress, the acoustic characteristics of voice and the estimated workload of medical indicators measured by the study partners.
– records in airplane simulator with failures during flights (with Aéroformation, Base Opérationnelle Mobile AéroPortée). The simulator allows an experimental protocol to induce a controlled number of incidents.
– recordings during the experimental laboratory set up to study sleep inertia (with Laboratoire d’Anthropologie Appliquée LAA, hôpital Henri Mondor and B.E.A). For the study of sleep and alertness of the crews, the LAA has developed an experimental laboratory to monitor sleep and from which it was possible to record the voice of a pilot when he awoke and performed aeronautical tasks. The study had to identify the non-medical characteristics, so vocal ones, modified by sleep inertia. 
– fatigue of a crew in short-haul flights (with Bureau d’Enquêtes et d’Analyses, Air France). The experiment was designed to estimate from voice analysis the fatigue of a crew during a day of short-haul flights with multiple rotations on French territory.
– car driver fatigue (with Bureau d’Enquêtes et d’Analyses). The object was to do a driving experiment for which the workload could be comparable with that of pilots in order to test the parameters of acoustic analysis of voice. 
– Sleep Deprivation and Workload : (with ENAC-SEFA and BEA). Students Pilots have been recorded during their practical studies on a Airbus A320 flight simulator (Airbus Training Center). 
Two flights of four hours duration are used for the experiment. They induce an important workload. Early done in the morning, asleep deprivation is possible. Pilots complete questionnaires to a self-assessment of their workload and to estimate their degree of anxiety. The purpose of acoustic analysis is to estimate to what degree and how the workload modifies vowel features (end of the flight). An examination of the voices at the beginning of the flights will also be performed.
Psychological Context
– Stroop Test (with laboratories of Phonetics from Mons University (Belgium) and from Barcelone University (Spain)). This test also called Color-Word test causes an interference effect of reading a word on the name of its color. For example when it comes to name the color of the word “red” when it is written in green: the word “red” is written in green ink and the speaker must pronounce “green.” Acoustic changes in the signal from a pronunciation without interference have been searched.
– Emotion from Images (with R. Da Silva Neves, researcher in cognitive sciences). The images (International Affective Picture System IAPS) developed by the “Center for Emotion and Attention” from the University of Florida has the advantage of classifying and rating pictures depending on emotion they induce. The laboratory experiments led the audience to speak by watching the pictures in an attempt to link their respective notes and the values of the acoustic features
measured. 
Physical Fatigue (out of aviation context) – laboratory experiment. It was obtained using an exercise bike according to a protocol of effort avoiding excessive shortness of breath of the speaker and with a simultaneous measurement of heart rate.
Drowsiness and Mental Workload research with the LLCE Laboratory of Toulouse Jean Jaurès University, Agnès Daurat and Radouane El Yagoubi and with Stanislas Boyer for his PhD of the Aeronautical and Astronautical Doctoral School of University of Toulouse . The abstract of the thesis is the following.
Operational requirements of aircraft pilots may cause drowsiness and inadequate mental load levels (i. e., too low or too high) during flights. Sleep debts and circadian disruptions linked to various factors (e. g., long working periods, irregular work schedules, etc.) require pilots to challenge their biological limits. Moreover, pilots’ mental workload exhibits strong fluctuations during flights : higher during critical phases (i. e., take on and landing), it becomes very low during cruising phases. When the mental load becomes too high or, conversely, too low, performance decreases and flight errors may manifest. Implementation of detection methods of drowsiness and mental load levels in near real time is a major challenge for monitoring and controlling flight activity. The aim of this thesis is therefore to determine if the human voice can serve to detect on one hand the drowsiness and on the other hand the mental load level of an individual.
In a first study, the voice of participants was recorded during a reading task before and after a night of total sleep deprivation (TSD). Drowsiness variations linked to TSD were assessed using self-evaluative and electrophysiological measures ElectroEncephaloGraphy [EEG] and Evoked Potentials [EPs]). Results showed significant variations aer the TSD in many acoustic features related to : (a) the amplitude of the glottal pulses (amplitude modulation frequency), (b) the shape of the acoustic wave (Euclidean length of the signal and its associated features) and (3) the spectrum of the vowel signal (harmonic-to-noise ratio, second formant frequency, skewness, spectral center of gravity, energy differences, spectral tilt and Mel-frequency cepstral coefficients). Most spectral features showed different sensitivity to sleep deprivation depending on the vowel type. Significant correlations were found between several acoustic features and several objective indicators (EEG and PEs) of drowsiness. In a second study, voices were recorded during a task featuring word-list recall. The difficulty of the task was manipulated by varying the number of words in each list (i. e., between one and seven, corresponding to seven mental load conditions). Evoked pupillary response – known to be a useful proxy of mental load – was recorded simultaneously with speech to attest variations in mental load level during the experimental task. Results showed that classical features (fundamental frequency and its standard deviation, shimmer, number of periods and harmonic-to-noise ratio) and original features (amplitude modulation frequency and short-term variation in digital amplitude length) were particularly sensitive to variations in mental load. Variations in these acoustic features were correlated to those of the pupil size.
Results suggest that the acoustic features of the human voice identified during these expe-
riments could represent relevant indicators for the detection of drowsiness and mental load levels of an individual. Findings open up many research and applications perspectives in the field of transport safety, particularly in the aeronautical sector.
Acoustic analysis have focused on signals of French oral vowels by measuring:
– fundamental frequency : mean, standard deviation, by period, jitters …
– micro-tremors of vocal cords ;
– shimmer ;
– spectrum: formants, L.P.C coefficients, center of gravity, frequency for an energetical balance, cumulative distributions of sound level and spectral balance frequency, spectral tilt, energy ratios, skewness, kurtosis, spectral distances (M.F.C.C coefficients) ;
– speech rate ;
– maximal Lyapunov exponent and phase portrait ;
– Digital Amplitude Length and associated parameters ;
– amplitude modulation of maximal glottal pulses.
Measurements were initially performed using a specific software (ILS) and then progressively developed using Matlab ® allowing today to have a tool for automatic analysis of segmented vowels. Measurements were also performed on stop consonants using a software for time-frequency analysis (N! Power ®, Signal Technology Inc.).. The feature measured was the Voice Onset Time. However, the large natural variability of signals consonants led to stop the search for acoustic characteristics modified by psycho-physiological disturbances. This variability would mask the effects of these factors. All the above parameters have not been measured simultaneously for all experiments due to progress in the research and in the analysis and calculation tools.
Among the measured features three of them are original: the frequency of spectral balance issued from cumulative distributions of sound level in the spectrum, the Digital Amplitude Length as a measure of length of time signal and the frequency modulation of glottal amplitude maxima. Their respective presentation is discussed in the summary of the work (see Publication) as well as the details of the results and the first elements of a study to model the time evolution of acoustic voice changes due to fatigue.
