Background

The analysis of time series data is key in various scientific disciplines. They emerge, for example, from measurements on black-box systems or as results of generative processes. Time series data can be analysed from two different angles: in the frequency domain in which the spectral composition of the signal is revealed, and in the time domain in which the distribution of the values of the signal is analysed. With the latter, statistical properties can be assessed; not only the more common ones like mean and variance, but also higher order statistical properties like skew and kurtosis.

In this experiment we intend to investigate if the higher order properties of time series data can be made perceivable with sonification - the perceptualisation of data by auditory means. We seek to find perceptual dimensions which correlate with the statistical properties. With audification - the interpretation of time series data as wave form and the simplest form of sonification - the mean is not perceivable, the variance correlates to the energy in the signal and hence to the loudness. But which are the perceptual dimension can be found for skew and kurtosis? And what are the just noticeable differences?

The Experiment

The experiment consists of three parts in which the participant is asked to assess the similarity of noise signals. In the first part, a training phase, the participants gets used to the user interface and the sound quality that is presented. The following two parts are two different sets of sounds. In each example the participant will hear sequences of 3 different sounds and is asked to select the two which he perceives having the same quality. They are not exactly the same, but share a similar quality. Participants may replay the sequence as often as they wish. The experiment takes approximately 20 min and may be stopped at any time.

The introduction to the problem domain given to participants of the listening experiment can be downloaded here.

Audio examples:

• Time Series Experiment TimeSeriesExperiment - Audio Example 1 (mp3) (noise with -0.0008 skew and 14.02 kurtosis)
• Time Series Experiment TimeSeriesExperiment - Audio Example 2 (mp3) (noise with -0.35 skew and 8.74 kurtosis)
• Time Series Experiment TimeSeriesExperiment - Audio Example 3 (mp3) (noise with 0.0019 skew and 0.167 kurtosis)

SC3- Code examples

• Time Series Experiment: PDFShaper (probability density function shaper) is a tool that allows to draw probability functions in a user interface and hear the audification of the resulting shaped noise in real time (no external requirements, mac only).
• Time Series Experiment: TSAnalyser(GUI) are SC3 classes that facilitate the audification of arbitrary time series data including the possibility to scramble the signal (part of the SonEnvir framework). TSExperiment is the code to the listening experiment conducted to investigate whether kurtosis and skew are perceivable in audifications (requires TSAnalyser).

References:

• A. de Campo et.al. "Sonification as an interdisciplinary working process " in the Proceedings of ICAD 2006, London, UK.
• C. Frauenberger, A. de Campo, "Analysing Time series data", to appear in the Proceedings of ICAD 2007, Montreal.