WP1a

Constructing Salience: Who is the Subject of Musical Machine Listening Research?

Nick Seaver, Assistant Professor, Department of Anthropology and Director, Science, Technology & Society, Tufts University

Machine listening is a major area of research for computer scientists working in the field of Music Information Retrieval (MIR), which automatically classifies audio recordings with labels such as genre or instrumentation; it is commonly used in commercial music recommendation. Machine listening systems learn from ‘ground truth’ datasets – examples labelled by trusted experts – and then extrapolate to label new materials. Some researchers in MIR have examined and critiqued what machine listening listens to: some successful genre classifiers, for instance, have been shown to rely on inaudible patterns in audio data (Rodriguez Algarra 2016), raising questions about how machine listening relates to human listening and what constitutes algorithmic ‘success’.

This study examines ongoing controversies about how machine listening should attend to musical sound, investigating the construction of salience (Jaton 2017) in this community: the processes and terms by which a machine listening system comes to be understood as listening ‘correctly’. The topic is of great concern in music, since machine listening is being built into the ‘performative infrastructures’ (Thrift 2005) of recommendation, with the potential to shape music listening behaviour in its image.