TSAM – Timbre Space Analyzer And Mapper

Overview

The Timbre Space Analyzer and Mapper (TSAM) is a Max/MSP tool for the analysis, modeling, and mapping of the timbre of sound synthesizers. It includes a background engine running in MATLAB used for the offline computation of the mapping and descriptor quality metrics. It natively hosts any VST software synthesizers and it can be extended to work with external hardware devices.

Synthesis algorithms often have a large number of adjustable parameters that determine the generated sound and its resultant psychoacoustic features. The relationship between parameters and timbre is important for end users, but it is generally unknown, complex, and difficult to analytically derive. The TSAM introduces a strategy for the analysis of the sonic response of synthesizers subject to the variation of an arbitrary set of parameters. It uses an extensive set of sound descriptors which are ranked using a novel metric based on statistical analysis. This enables the study of how changes to a synthesis parameter affect timbral descriptors, and provides a multidimensional model for the mapping of the synthesis control through specific timbre spaces. The analysis, modeling and mapping are integrated in the TSAM, which enables further investigation into synthesis sonic response and on perceptually related sonic interactions.

Timbre Response Analysis

• A large set of sound descriptors are computed for every synthesis state.

• Two automatically analysis modes are supported, and multiple sound descriptor vectors per state are combined accordingly.
  • Sustain mode: mean and range of descriptor vectors, for static timbres or almost periodic sounds.
  • Envelope mode: concatenation of descriptor vectors, for dynamic timbres, texture-like, or ADSR envelopes applied to parameters.

• Using the multiple descriptor vectors per state, we compute a quality metric for each descriptor, capturing four characteristics.
  • Noisiness: deviation of the descriptor from its mean or trend given a synthesis state.
  • Variance: spread of descriptor value across the synthesis states set.
  • Independence: uniqueness of the descriptor variation pattern across the synthesis states set.
  • Correlation: coherence of the descriptor variation with synthesis parameters across the synthesis states set.

• Summary of descriptors’ quality metric, quality components weighting, and relation between parameters and descriptors are provided.

Timbre Space Modeling and Mapping

• The pairwise relationship between the combined descriptor vectors and the synthesis states provides a model of the sound synthesizer.

• The low-dimensional timbre space, typically 2D or 3D, is be obtained by
  • explicit selection of sound descriptors, or
  • dimensionality reduction of a subset of sound descriptors.

• The quality metrics are essential in selecting the descriptors used to compute the timbre-based mapping.

• The interaction with the synthesis parameter is realized by mapping the control dimension onto the coordinates low-dimensional timbre space, and retrieving the synthesis state associated with the closest entry.

• The mapping is performed by neural network trained with the inverse of the homomorphic uniform redistribution of the timbre space.

• Spatial interpolation copes with the coarse parameter step-size used to reduces the analysis time.

• Parameter discontinuities due to non on-to-one descriptors-to-parameters relationship are avoided limiting the the instantaneous neighborhood in the search space.

• Interactive timbre space visuals and options to customize the mapping at runtime are included.

Applications

The TSAM integrates functionalities to analyze and map the timbre of sound synthesizers. Preliminary studies demonstrated that the adoption of large set of descriptors and their selection based on the novel quality metric improve the accuracy of the timbre-based mapping. TSAM applications includes:

• Study of the sonic response of synthesizers, and the relationship between parameters and perceptually related features of the generated sound

• Mapping to synthesis parameters with explicit control of prominent timbral characteristics.

• Minimal loss few-to-many mapping providing a significant dimensionality reduction of the synthesis control space.