Physics-Informed Differentiable Method for Piano Modeling

This page includes of audio examples for the article Simionato R., Fasciani S., Holm S., Physics-Informed Differentiable Method for Piano Modeling, in Frontiers in Signal Processing, 2023.

The article presents a method for piano sound synthesis informed by the physics of the instrument, combining deep learning with traditional digital signal processing techniques. The proposed model is designed to learn to synthesize only the the quasi-harmonic content of individual piano notes using physics-based formulas, whose parameters are automatically estimated from real audio recordings. In doing so, the model emulates the inharmonicity of the piano and the partials’ amplitude envelopes. The inharmonic/noisy component of the of the piano sound is not modeled, hence there is a significant difference in the timbre of the real recording versus the predicted sound.

Associated code, dataset, and trained models are available on GitHub.

Each example includes the following:

Target Output Audio Signal (true output recorded from a Yamaha Disklavier
527 MX100A)
Predicted Output Audio Signal (generated by the inference of the trained model, only harmonic component is modeled)
Spectrum of Target and Predicted Output Audio Signals
Spectrograms of Target and Predicted Output Audio Signals

The examples refer to two scenarios:

Scenario A. The training set includes all dataset’s recordings, excluding those related to the D4 key.
Scenario B. The training set includes all dataset’s recordings, excluding those related to the 90 velocity.

The title of the waveform plot details the key and MIDI velocity.