The method of reassignment is a technique for sharpening a timefrequency representation by mapping the data to timefrequency coordinates that are nearer to the true region of support of the analyzed signal. The method has been independently introduced by several parties under various names, including method of reassignment, remapping, timefrequency reassignment, and modified movingwindow method.^{[1]} In the case of the spectrogram or the shorttime Fourier transform, the method of reassignment sharpens blurry timefrequency data by relocating the data according to local estimates of instantaneous frequency and group delay. This mapping to reassigned timefrequency coordinates is very precise for signals that are separable in time and frequency with respect to the analysis window.
Introduction
Reassigned spectral surface for the onset of an acoustic bass tone having a sharp pluck and a fundamental frequency of approximately 73.4 Hz. Sharp spectral ridges representing the harmonics are evident, as is the abrupt onset of the tone. The spectrogram was computed using a 65.7 ms Kaiser window with a shaping parameter of 12.
Many signals of interest have a distribution of energy that varies in time and frequency. For example, any sound signal having a beginning or an end has an energy distribution that varies in time, and most sounds exhibit considerable variation in both time and frequency over their duration. Timefrequency representations are commonly used to analyze or characterize such signals. They map the onedimensional timedomain signal into a twodimensional function of time and frequency. A timefrequency representation describes the variation of spectral energy distribution over time, much as a musical score describes the variation of musical pitch over time.
In audio signal analysis, the spectrogram is the most commonly used timefrequency representation, probably because it is wellunderstood, and immune to socalled "crossterms" that sometimes make other timefrequency representations difficult to interpret. But the windowing operation required in spectrogram computation introduces an unsavory tradeoff between time resolution and frequency resolution, so spectrograms provide a timefrequency representation that is blurred in time, in frequency, or in both dimensions. The method of timefrequency reassignment is a technique for refocussing timefrequency data in a blurred representation like the spectrogram by mapping the data to timefrequency coordinates that are nearer to the true region of support of the analyzed signal.
The spectrogram as a timefrequency representation
One of the bestknown timefrequency representations is the spectrogram, defined as the squared magnitude of the shorttime Fourier transform. Though the shorttime phase spectrum is known to contain important temporal information about the signal, this information is difficult to interpret, so typically, only the shorttime magnitude spectrum is considered in shorttime spectral analysis.
As a timefrequency representation, the spectrogram has relatively poor resolution. Time and frequency resolution are governed by the choice of analysis window and greater concentration in one domain is accompanied by greater smearing in the other.
A timefrequency representation having improved resolution, relative to the spectrogram, is the Wigner–Ville distribution, which may be interpreted as a shorttime Fourier transform with a window function that is perfectly matched to the signal. The Wigner–Ville distribution is highly concentrated in time and frequency, but it is also highly nonlinear and nonlocal. Consequently, this distribution is very sensitive to noise, and generates crosscomponents that often mask the components of interest, making it difficult to extract useful information concerning the distribution of energy in multicomponent signals.
Cohen's class of bilinear timefrequency representations is a class of "smoothed" Wigner–Ville distributions, employing a smoothing kernel that can reduce sensitivity of the distribution to noise and suppresses crosscomponents, at the expense of smearing the distribution in time and frequency. This smearing causes the distribution to be nonzero in regions where the true Wigner–Ville distribution shows no energy.
The spectrogram is a member of Cohen's class. It is a smoothed Wigner–Ville distribution with the smoothing kernel equal to the Wigner–Ville distribution of the analysis window. The method of reassignment smooths the Wigner–Ville distribution, but then refocuses the distribution back to the true regions of support of the signal components. The method has been shown to reduce time and frequency smearing of any member of Cohen's class ^{[2]} .^{[3]} In the case of the reassigned spectrogram, the shorttime phase spectrum is used to correct the nominal time and frequency coordinates of the spectral data, and map it back nearer to the true regions of support of the analyzed signal.
The method of reassignment
Pioneering work on the method of reassignment was published by Kodera, Gendrin, and de Villedary under the name of Modified Moving Window Method ^{[4]} Their technique enhances the resolution in time and frequency of the classical Moving Window Method (equivalent to the spectrogram) by assigning to each data point a new timefrequency coordinate that betterreflects the distribution of energy in the analyzed signal.
In the classical moving window method, a timedomain signal, x(t) is decomposed into a set of coefficients, \epsilon( t, \omega ), based on a set of elementary signals, h_{\omega}(t), defined
h_{\omega}(t) = h(t) e^{j \omega t}
where h(t) is a (realvalued) lowpass kernel function, like the window function in the shorttime Fourier transform. The coefficients in this decomposition are defined
\begin{align} \epsilon( t, \omega ) &= \int x(\tau) h( t  \tau ) e^{ j \omega \left[ \tau  t \right]} d\tau \\ &= e^{ j \omega t} \int x(\tau) h( t  \tau ) e^{ j \omega \tau } d\tau \\ &= e^{ j \omega t} X(t, \omega) \\ &= X_{t}(\omega) = M_{t}(\omega) e^{j \phi_{\tau}(\omega)} \end{align}
where M_{t}(\omega) is the magnitude, and \phi_{\tau}(\omega) the phase, of X_{t}(\omega), the Fourier transform of the signal x(t) shifted in time by t and windowed by h(t).
x(t) can be reconstructed from the moving window coefficients by
\begin{align} x(t) & = \iint X_{\tau}(\omega) h^{*}_{\omega}(\tau  t) d\omega d\tau \\ & = \iint X_{\tau}(\omega) h( \tau  t ) e^{ j \omega \left[ \tau  t \right]} d\omega d\tau \\ &= \iint M_{\tau}(\omega) e^{j \phi_{\tau}(\omega)} h( \tau  t ) e^{ j \omega \left[ \tau  t \right]} d\omega d\tau \\ &= \iint M_{\tau}(\omega) h( \tau  t ) e^{ j \left[ \phi_{\tau}(\omega)  \omega \tau+ \omega t \right] } d\omega d\tau \end{align}
For signals having magnitude spectra, M(t,\omega), whose time variation is slow relative to the phase variation, the maximum contribution to the reconstruction integral comes from the vicinity of the point t,\omega satisfying the phase stationarity condition
\begin{matrix} \frac{\partial}{\partial \omega} \left[ \phi_{\tau}(\omega)  \omega \tau + \omega t\right] & = 0 \\ \frac{\partial}{\partial \tau} \left[ \phi_{\tau}(\omega)  \omega \tau + \omega t \right] & = 0 \end{matrix}
or equivalently, around the point \hat{t}, \hat{\omega} defined by
\begin{align} \hat{t}(\tau, \omega) & = \tau  \frac{\partial \phi_{\tau}(\omega)}{\partial \omega} =  \frac{\partial \phi(\tau, \omega)}{\partial \omega} \\ \hat{\omega}(\tau, \omega) & = \frac{\partial \phi_{\tau}(\omega)}{\partial \tau} = \omega + \frac{\partial \phi(\tau, \omega)}{\partial \tau} . \end{align}
This phenomenon is known in such fields as optics as the principle of stationary phase, which states that for periodic or quasiperiodic signals, the variation of the Fourier phase spectrum not attributable to periodic oscillation is slow with respect to time in the vicinity of the frequency of oscillation, and in surrounding regions the variation is relatively rapid. Analogously, for impulsive signals, that are concentrated in time, the variation of the phase spectrum is slow with respect to frequency near the time of the impulse, and in surrounding regions the variation is relatively rapid.
In reconstruction, positive and negative contributions to the synthesized waveform cancel, due to destructive interference, in frequency regions of rapid phase variation. Only regions of slow phase variation (stationary phase) will contribute significantly to the reconstruction, and the maximum contribution (center of gravity) occurs at the point where the phase is changing most slowly with respect to time and frequency.
The timefrequency coordinates thus computed are equal to the local group delay, \hat{t}_{g}(t,\omega), and local instantaneous frequency, \hat{\omega} _{i}(t,\omega), and are computed from the phase of the shorttime Fourier transform, which is normally ignored when constructing the spectrogram. These quantities are local in the sense that they represent a windowed and filtered signal that is localized in time and frequency, and are not global properties of the signal under analysis.
The modified moving window method, or method of reassignment, changes (reassigns) the point of attribution of \epsilon(t,\omega) to this point of maximum contribution \hat{t}(t,\omega), \hat{\omega}(t,\omega), rather than to the point t,\omega at which it is computed. This point is sometimes called the center of gravity of the distribution, by way of analogy to a mass distribution. This analogy is a useful reminder that the attribution of spectral energy to the center of gravity of its distribution only makes sense when there is energy to attribute, so the method of reassignment has no meaning at points where the spectrogram is zerovalued.
Efficient computation of reassigned times and frequencies
In digital signal processing, it is most common to sample the time and frequency domains. The discrete Fourier transform is used to compute samples X(k) of the Fourier transform from samples x(n) of a time domain signal. The reassignment operations proposed by Kodera et al. cannot be applied directly to the discrete shorttime Fourier transform data, because partial derivatives cannot be computed directly on data that is discrete in time and frequency, and it has been suggested that this difficulty has been the primary barrier to wider use of the method of reassignment.
It is possible to approximate the partial derivatives using finite differences. For example, the phase spectrum can be evaluated at two nearby times, and the partial derivative with respect to time be approximated as the difference between the two values divided by the time difference, as in
\begin{matrix} \frac{\partial \phi(t, \omega)}{\partial t} & \approx \frac{1}{\Delta t} \left[ \phi(t + \frac{\Delta t}{2}, \omega)  \phi(t  \frac{\Delta t}{2}, \omega) \right] \\ \frac{\partial \phi(t, \omega)}{\partial \omega} & \approx \frac{1}{\Delta \omega} \left[ \phi(t, \omega+ \frac{\Delta \omega}{2})  \phi(t, \omega\frac{\Delta \omega}{2}) \right] \end{matrix}
For sufficiently small values of \Delta t and \Delta \omega, and provided that the phase difference is appropriately "unwrapped", this finitedifference method yields good approximations to the partial derivatives of phase, because in regions of the spectrum in which the evolution of the phase is dominated by rotation due to sinusoidal oscillation of a single, nearby component, the phase is a linear function.
Independently of Kodera et al., Nelson arrived at a similar method for improving the timefrequency precision of shorttime spectral data from partial derivatives of the shorttime phase spectrum. ^{[5]} It is easily shown that Nelson's cross spectral surfaces compute an approximation of the derivatives that is equivalent to the finite differences method.
Auger and Flandrin showed that the method of reassignment, proposed in the context of the spectrogram by Kodera et al., could be extended to any member of Cohen's class of timefrequency representations by generalizing the reassignment operations to
\begin{matrix} \hat{t} (t,\omega) & = t  \frac{\iint \tau \cdot W_{x}(t\tau,\omega \nu) \cdot \Phi(\tau,\nu) d\tau d\nu} {\iint W_{x}(t\tau,\omega \nu) \cdot \Phi(\tau,\nu) d\tau d\nu } \\ \hat{\omega} (t,\omega) & = \omega  \frac{\iint \nu \cdot W_{x}(t\tau,\omega \nu) \cdot \Phi(\tau,\nu) d\tau d\nu} {\iint W_{x}(t\tau,\omega \nu) \cdot \Phi(\tau,\nu) d\tau d\nu} \end{matrix}
where W_{x}(t,\omega) is the Wigner–Ville distribution of x(t), and \Phi(t,\omega) is the kernel function that defines the distribution. They further described an efficient method for computing the times and frequencies for the reassigned spectrogram efficiently and accurately without explicitly computing the partial derivatives of phase. ^{[2]}
In the case of the spectrogram, the reassignment operations can be computed by
\begin{matrix} \hat{t} (t,\omega) & = t  \Re \Bigg\{ \frac{ X_{\mathcal{T}h}(t,\omega) \cdot X^*(t,\omega) } {  X(t,\omega) ^2 } \Bigg\} \\ \hat{\omega}(t,\omega) & = \omega + \Im \Bigg\{ \frac{ X_{\mathcal{D}h}(t,\omega) \cdot X^*(t,\omega) } {  X(t,\omega) ^2 } \Bigg\} \end{matrix}
where X(t,\omega) is the shorttime Fourier transform computed using an analysis window h(t), X_{\mathcal{T}h}(t,\omega) is the shorttime Fourier transform computed using a timeweighted anlaysis window h_{\mathcal{T}}(t) = t \cdot h(t) and X_{\mathcal{D}h}(t,\omega) is the shorttime Fourier transform computed using a timederivative analysis window h_{\mathcal{D}}(t) = \frac{d}{dt}h(t).
Using the auxiliary window functions h_{\mathcal{T}}(t) and h_{\mathcal{D}}(t), the reassignment operations can be computed at any timefrequency coordinate t,\omega from an algebraic combination of three Fourier transforms evaluated at t,\omega. Since these algorithms operate only on shorttime spectral data evaluated at a single time and frequency, and do not explicitly compute any derivatives, this gives an efficient method of computing the reassigned discrete shorttime Fourier transform.
One constraint in this method of computation is that the  X(t,\omega) ^2 must be nonzero. This is not much of a restriction, since the reassignment operation itself implies that there is some energy to reassign, and has no meaning when the distribution is zerovalued.
Separability
The shorttime Fourier transform can often be used to estimate the amplitudes and phases of the individual components in a multicomponent signal, such as a quasiharmonic musical instrument tone. Moreover, the time and frequency reassignment operations can be used to sharpen the representation by attributing the spectral energy reported by the shorttime Fourier transform to the point that is the local center of gravity of the complex energy distribution.
For a signal consisting of a single component, the instantaneous frequency can be estimated from the partial derivatives of phase of any shorttime Fourier transform channel that passes the component. If the signal is to be decomposed into many components,
x(t) = \sum_{n} A_{n}(t) e^{j \theta_{n}(t)}
and the instantaneous frequency of each component is defined as the derivative of its phase with respect to time, that is,
\omega_{n}(t) = \frac{d \theta_{n}(t)}{d t},
then the instantaneous frequency of each individual component can be computed from the phase of the response of a filter that passes that component, provided that no more than one component lies in the passband of the filter.
This is the property, in the frequency domain, that Nelson called separability ^{[5]} and is required of all signals so analyzed. If this property is not met, then the desired multicomponent decomposition cannot be achieved, because the parameters of individual components cannot be estimated from the shorttime Fourier transform. In such cases, a different analysis window must be chosen so that the separability criterion is satisfied.
If the components of a signal are separable in frequency with respect to a particular shorttime spectral analysis window, then the output of each shorttime Fourier transform filter is a filtered version of, at most, a single dominant (having significant energy) component, and so the derivative, with respect to time, of the phase of the X(t,\omega_{0}) is equal to the derivative with respect to time, of the phase of the dominant component at \omega_{0}. Therefore, if a component, x_{n}(t), having instantaneous frequency \omega_{n}(t) is the dominant component in the vicinity of \omega_{0}, then the instantaneous frequency of that component can be computed from the phase of the shorttime Fourier transform evaluated at \omega_{0}. That is,
\begin{matrix} \omega_{n}(t) &= \frac{\partial}{\partial t} \arg\{ x_{n}(t) \} \\ &= \frac{\partial }{\partial t} \arg\{ X(t,\omega_{0}) \} \end{matrix}
Longwindow reassigned spectrogram of the word "open", computed using a 54.4 ms Kaiser window with a shaping parameter of 9, emphasizing harmonics.
Shortwindow reassigned spectrogram of the word "open", computed using a 13.6 ms Kaiser window with a shaping parameter of 9, emphasizing formants and glottal pulses.
Just as each bandpass filter in the shorttime Fourier transform filterbank may pass at most a single complex exponential component, two temporal events must be sufficiently separated in time that they do not lie in the same windowed segment of the input signal. This is the property of separability in the time domain, and is equivalent to requiring that the time between two events be greater than the length of the impulse response of the shorttime Fourier transform filters, the span of nonzero samples in h(t).
In general, there is an infinite number of equally valid decompositions for a multicomponent signal. The separability property must be considered in the context of the desired decomposition. For example, in the analysis of a speech signal, an analysis window that is long relative to the time between glottal pulses is sufficient to separate harmonics, but the individual glottal pulses will be smeared, because many pulses are covered by each window (that is, the individual pulses are not separable, in time, by the chosen analysis window). An analysis window that is much shorter than the time between glottal pulses may resolve the glottal pulses, because no window spans more than one pulse, but the harmonic frequencies are smeared together, because the main lobe of the analysis window spectrum is wider than the spacing between the harmonics (that is, the harmonics are not separable, in frequency, by the chosen analysis window).
References

^ Hainsworth, Stephen (2003). "Chapter 3: Reassignment methods". Techniques for the Automated Analysis of Musical Audio (PhD). University of Cambridge.

^ ^{a} ^{b} F. Auger and P. Flandrin (May 1995). "Improving the readability of timefrequency and timescale representations by the reassignment method". IEEE Transactions on Signal Processing 43 (5): 1068–1089.

^ P. Flandrin, F. Auger, and E. ChassandeMottin, Timefrequency reassignment: From principles to algorithms, in Applications in TimeFrequency Signal Processing (A. PapandreouSuppappola, ed.), ch. 5, pp. 179 – 203, CRC Press, 2003.

^ K. Kodera, R. Gendrin, and C. de Villedary (Feb 1978). "Analysis of timevarying signals with small BT values". IEEE Transactions on Acoustics, Speech and Signal Processing 26 (1): 64–76.

^ ^{}a ^{b} D. J. Nelson (Nov 2001). "Crossspectral methods for processing speech". Journal of the Acoustical Society of America 110 (5): 2575–2592.
Further reading

S. A. Fulop and K. Fitz, A spectrogram for the twentyfirst century, Acoustics Today, vol. 2, no. 3, pp. 26–33, 2006.

S. A. Fulop and K. Fitz, Algorithms for computing the timecorrected instantaneous frequency (reassigned) spectrogram, with applications, Journal of the Acoustical Society of America, vol. 119, pp. 360 – 371, Jan 2006.
External links

TFTB — TimeFrequency ToolBox

SPEAR  Sinusoidal Partial Editing Analysis and Resynthesis

Loris  Opensource software for sound modeling and morphing

SRA  A webbased research tool for spectral and roughness analysis of sound signals (supported by a Northwest Academic Computing Consortium grant to J. Middleton, Eastern Washington University)
This article was sourced from Creative Commons AttributionShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, EGovernment Act of 2002.
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a nonprofit organization.