Motion Compensated Temporal Filtering

Vicente González Ruiz - Depto Informática - UAL

January 24, 2023

1 Idea and notation

Motion Compensated Temporal Filtering (MCTF) [3] is basically a motion compensated random-access mode in which the last P-type frame is a B-type frame (see Fig. 1). MCTF can be considered a DWT where the input samples are the original video images and the output coefficients is a sequence of residue images. Some of these frame-coeffs contain low frequency information (in the temporal domain), and others represent high frequency temporal information. We will use average-frame (or simply average) to refer to a low-frequency frame, and residue to refer to a high-frequency frame. In general, the number of averages is smaller tha the name of residues.

Figure 1: The MCTF scheme.

2 Objectives of MCTF

The main goals of motion compensation are:

  1. Reduce the entropy of the residuals, and if
  2. Increase the temporal scalability, compared to low-delay and random-access modes [1]. MCTF is a dyadic temporal multirresolution approach.

3 MCTF uses ME

In order to exploit the temporal redundancy, the bidirectional predictions used in MCTF are generated using ME (Motion Estimation) [2]. The motion information (usually in the form of motion vector fields with a given density1) must be known by both, the encoder and the decoder, which runs the inverse MCTF. Therefore, MCTF can be considered an adaptive transform, and, as many others adaptive systems, the side information must be transmitted to or regenerated by the decoder.

4 (Expected) Entropies and dynamic ranges

Usually, MCTF (as many other transforms), increases the number of bits necessary to represent the residues (coefficients), but also decreases the entropy, because most of the information will be concentrated in a small number of residues [1].

5 References

[1]   V. González-Ruiz. Motion Compensation.

[2]   V. González-Ruiz. Motion Estimation.

[3]   J.-R. Ohm. Three-dimensional subband coding with motion compensation. IEEE Transactions on Image Processing, 3:559–571, 1994.

1Number of motion vectors per pixel.