Allow "nearest-exact" interpolation mode?

[pmeier]

Currently the following interpolation modes are allowed

https://github.com/pytorch/vision/blob/7046e56fe4370e94339b3e8b6fd011e285294a3a/torchvision/transforms/functional.py#L21-L32

Since torch==1.11.0 (pytorch/pytorch#64501 of @vfdev-5 to be exact), torch.nn.functional also supports mode="nearest-exact":

Mode mode='nearest-exact' matches Scikit-Image and PIL nearest neighbours interpolation algorithms and fixes known issues with mode='nearest'. This mode is introduced to keep backward compatibility. Mode mode='nearest' matches buggy OpenCV's INTER_NEAREST interpolation algorithm.

Given that we are aligning more with PIL and "nearest" is described as "buggy", can we add support for "nearest-exact"?

If yes, we should also think about changing all our default values to it. That might be a bit cumbersome for the users, but we could also remap its name. Meaning after the whole deprecation period is through, "nearest" just maps to "nearest-exact" of interpolate and we have a "nearest-legacy" or the like that maps to "nearest". We already do a name mapping for other interpolation modes:

https://github.com/pytorch/vision/blob/7046e56fe4370e94339b3e8b6fd011e285294a3a/torchvision/transforms/functional_tensor.py#L412-L414

Deprecation process could look like this where r denotes the current release.

• r+1: add "nearest-exact" as valid interpolation mode and deprecate not passing a value explicitly. Plus, this should add a "nearest-legacy" that aliases the current "nearest". We also need to warn if "nearest" is passed that the behavior will change in the future.

  def foo(..., mode=None):
      if mode is None:
          warnings.warn("Not passing a default value is deprecated.")
          mode = "nearest-legacy"
      elif mode == "nearest":
          warnings.warn(
              "Nearest is deprecated. Use nearest-legacy for the current behavior "
              "or nearest-exact for the future behavior"
          )

• r+2: fail if interpolation mode is not passed explicitly or "nearest" is passed.

• r+3: re-introduce "nearest" as default value for the interpolation mode, but map it internally to "nearest-exact".

cc @vfdev-5 @datumbox

[bhack]

Can we close this?

[NicolasHug]

Looks like the first part of this issue, i.e. introducing nearest-exact mode, was addressed in https://github.com/pytorch/vision/pull/6754.

The other part is to decide whether we want to change the default of the APIs that use "nearest" to change it to "nearest-exact". To decide whether this is worth doing we should first try to measure whether that change can lead to an accuracy drop. E.g. if we train on nearest but evaluate with nearest-exact (or the other way around), are we losing accuracy?

In any case, we should avoid remapping "nearest" to "nearest-exact": that would make torchvision's resize inconsistent with torch's interpolate, which would cause a lot of confusion (and also inconsistent with opencv).