How to know which VideoCapture backends are actually available?

[kipcole9]

@cocoa-xu sorry for the many small issues, I'm still learning ...

Evision.VideoCapture.getBackends/0 docs say:

  @spec getBackends() :: [[number()]] | {:error, String.t()}

Returns list of all available backends

• The spec says the return type is [[number()]] but the actual result appears to be [number()]
• The returned result list, using the precompiled binaries, is:

  iex> Evision.VideoIORegistry.getBackends
  [1900, 1800, 2300, 1200, 2000, 2200, 2500]

• Evision.cv_CAP_FFMPEG == 1900
• However opening a video with this backend returns an error suggesting that the backend is configured, but not available:

  # Opening the video returns no error, but there is no metadata and the video is not open
  iex> video = "./test/support/video/video_sample.mp4" |> Evision.VideoCapture.videoCapture(apiPreference: 1900)     %Evision.VideoCapture{
  fps: 0.0,
  frame_count: 0.0,
  frame_width: 0.0,
  frame_height: 0.0,
  isOpened: false,
  ref: #Reference<0.2048151986.449445908.177405>
  }
  # Introspecting the backend for the video returns an error
  iex> Evision.VideoCapture.getBackendName video                                                                     
  {:error,
  "OpenCV(4.6.0) /Users/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/videoio/src/cap.cpp:333: error: (-215:Assertion failed) api != 0 in function 'getBackendName'\n"}

• Opening with Evision.cv_CAP_AVFOUNDATION appears ok:

  iex> Evision.cv_CAP_AVFOUNDATION
  1200
  iex(38)> video = "./test/support/video/video_sample.mp4" |> Evision.VideoCapture.videoCapture(apiPreference: 1200)
  %Evision.VideoCapture{
  fps: 30.0,
  frame_count: 173.0,
  frame_width: 1920.0,
  frame_height: 1080.0,
  isOpened: true,
  ref: #Reference<0.2048151986.449445908.177406>
  }
  iex> Evision.VideoCapture.getBackendName video
  "AVFOUNDATION"

[kipcole9]

Further experimentation leads me to believe my assumption may be correct. The list returned is configured backends, not necessarily available. I set:

export OPENCV_VIDEOIO_DEBUG=1

And then ran again:

iex> video = "./test/support/video/video_sample.mp4" |> Evision.VideoCapture.videoCapture(apiPreference: 1800)
[ WARN:0@0.015] global /Users/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/videoio/src/cap.cpp (132) open VIDEOIO(GSTREAMER): trying capture filename='./test/support/video/video_sample.mp4' ...
[ WARN:0@0.015] global /Users/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/videoio/src/cap.cpp (194) open VIDEOIO(GSTREAMER): backend is not available (plugin is missing, or can't be loaded due dependencies or it is not compatible)

%Evision.VideoCapture{
  fps: 0.0,
  frame_count: 0.0,
  frame_width: 0.0,
  frame_height: 0.0,
  isOpened: false,
  ref: #Reference<0.706722183.202768404.32555>
}

# And the same for ffmpeg
iex> video = "./test/support/video/video_sample.mp4" |> Evision.VideoCapture.videoCapture(apiPreference: 1900)
[ WARN:0@0.015] global /Users/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/videoio/src/cap.cpp (132) open VIDEOIO(FFMPEG): trying capture filename='./test/support/video/video_sample.mp4' ...
[ WARN:0@0.016] global /Users/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/videoio/src/cap.cpp (194) open VIDEOIO(FFMPEG): backend is not available (plugin is missing, or can't be loaded due dependencies or it is not compatible)

%Evision.VideoCapture{
  fps: 0.0,
  frame_count: 0.0,
  frame_width: 0.0,
  frame_height: 0.0,
  isOpened: false,
  ref: #Reference<0.2538617247.2887122964.204234>
}

I will close this issue and note in Image about the actual behaviour. Now to work out which backends are actually available!

[kipcole9]

Arrrgggh, probably the issue is compatibility, not availability. I have ffmeg 5.1 installed which is not supported by OpenCV. I'm trying again with ffmpeg 4.x.

[kipcole9]

If I install ffmpeg 4.x and install evision from source then ffpmeg becomes available. It also correctly reports the frame count with the ffmpeg backend.

I wasn't able to get ffmpeg to work with the compiled evision NIF even though it has ffmpeg backend enabled. Is that what you would expect?

[kipcole9]

The question remains after all this: Is it possible to know which backends are both configured and available other than trying to open some media with a specific backend?

[cocoa-xu]

I wasn't able to get ffmpeg to work with the compiled evision NIF even though it has ffmpeg backend enabled. Is that what you would expect?

Kind of yes, I guess. This option needs to be enabled, and FFMPEG support will only be available when building from source

# optional
## enable FFmpeg
##   this will allow cmake to auto-detect FFmpeg libraries installed on the host
##   on Windows, OpenCV will download prebuilt FFmpeg libraries
##   for more information, please visit https://github.com/opencv/opencv/tree/4.x/3rdparty/ffmpeg
export CMAKE_OPENCV_OPTIONS="-D WITH_FFMPEG=ON"

https://github.com/cocoa-xu/evision#compile-evision-from-source

[kipcole9]

Thank you so much @cocoa-xu. Interestingly, ffmpeg backend is working even though I didn't export CMAKE_OPENCV_OPTIONS="-D WITH_FFMPEG=ON" (MacOS).

The question remains after all this: Is it possible to know which backends are both configured and available other than trying to open some media with a specific backend?

If this is a silly question or no alternative API to find out what is available then please close the issue.

[cocoa-xu]

Evision.VideoIORegistry.hasBackend/1 should do the job but I need to fix #158 first to get it working. As a workaround, it's possible to call the underlying NIF directly like this

iex> :evision_nif.videoio_registry_hasBackend(api: 1900)
false

[kipcole9]

Oh, that's excellent @cocoa-xu, thank you. Sorry for so much noise early in the morning (I'm in London the next 2 weeks).

[cocoa-xu]

No worries! I will release v0.1.26 sometime this week with OpenCV 4.6.0 after fixing all the issues that can be fixed currently.

[cocoa-xu]

I'll track this issue in #158. :)