search

pytorch / vision

Datasets, Transforms and Models specific to Computer Vision
https://pytorch.org/vision
BSD 3-Clause "New" or "Revised" License
16.26k stars 6.96k forks source link

License Problem of vision #5839

Closed wqh17101 closed 2 years ago

wqh17101 commented 2 years ago

🚀 The feature

As the video related API added in, please care about the license of video component.

For example, if you use ffmpeg( with x264(GPL license) ) or pyav (with (ffmpeg with x264)) without process isolation, you will meet the license problem for change vision's license to GPL .

Motivation, pitch

Watch out that the GPL is contagious. Which will cause big problem to a BSD project.

Alternatives

No response

Additional context

No response

datumbox commented 2 years ago

Thanks for reporting. My understanding is that ffmpeg's licensing is the reason why we don't offer precompiled binaries with video support. I'm not too knowledgeable on the topic, so let me cc a few people who know more on the topic than I do.

@fmassa @prabhat00155 @bjuncek Could you provide a summary of the licensing situation?

wqh17101 commented 2 years ago

For pyav you can use python -m av --version to see the real runtime license like

PyAV v9.0.1
library configuration: --enable-gpl --enable-version3 --enable-shared --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
library license: GPL version 3 or later
libavcodec     58. 91.100
libavdevice    58. 10.100
libavfilter     7. 85.100
libavformat    58. 45.100
libavutil      56. 51.100
libswresample   3.  7.100
libswscale      5.  7.100

As you can see it is GPL.

The root reason of the license is what FFMPEG built with. The same situation for Pyav. Something you can check for this. link

So for pyav , the solution is to build pyav by source that you can chose what component of ffmpeg to build with,then pyav will use your custom ffmpeg without GPL component.

fmassa commented 2 years ago

Hi @wqh17101 ,

Thanks for opening this issue.

From what I remember (but @andfoy or @bjuncek can correct me if I'm missing something) is indeed similar to what you mentioned.

What we originally did (but maybe was changed in recent iterations of the binaries) was that the video components of torchvision were opt-in and not a hard-requirement to get torchvision installed. I think we didn't even add pyav in our requirements.txt, so that if the user wanted to depend on video they would need to either:

I'm probably missing some subtleties in here, so I'll leave to @andfoy and @bjuncek to chime in

bjuncek commented 2 years ago

Hi all,

Thanks for the issue; as @fmassa and @datumbox pointed out, at the moment we do not have hard requirements nor do we package binaries for ffmpeg nor pyav precisely because of licensing reasons.

For pyav (and openCV) for example, the license depends on the installation type. The codebase itself is BSD3, however the binaries are GPL as they link statically to FFMPEG built with --enable-gpl flags as you correctly point out. Similarly, if you were to link torchvision to any of the binaries with GPL above, the entire binary becomes GPL. Without explicit static linking to either library however, torchvision can maintain the BSD3 license.

If you want torchvision with video support where binaries that not go into GPL territory, the only option is to build FFMPEG without --enable-gpl flag. Most parts of FFMPEG are actually LGPL which, to the best of my knowledge, enables linking without spidering license change. Note that you cannot use conda-forge or pip pre-built binaries as they are both GPL licensed. Then, if you were to link pyav (have to build with --no-binary flag) and torchvision built from source to this FFMPEG installation, resulting binaries would technically still be licensed under BSD3 as LGPL doesn't require license change.

What does non-GPL FFMPEG mean for a user? You can see the complete list of GPL parts of FFMPEG here. Most notable parts for us are the external libraries such as libx264 and libx265. For 4.2 version of FFMPEG, I've found some stability issues with BSD3/LGPL alternatives, however this was almost a year ago now so one's mileage may vary.

Unfortunately for now, we don't really have a good resolution, thus we're avoiding hard dependencies on GPL binaries in order to avoid switching the license.

cc @andfoy who might have extra info and @rgommers for visibility

wqh17101 commented 2 years ago

I have an idea that you can use h264_cuvid and nvenc_h264 to replace libx264 and hevc_cuvid and nvenc_hevc to replace libx265 on GPU vision. Also you can use Openh264 to repalce libx264 on CPU.

There are many candidates.

Maybe just offer a basic FFMPEG to make video pipeline work.

bjuncek commented 2 years ago

@andfoy was toying with the idea of providing LGPL compatible version of ffmpeg to pytorch channel. Maybe we should pick this up again?

andfoy commented 2 years ago

We used to provide an LGPL version of FFmpeg that linked against OpenH264, but it was only for 4.2

wqh17101 commented 2 years ago

Never mind, the latest minor updating version is 4.4.2,just update and recompile. @andfoy @bjuncek

rgommers commented 2 years ago

@bjuncek's explanation looks correct and complete to me. In addition, let's link his PyAV license issue (https://github.com/PyAV-Org/PyAV/issues/805) which clarifies licensing status of PyAV binaries on PyPI and conda-forge.

It would be good to make some updates to the README to clarify the license situation. There's a section https://github.com/pytorch/vision#video-backend which mentions PyAV and FFmpeg without context. There's a section with notes on licenses of trained models at https://pypi.org/project/torchvision/. I'd suggest:

  1. Add a Licensing of binaries and models section at the bottom of the README, explaining the situation.
  2. Add a reference to that section from the "video backend" section.
  3. Keep the LICENSE file itself and the BSD in the metadata unchanges.

@andfoy was toying with the idea of providing LGPL compatible version of ffmpeg to pytorch channel. Maybe we should pick this up again?

I'd suggest opening a new issue for this. That's a bigger discussion, and much more work, than to make the current licensing status better documented.

datumbox commented 2 years ago

I've checked with our Open-source and legal teams and they confirmed that:

TorchVision doesn't have a required dependency on or distribute any GPL library, so there are no issues with the BSD license. We don't prevent a user from combining TorchVision with GPL software, but no GPL software is needed for TorchVision to operate.

I will therefore close the issue as there is no licensing problem in TorchVision.