search

MozillaSecurity / corpus-replicator

A corpus generation tool
Mozilla Public License 2.0
20 stars 3 forks source link

Using image_heic_heic_imagemagick.yml requires ffmpeg installed #6

Open dlemstra opened 1 year ago

dlemstra commented 1 year ago

I am here because of a response to an issue in the oss-fuzz repsitory: #10610. Thanks for adding image_heic_heic_imagemagick.yml so we can generate corpus for heic again. But for some reason this requires the installation of ffmpeg and I am not sure why this needs to be installed. It looks like ffmpeg is used for other recipes but not for the heic corpus creation. The issue can be reproduce in this devcontainer:

https://github.com/ImageMagick/ImageMagick/tree/main/.devcontainer/security

And running the following commands:

apt install -y python3-pip
python3 -m pip install corpus-replicator
corpus-replicator -o test image_heic_heic_imagemagick.yml image
root@68e87d148a27:/workspaces/ImageMagick-Linux# corpus-replicator -o test image_heic_heic_imagemagick.yml image
usage: corpus-replicator [-h] [--log-level {DEBUG,INFO}] [-o OUTPUT] [--version]
                         recipes [recipes ...] {animation,audio,image,video} ...
corpus-replicator: error: Please install FFmpeg.
root@68e87d148a27:/workspaces/ImageMagick-Linux/foobar# corpus-replicator --version
corpus-replicator 1.1.0
tysmith commented 1 year ago

Hi, yes this is expected. corpus-replicator works by generating a base template file and then re-encoding the template to generate the corpus. FFmpeg is used to generate the base template file since it is able to generate unique audio, image and video files via lavfi.

I realize that in this case ImageMagick could be used as an image template generator and that would remove the dependent in this scenario but that's out of scope at the moment.

Are you able to install and use FFmpeg?

dlemstra commented 1 year ago

This would be possible but I don't want to do this because this will probably result in the oss-fuzzer using it. We build all our dependencies from source and I don't want to add and maintain a ffmpeg dependency.

tysmith commented 1 year ago

I see, let me see if I can come up with a better solution for templates.

tysmith commented 1 year ago

@dlemstra Sorry for the delay, what about installing corpus-replicator and deps, generating the corpus and then finally removing the deps? I have a patch here: https://github.com/tysmith/oss-fuzz/tree/im-corpus I've tested it locally and all deps are removed before the build starts. Let me know if that works for you and I can create a PR.

Also we can generate more than just HEIC files using corpus-replicator if you'd like but we can address that later.

dlemstra commented 1 year ago

I don't know when I will have time for this but it looks like you are still temporary installing ffmpeg? I don't mind waiting until you have removed this dependency or are you no longer planning to do this?

tysmith commented 1 year ago

I'd prefer to either leave it as is or replace the dependency with something that provides the same functionality. The functionality in this case is the ability to generate template images, audio and video, and also a variety of content for each media type.

To directly answer the question, yes the patch is installing FFmpeg, generating the corpus and removing FFmpeg (and dependencies) before continuing. I don't plan to remove the dependency on FFmpeg in the short term unless an adequate replacement is found.