search

Orange-OpenSource / Cool-Chic

Low-complexity neural image & video codec.
https://orange-opensource.github.io/Cool-Chic/
BSD 3-Clause "New" or "Revised" License
102 stars 6 forks source link

Error in Cool-Chic v3.0: pyo3_runtime.PanicException: Invalid model parameter: `scale` must be positive. #10

Closed pineforest2 closed 2 months ago

pineforest2 commented 2 months ago

Firstly, I want to use Cool-Chic v3.0 to compress videos. I use the UVG-Beauty-1080p video to test it, just like

python src/encode.py                                \
    --input=~/data/UVG/Beauty_1920x1080_120fps_420_8bit_YUV.yuv \
    --output=samples/bitstream.bin                  \
    --workdir=./my_temporary_workdir/               \
    --lmbda=0.0002                                  \
    --start_lr=1e-2                                 \
    --layers_synthesis=40-1-linear-relu,3-1-linear-relu,X-3-residual-relu,X-3-residual-none \
    --upsampling_kernel_size=8                      \
    --layers_arm=24,24                              \
    --n_ctx_rowcol=3                                \
    --n_ft_per_res=1,1,1,1,1,1,1                    \
    --n_itr=1000                                    \
    --n_train_loops=1

After running the program for a period of time, the following error will occur.

Loading a video encoder from ./my_temporary_workdir//video_encoder.pt /home/wsl/vc/Cool-Chic/src/models/video_encoder.py:422: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. raw_data = torch.load(load_path, map_location='cpu') /home/wsl/vc/Cool-Chic/src/models/frame_encoder.py:1373: FutureWarning: You are using torch.load with weights_only=False (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for weights_only will be flipped to True. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via torch.serialization.add_safe_globals. We recommend you start setting weights_only=True for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature. loaded_data = torch.load(raw_bytes, map_location='cpu') Encoding to samples/bitstream.bin thread '' panicked at src/pybindings/stream/model.rs:547:5: Invalid model parameter: scale must be positive. note: run with RUST_BACKTRACE=1 environment variable to display a backtrace Traceback (most recent call last): File "/home/wsl/vc/Cool-Chic/src/encode.py", line 203, in encode_video(video_encoder, args.output) File "/home/wsl/vc/Cool-Chic/src/bitstream/encode.py", line 137, in encode_video encode_frame(video_encoder, frame_encoder, frame_bitstream_path) File "/home/wsl/miniconda3/envs/newest/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/wsl/vc/Cool-Chic/src/bitstream/encode.py", line 379, in encode_frame range_coder_latent.encode( File "/home/wsl/vc/Cool-Chic/src/bitstream/range_coder.py", line 88, in encode encoder.encode(x, self.model_family, mu, scale) pyo3_runtime.PanicException: Invalid model parameter: scale must be positive.

theoladune commented 2 months ago

Hi,

Just a few questions to better understand your issue.

  1. Could you indicate the hash of the specific commit you're using? You mentioned Cool-chic 3.0 and video, but we introduced video in Cool-chic 3.1.

  2. Are you using the proposed requirements.txt or a custom made environment? If so, what's your python and pytorch version?

pineforest2 commented 2 months ago

Hello, thank you!

  1. Could you indicate the hash of the specific commit you're using? You mentioned Cool-chic 3.0 and video, but we introduced video in Cool-chic 3.1.

8f3d29766c308da01f70b66163d7e657182ab640

I test the paper 'C3: High-performance and low-complexity neural compression from a single image or video', which seems to correspond to Cool-Chic v3.0. And there seems to be a related introduction in the README.

  1. Are you using the proposed requirements.txt or a custom made environment? If so, what's your python and pytorch version?

Not using the proposed requirements.txt. My version:

theoladune commented 2 months ago

Thanks for the additional info on your issue. I do not have (yet) a definitive answer to it but here, but here are a few things that you can try out.

Cool-chic version

Cool-chic is pretty much identical to C3 for image compression. If you want to do image compression, you should use the latest release on the master branch (v3.3) which only offers image compression, with performance slightly above C3 and a fast CPU-only decoder. Use that if you want to do image compression.

If you are interested in video compression, note that Cool-chic differs vastly from C3 for video coding. If you want to try out Cool-chic for video compression, you should go back to the version v3.1.

scale must be positive error

This is an issue related to the entropy coding, i.e. the actual writing of a binary file. This is performed by the constriction package and I've already noticed that using too recent versions of this package can lead to issues. You should try using the recommended version (0.3.0) instead of the most recent one (0.3.5).

If you're doing image compression only, you can use the latest Cool-chic release which no longer relies on the constriction package for the entropy coding.

As a last resort, perhaps you don't need to write a bitstream and you can just look at the results_best.tsv generated, which would give an accurate estimate of the compression performance without writing an actual binary file.

Hope this helps! :)

pineforest2 commented 2 months ago

Thank you very much! I will seriously consider your opinion.