search

tongdaxu / Idempotence-and-Perceptual-Image-Compression

Official Implementation for (ICLR 2024) Idempotence and Perceptual Image Compression
29 stars 1 forks source link

the result that imagenet model generated is not good enough, I think the diffusion config sets more steps? #1

Open AlbertLin0 opened 5 months ago

AlbertLin0 commented 5 months ago

the label picture is 00000 the per_recon picture is

00000 It looks like the diffusion model has not iterated enough steps, (one thousand)

tongdaxu commented 5 months ago

Hi @AlbertLin0, this result is pretty strange and we have never seen such a reconstruction. We believe it is a bug, can you provide the exact command line and config?

AlbertLin0 commented 5 months ago

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

tongdaxu commented 5 months ago

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

This is strange. The configs look ok to me. I am traveling with limited gpu access so I might not be able to actually run this until May 13. Channing q from 1 to 2 should not bring a huge difference. Can you please try ImageNet + hyperprior ? So I can check whether it is the problem of diffusion model or elic.

tongdaxu commented 5 months ago

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2?

the diffusion model might be buggy for torch 1.9, see https://github.com/huggingface/diffusers/pull/6730#issuecomment-1944036738

AlbertLin0 commented 5 months ago

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2?

the diffusion model might be buggy for torch 1.9, see huggingface/diffusers#6730 (comment)

I have tried torch 2.1.2 and torch 2.2.0 with imagenet config, and the results are similar as the nosiy picture above. But, the results of ffhq config are pretty good. And I'm tring to run diffusion-posterior-sampling...

tongdaxu commented 5 months ago

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2? the diffusion model might be buggy for torch 1.9, see huggingface/diffusers#6730 (comment)

I have tried torch 2.1.2 and torch 2.2.0 with imagenet config, and the results are similar as the nosiy picture above. But, the results of ffhq config are pretty good. And I'm tring to run diffusion-posterior-sampling...

if you have time, it’s is helpful to run a sr x4 DPS and check the result. So we will know whether it’s the problem of environment or our code. And I can debug faster later.

AlbertLin0 commented 5 months ago

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2? the diffusion model might be buggy for torch 1.9, see huggingface/diffusers#6730 (comment)

I have tried torch 2.1.2 and torch 2.2.0 with imagenet config, and the results are similar as the nosiy picture above. But, the results of ffhq config are pretty good. And I'm tring to run diffusion-posterior-sampling...

if you have time, it’s is helpful to run a sr x4 DPS and check the result. So we will know whether it’s the problem of environment or our code. And I can debug faster later.

configs

I have put 'elic' operater to DPS, the args is 

"args":[
"--model_config=configs/imagenet_model_config.yaml",
"--diffusion_config=configs/diffusion_config.yaml",
"--task_config=configs/elic_config.yaml"
]

the imagenet_model_config and diffusion_config is default, and the elic_config:


conditioning:
method: ps
params:
scale: 0.3

data: name: ffhq root: ./data/samples/

measurement: operator: name: elic q: 2

noise: name: gaussian sigma: 0.05


### results:
input:
![input00001](https://github.com/tongdaxu/Idempotence-and-Perceptual-Image-Compression/assets/58505644/f5849e82-6819-4f63-83bf-8cf9e918f899)

label:
![label00001](https://github.com/tongdaxu/Idempotence-and-Perceptual-Image-Compression/assets/58505644/6350639d-034c-4224-8575-56f79884a614)

recon:
![recon00001](https://github.com/tongdaxu/Idempotence-and-Perceptual-Image-Compression/assets/58505644/987dc5f8-7736-48ed-98a0-afd384ec2f53)
tongdaxu commented 5 months ago

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2? the diffusion model might be buggy for torch 1.9, see huggingface/diffusers#6730 (comment)

I have tried torch 2.1.2 and torch 2.2.0 with imagenet config, and the results are similar as the nosiy picture above. But, the results of ffhq config are pretty good. And I'm tring to run diffusion-posterior-sampling...

if you have time, it’s is helpful to run a sr x4 DPS and check the result. So we will know whether it’s the problem of environment or our code. And I can debug faster later.

configs

I have put 'elic' operater to DPS, the args is

            "args":[
                "--model_config=configs/imagenet_model_config.yaml",
                "--diffusion_config=configs/diffusion_config.yaml",
                "--task_config=configs/elic_config.yaml"
            ]

the imagenet_model_config and diffusion_config is default, and the elic_config:

conditioning:
  method: ps
  params:
    scale: 0.3

data:
  name: ffhq
  root: ./data/samples/

measurement:
  operator:
    name: elic
    q: 2

  noise: 
    name: gaussian
    sigma: 0.05

results:

input: input00001

label: label00001

recon: recon00001

How is Elic on imagenet model works? Or is that already image net model and you are just testing with FFHQ?

AlbertLin0 commented 5 months ago

hi @tongdaxu, The command line is:

--model_config=configs/imagenet_model_config.yaml
--diffusion_config=configs/diffusion_config.yaml
--task_config=configs/elic_config.yaml
--save_dir=results

the config: I just change the 'q' from 1 to 2 and others is default and the python version is 3.9.13, I dont know whether the version of packages would cause this problem the ffhq diffusion model has the beautiful result.

Also would you like to try a different torch version like torch 2.1 and 2.2? the diffusion model might be buggy for torch 1.9, see huggingface/diffusers#6730 (comment)

I have tried torch 2.1.2 and torch 2.2.0 with imagenet config, and the results are similar as the nosiy picture above. But, the results of ffhq config are pretty good. And I'm tring to run diffusion-posterior-sampling...

if you have time, it’s is helpful to run a sr x4 DPS and check the result. So we will know whether it’s the problem of environment or our code. And I can debug faster later.

configs

I have put 'elic' operater to DPS, the args is

            "args":[
                "--model_config=configs/imagenet_model_config.yaml",
                "--diffusion_config=configs/diffusion_config.yaml",
                "--task_config=configs/elic_config.yaml"
            ]

the imagenet_model_config and diffusion_config is default, and the elic_config:

conditioning:
  method: ps
  params:
    scale: 0.3

data:
  name: ffhq
  root: ./data/samples/

measurement:
  operator:
    name: elic
    q: 2

  noise: 
    name: gaussian
    sigma: 0.05

results:

input: input00001 label: label00001 recon: recon00001

How is Elic on imagenet model works? Or is that already image net model and you are just testing with FFHQ?

I just want to figure out why imagenet model generate noisy picture with your code. So the test is example DPS provided.

AuroraZengfh commented 5 months ago

hi, @AlbertLin0, I'm also running the codebase. It looks fine for me to run through elic with imagenet. All my configs are default except for the path of pretrained model and datasets.

Label picture is: 00000 per_recon picture is: per_recon_00000

It looks like that your environment has something not compatible. Here's my environment and my python is 3.8.18. Hope it will help you figure out.

Package Version Editable project location

absl-py 2.1.0 accelerate 0.28.0 addict 2.4.0 aiosignal 1.3.1 annotated-types 0.6.0 antlr4-python3-runtime 4.9.3 anykeystore 0.2 apex 0.1 asttokens 2.4.1 attrs 23.2.0 av 12.0.0 backcall 0.2.0 bcrypt 4.1.2 beautifulsoup4 4.12.3 cachetools 5.3.3 certifi 2023.11.17 cffi 1.16.0 cfgv 3.4.0 charset-normalizer 3.3.2 click 8.1.7 cloudpickle 3.0.0 cmake 3.28.3 colossalai 0.3.6 comm 0.2.1 contexttimer 0.3.3 contourpy 1.1.1 cryptacular 1.6.2 cryptography 42.0.5 cycler 0.12.1 decorator 5.1.1 defusedxml 0.7.1 Deprecated 1.2.14 distlib 0.3.8 einops 0.7.0 executing 2.0.1 fabric 3.2.2 filelock 3.13.1 fonttools 4.50.0 frozenlist 1.4.1 fsspec 2023.12.2 ftfy 6.1.3 google 3.0.0 google-auth 2.29.0 google-auth-oauthlib 1.0.0 greenlet 3.0.3 grpcio 1.62.1 huggingface-hub 0.20.3 hupper 1.12.1 Hydra 2.5 hydra-core 1.3.2 hydra-submitit-launcher 1.2.0 identify 2.5.35 idna 3.6 importlib-metadata 7.0.1 importlib-resources 6.1.1 invoke 2.2.0 ipython 8.12.3 ipywidgets 8.1.1 jedi 0.19.1 Jinja2 3.1.3 jsonschema 4.21.1 jsonschema-specifications 2023.12.1 jupyterlab-widgets 3.0.9 kiwisolver 1.4.5 lit 18.1.1 Markdown 3.6 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.7.5 matplotlib-inline 0.1.6 mdurl 0.1.2 mmengine 0.10.3 mpmath 1.3.0 msgpack 1.0.8 mypy-extensions 1.0.0 networkx 3.1 ninja 1.11.1.1 nodeenv 1.8.0 numpy 1.24.4 nvidia-cublas-cu11 11.10.3.66 nvidia-cublas-cu12 12.1.3.1 nvidia-cuda-cupti-cu11 11.7.101 nvidia-cuda-cupti-cu12 12.1.105 nvidia-cuda-nvrtc-cu11 11.7.99 nvidia-cuda-nvrtc-cu12 12.1.105 nvidia-cuda-runtime-cu11 11.7.99 nvidia-cuda-runtime-cu12 12.1.105 nvidia-cudnn-cu11 8.5.0.96 nvidia-cudnn-cu12 8.9.2.26 nvidia-cufft-cu11 10.9.0.58 nvidia-cufft-cu12 11.0.2.54 nvidia-curand-cu11 10.2.10.91 nvidia-curand-cu12 10.3.2.106 nvidia-cusolver-cu11 11.4.0.1 nvidia-cusolver-cu12 11.4.5.107 nvidia-cusparse-cu11 11.7.4.91 nvidia-cusparse-cu12 12.1.0.106 nvidia-nccl-cu11 2.14.3 nvidia-nccl-cu12 2.19.3 nvidia-nvjitlink-cu12 12.4.99 nvidia-nvtx-cu11 11.7.91 nvidia-nvtx-cu12 12.1.105 oauthlib 3.2.2 omegaconf 2.3.0 opencv-python 4.9.0.80 opt-einsum 3.3.0 packaging 23.2 paramiko 3.4.0 parso 0.8.3 PasteDeploy 3.1.0 pbkdf2 1.3 pexpect 4.9.0 pickleshare 0.7.5 pillow 10.2.0 pip 23.3.1 pkgutil_resolve_name 1.3.10 plaster 1.1.2 plaster-pastedeploy 1.0.1 platformdirs 4.2.0 pre-commit 3.5.0 prompt-toolkit 3.0.43 protobuf 5.26.0 psutil 5.9.8 ptyprocess 0.7.0 pure-eval 0.2.2 pyasn1 0.5.1 pyasn1-modules 0.3.0 pycparser 2.21 pydantic 2.6.4 pydantic_core 2.16.3 Pygments 2.17.2 PyNaCl 1.5.0 pyparsing 3.1.2 pyramid 2.0.2 pyramid-mailer 0.15.1 pyre-extensions 0.0.23 python-dateutil 2.9.0.post0 python3-openid 3.2.0 PyYAML 6.0.1 ray 2.9.3 referencing 0.34.0 regex 2023.12.25 repoze.sendmail 4.4.1 requests 2.31.0 requests-oauthlib 2.0.0 rich 13.7.1 rpds-py 0.18.0 rsa 4.9 safetensors 0.4.2 sentencepiece 0.2.0 setuptools 68.2.2 six 1.16.0 soupsieve 2.5 SQLAlchemy 2.0.28 stack-data 0.6.3 submitit 1.5.1 sympy 1.12 tensorboard 2.14.0 tensorboard-data-server 0.7.2 termcolor 2.4.0 timm 0.4.12 tokenizers 0.15.1 tomli 2.0.1 torch 2.0.0+cu118 torchvision 0.15.1+cu118 tqdm 4.66.1 traitlets 5.14.1 transaction 4.0 transformers 4.37.1 translationstring 1.4 triton 2.0.0 typing_extensions 4.9.0 typing-inspect 0.9.0 urllib3 2.1.0 velruse 1.1.1 venusian 3.1.0 virtualenv 20.25.1 wcwidth 0.2.13 WebOb 1.8.7 Werkzeug 3.0.1 wheel 0.41.2 widgetsnbextension 4.0.9 wrapt 1.16.0 WTForms 3.1.2 wtforms-recaptcha 0.3.2 xformers 0.0.17 yapf 0.40.2 zipp 3.18.1 zope.deprecation 5.0 zope.interface 6.2 zope.sqlalchemy 3.1

tongdaxu commented 5 months ago

Hi @AlbertLin0, I have asked my intern @AuroraZengfh to run the ELIC Imagenet from scratch, and the result is fine. I have asked him to post the result and his environment. Have you solved this issue? Aligning to his environment and try again might be helpful.