Can't Run DreamClear for Image Restoration — Deprecation and Initialization Issues

[PixelArmony]

Hello, shallowdream204

I'm having trouble running DreamClear for image restoration. I followed the installation instructions and have all the necessary models downloaded. My setup involves using Windows with a conda environment and Python 3.9.

Here's what I did:

1. Created a new conda environment with Python 3.9.
2. Installed DreamClear using git clone.
3. Downloaded all the required models to the D:\AI\DreamClear\models directory.

However, when I run the command - (my test image folder is "D:\AI\DreamClear\JPEGS"):

python.exe test_1024.py configs/DreamClear/DreamClear_Test.py --dreamclear_ckpt D:\AI\DreamClear\models\DreamClear-1024.pth --swinir_ckpt D:\AI\DreamClear\models\general_swinir_v1.ckpt --vae_ckpt D:\AI\DreamClear\models\sd-vae-ft-ema\diffusion_pytorch_model.bin --lre --cfg_scale 4.5 --color_align wavelet --image_path D:\AI\DreamClear\JPEGS --npz_path D:\AI\DreamClear\npz --save_dir D:\AI\DreamClear\output

I encounter multiple issues:

1. Deprecation Warnings for torch.distributed.launch and suggestions to use torchrun, which isn't installed in my environment.
2. TypeError: Accelerator Initialization due to unexpected keyword arguments like even_batches.

I also tried running the script directly, but it doesn’t seem to resolve the issue. Here’s the specific error I get:

_TypeError: init() got an unexpected keyword argument 'evenbatches'

I haven't modified the script files, updated or install any packages and prefer not to, so I'm at an impasse. Any assistance or updates to the script from the development team would be appreciated.

Thanks!

[shallowdream204]

Sorry, I haven't tested DreamClear on the Windows system. Please ensure that the PyTorch environment is correctly installed.

[PixelArmony]

Hello shallowdream204,

I find your recommendation quite surprising because I already provided all the necessary information. But, here's a detailed recap:

1. Windows with a conda environment with Python 3.9 according to your installation procedure:

   conda create -n dreamclear python=3.9 -y
   conda activate dreamclear

2. Installed DreamClear using the git clone command and your link provided and installed here: "D:\AI\DreamClear"

   git clone https://github.com/shallowdream204/DreamClear.git
   cd DreamClear

3. Used the requirements.txt file, provided by you: pip3 install -r requirements.txt

4. Downloaded all the pre-trained AI models including with this link as you recommended and also provided by you at: https://huggingface.co/shallowdream204/DreamClear/tree/main

5. Since there were no instructions on where to place the downloaded models, I created a directory at: D:\AI\DreamClear\models

Executed the Python command you provided, using my models directory path and the "JPEGS" path for my images. "D:\AI\DreamClear\JPEGS":

python.exe test_1024.py configs/DreamClear/DreamClear_Test.py --dreamclear_ckpt D:\AI\DreamClear\models\DreamClear-1024.pth --swinir_ckpt D:\AI\DreamClear\models\general_swinir_v1.ckpt --vae_ckpt D:\AI\DreamClear\models\sd-vae-ft-ema\diffusion_pytorch_model.bin --lre --cfg_scale 4.5 --color_align wavelet --image_path D:\AI\DreamClear\JPEGS --npz_path D:\AI\DreamClear\npz --save_dir D:\AI\DreamClear\output

And like I mentioned, I encounter multiple issues, such as:

• Deprecation warnings for torch.distributed.launch suggest using torchrun, which isn't installed by default in my environment. Rational conclusion: according to Python, it appears that the torchrun Python library is missing from your requirements.txt file.
• A TypeError accelerator initialization due to unexpected keyword arguments like even_batches. Here is the specific message:

  _TypeError: init() got an unexpected keyword argument 'evenbatches'

As I mentioned, I haven't modified the script files, updated or installed any packages or library and prefer not to. You have now all the information that you need to help me. Only you can find a solution for this. I'm really counting on your expertise to resolve these errors and make DreamClear functional on my setup and for Windows.

Thank you in advance for your help, shallowdream204!

[flAIght-dev]

Same error here, with linux. Downgrading accelerate (no fixed version in requirements.txt) seems to fix the error:

pip install accelerate==0.34.2

I also had to rename the file in sd-vae-ft-ema from sd-vae-ft-ema_config.json to config json.

Models were downloaded to models/ and RealLQ250.zip uncompressed to data/

This is my command line:

$ python3 -m torch.distributed.launch --nproc_per_node 1 --master_port 1234 test_1024.py configs/DreamClear/DreamClear_Test.py --dreamclear_ckpt models/DreamClear-1024.pth --swinir_ckpt models/general_swinir_v1.ckpt --vae_ckpt models/sd-vae-ft-ema/ --lre --cfg_scale 4.5 --color_align wavelet --image_path data/RealLQ250/lq --npz_path data/RealLQ250/npz --save_dir validation

it starts but I get a few warnings and an OOM with 12Gb of free memory.

Also: An error occurred while trying to fetch models/sd-vae-ft-ema/: Error no file named diffusion_pytorch_model.safetensors found in directory models/sd-vae-ft-ema/.

so I converted the bin file to safetensor (I am NOT sure if this is the correct thig to do):

from safetensors.torch import save_file, load_file import torch model = torch.load("models/sd-vae-ft-ema/diffusion_pytorch_model.bin") save_file(model, "models/sd-vae-ft-ema/diffusion_pytorch_model.safetensors")

The warning is gone but OOM is still there. No idea about memory requirements, maybe this is expected.