search

Algolzw / image-restoration-sde

Image Restoration with Mean-Reverting Stochastic Differential Equations, ICML 2023. Winning solution of the NTIRE 2023 Image Shadow Removal Challenge.
https://algolzw.github.io/ir-sde/index.html
MIT License
587 stars 43 forks source link

Repeat 100it [00:45, 2.22it/s] #51

Closed xiaoyeqiufang closed 1 year ago

xiaoyeqiufang commented 1 year ago

What does the repeated statement below mean? I tried to reproduce it, but I've been stuck at this point, and it seems like it doesn't finish running in a short period of time.

Path already exists. Rename it to [/root/image-restoration-sde-main/experiments/deraining/ir-sde_archived_231005-125919] cosine schedule 100it [00:30, 3.29it/s] 100it [00:25, 3.88it/s] 100it [00:47, 2.10it/s] 100it [00:43, 2.32it/s] 100it [00:25, 3.88it/s] 100it [00:41, 2.44it/s] 100it [00:52, 1.92it/s] 100it [00:51, 1.96it/s] 100it [00:45, 2.22it/s] 100it [00:48, 2.06it/s] 100it [00:45, 2.22it/s] 100it [00:45, 2.22it/s] 100it [00:45, 2.22it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:42, 2.36it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:45, 2.22it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 100it [00:45, 2.22it/s] 100it [00:25, 3.88it/s] 100it [00:25, 3.88it/s] 36it [00:09, 3.85it/s]

Algolzw commented 1 year ago

I haven't met this problem before. Sometimes it will be stuck due to the memory issue. Have you checked your memory and GPU usage?

xiaoyeqiufang commented 1 year ago

I haven't met this problem before. Sometimes it will be stuck due to the memory issue. Have you checked your memory and GPU usage?

This should be a validation process during the training process, with each representative validating an image.

wangwangzhang9527 commented 1 year ago

I haven't met this problem before. Sometimes it will be stuck due to the memory issue. Have you checked your memory and GPU usage?

Hi! I'm having the same problem with the image deblurring training process, this seems to be taking too long to validate, it takes about 4 hours to validate on the 4090, how long does your test or validation take? Am I setting it up wrong somewhere?

Algolzw commented 1 year ago

@wangwangzhang9527 Hi! For deblurring I highly recommend you use less number of images for validation. And you can only test once on the full test set using the last checkpoint.

wangwangzhang9527 commented 1 year ago

@wangwangzhang9527 Hi! For deblurring I highly recommend you use less number of images for validation. And you can only test once on the full test set using the last checkpoint.

Thank you for your timely and useful response. This solved my confusion.