search

millardjn / rusty_sr

Deep learning superresolution in pure rust
203 stars 21 forks source link

rusty_sr seems to be stuck #3

Closed jedisct1 closed 7 years ago

jedisct1 commented 7 years ago

Hi,

And congrats for kicking off this great project and its underlying library.

I tried with a random image of mine, but the process never completed. CPU usage topped to all cores being 100% busy, then dropped to 0 after a few minutes and then nothing happened any more. No output being generated either.

rust-nightly (f80514426 2017-02-10), macOS.

I just ran rusty_sr /tmp/tof-12637.jpg /tmp/yz.jpg.

Input image attached: tof-12637

millardjn commented 7 years ago

Hmm, this image runs fine on my setup and a completely silent death is unusual for rust. Is your OS kernel silently killing the process due to OutOfMemory (check /var/log/messages)?

This has unearthed a different problem too! I was originally just supporting .png outputs, but that wasn't made obvious in the readme! And PNG files were being saved to .jpg paths! So I've the included the code to use the format inference built into the image crate to also support jpeg outputs. However, there seems to be some encoding problem in the image crate, the image generated isn't a valid jpg file. I'll open a issue over on their repo, and try and get to the bottom of this.

For now, could you try again with a .png file output, ensure that you have compiled with cargo rustc --release -- -C target-cpu=native., and watch to see if there is an OOM Killer event.

millardjn commented 7 years ago

Looks like the image team are already onto the jpeg issues: https://github.com/PistonDevelopers/image/pull/619

jedisct1 commented 7 years ago

Hi!

I tried with a PNG output, but it didn't make a difference. No OOM issues either, it resembles more of a deadlock.

millardjn commented 7 years ago

Ah, so the process is still alive but not doing anything. My best guesses are it is stalling on the file write, or I haven't got the memory consistency/condvar notification right in the multithreaded matrix multiply. I've tightened the memory ordering in that crate past what i thought was required just in case. A cargo update should pick up the new version.

I've also added a Writing file... output to Rusty_SR after the compute stage is finished, to help distinguish blocking of the file write. A typical output would now look like: Upscaling using imagenet neural net parameters... Writing file... Done

If it isn't either of these, then I am temporarily out of ideas. If anyone else is seeing this kind of issue please feel free to jump in/post your system info.