search

Justin-Tan / high-fidelity-generative-compression

Pytorch implementation of High-Fidelity Generative Image Compression + Routines for neural image compression
Apache License 2.0
411 stars 77 forks source link

Doesn't work for single channel images #10

Closed markveillette closed 3 years ago

markveillette commented 3 years ago

Great repo! Thanks for implementing this.

I'm trying to apply this to gray-scale imagery with shape [1,L,L]. I noticed a couple spots where you assume the number of input channels is 3:

  1. In train.py you call create_model before you set args.image_dim.
    https://github.com/markveillette/high-fidelity-generative-compression/blob/052a34c4760a85a89e42b55846972c913f206ba1/train.py#L283

  2. In the generator, a 3 is hard coded in the output layer https://github.com/markveillette/high-fidelity-generative-compression/blob/052a34c4760a85a89e42b55846972c913f206ba1/src/network/generator.py#L141

Justin-Tan commented 3 years ago

Thanks for letting me know about this. I pushed a quick fix to the branch: https://github.com/Justin-Tan/high-fidelity-generative-compression/tree/minor_features

Let me know if you run into any issues. I'd be interested to see if the original model will work on grayscale images. If you want to use the pretrained models you could probably get away with freezing all weights except those associated with the generator and finetuning the generator weights from scratch or from the checkpoint.

markveillette commented 3 years ago

Thanks! I actually wanted to use this to images of weather data in this dataset: https://registry.opendata.aws/sevir/ so I needed to train from scratch. Early results are promising.

Justin-Tan commented 3 years ago

That's an interesting application. I'm sure you already know this, but of course there is some information loss associated with compression etc. - I'm not sure how much would be acceptable for satellite imagery though.