search

lanha / DSen2

Super-Resolution of Sentinel-2 Images: Learning a Globally Applicable Deep Neural Network
GNU General Public License v3.0
229 stars 69 forks source link

Getting started? #4

Closed PCdurham closed 5 years ago

PCdurham commented 5 years ago

I read the paper with excitement but I'm having difficulty installing DSen2. Can we please have some installation instructions? I can't find the setup.py file so even a pip install from a local file doesn't work for me. But I'm not that experienced in python. I usually just use pip or conda installs so perhaps I am missing something.

Thanks Patrice

lanha commented 5 years ago

You should manually install the python packages either with pip or conda. What problem are you encountering?

PCdurham commented 5 years ago

The pip install is not working because there is no setup.py file. I can try a conda route, but that is usually for binaries? Ultimately, the example you suggest does not find the code.
Thanks P.S. all requirements are installed, including Keras with the TF backend.

PCdurham commented 5 years ago

Detailed error: (tensorflow) C:\Users\Patrice>pip install git+git://github.com/lanha/DSen2.git Collecting git+git://github.com/lanha/DSen2.git Cloning git://github.com/lanha/DSen2.git to c:\users\patrice\appdata\local\temp\pip-req-build-3u85r_9h Complete output from command python setup.py egg_info: Traceback (most recent call last): File "", line 1, in File "C:\Users\Patrice\Anaconda3\envs\tensorflow\lib\tokenize.py", line 452, in open buffer = _builtin_open(filename, 'rb') FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\Patrice\AppData\Local\Temp\pip-req-build-3u85r_9h\setup.py'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in C:\Users\Patrice\AppData\Local\Temp\pip-req-build-3u85r_9h\

lanha commented 5 years ago

No need to install DSen2, if you have the dependencies already just go ahead and run the demo script.

PCdurham commented 5 years ago

Ok thanks, progress but I think my install of gdal isn't supporting jp2. Error below. But I re-installed it and I still get the error. Looks like the it failed to load the first band. File name and path is correct but it seems like line 218 that opens the bands returned null that triggered the bug further down?

This is for a project with the Italian Environment agency (ISPRA), with POLIMI, working on Italian rivers and hopefully making S2 a part of operational procedures to monitor rivers in Italy. It would be a huge advantage to have high quality 10m images across all bands since most rivers are not much wider than 50-60 m. So we are interested in getting your code to work soon.

jp2 reading bug: (base) C:\Users\Patrice>python D:\DSen2\testing\s2_tiles_supres.py E:\S2B_MSIL1C_20170928T191139_N0205_R056_T10UFV_20170928T191953.SAFE\MTD_MSIL1C.xml E:\Dsen2output.tif Using TensorFlow backend. Selected UTM Zone: UTM 10N Selected pixel region: xmin=0, ymin=0, xmax=10979, ymax=10979: Image size: width=10980 x height=10980 Selected 10m bands: B4 B3 B2 B8 Selected 20m bands: B5 B6 B7 B8A B11 B12 Selected 60m bands: Loading selected data from: Bands B2, B3, B4, B8 with 10m resolution, UTM 10N ERROR 4: `E:\S2B_MSIL1C_20170928T191139_N0205_R056_T10UFV_20170928T191953.SAFE/GRANULE/L1C_T10UFV_A002942_20170928T191953/IMG_DATA/T10UFV_20170928T191139_B04.jp2' not recognized as a supported file format. Traceback (most recent call last): File "D:\DSen2\testing\s2_tiles_supres.py", line 315, in buf_ysize=ymax - ymin + 1), 0, 3)[:, :, validated_10m_indices] File "C:\Users\Patrice\Anaconda3\lib\site-packages\numpy\core\numeric.py", line 1450, in rollaxis n = a.ndim AttributeError: 'NoneType' object has no attribute 'ndim'

lanha commented 5 years ago

Have you tried using different versions of gdal? Also have a look here https://www.gdal.org/formats_list.html you might need extra libraries to use some file formats.

PCdurham commented 5 years ago

It works! But I had to force the MDL_Path to the absolute in supres. That might be worth a note somewhere. Results open in QGIS and look fabulous. Now a substantive question: What about L2A data produced with Sen2Cor? How do you recommend we incorporate this method into a standard workflow that includes atmospheric correction?

Many thanks for the code. Patrice

On Mon, Feb 4, 2019 at 11:06 AM Charis Lanaras notifications@github.com wrote:

Have you tried using different versions of gdal? Also have a look here https://www.gdal.org/formats_list.html you might need extra libraries to use some file formats.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lanha/DSen2/issues/4#issuecomment-460208631, or mute the thread https://github.com/notifications/unsubscribe-auth/As7ZyHCG_Nh1jjyWwStcoQil0SO8ajsAks5vKBRDgaJpZM4ae8p8 .

lanha commented 5 years ago

Can't you run Sen2Cor after the super-resolution?

PCdurham commented 5 years ago

I don't think so. Sen2Cor uses the same xml file as Dsen2 does. Running it on 10m bands, without knowing the detail of the Sen2Cor code would be very risky. What is needed here is a retraining of the model for atmospherically corrected data. Without this, your method will always be limited to single tiles and any serious project needing a generalized approach will not be able to benefit from it. ESA now provide L2A data for Europe back to March 2018 and it has just gone global. I would assume that the deeper feature weights in the CNN will not change so a transfer learning approach should be able to give results rapidly. Having that extra model in the github repo is al that is needed to make this change...

PCdurham commented 5 years ago

I just tried something: I copied the L2A data back into a duplicate of the L1C data folder structure. I.e. I made a fake L1C folder that in fact had L2A corrected data. Then I ran Dsen2 on this data (just, 20m, not 60m). I compared the results of the Dsen2 output to the original Sen2cor output and visually, they are excellent. I then downsampled the Dsen2 results back to 20m and compared that to the original Sen2Cor. They are virtually identical, the histogram of differences has about 98% of values at 0-100 difference over DNs that range upwards of 5000. a regression gave a slope of 1.0 with a shift of just 4.9 DNs. I do think Dsen2 shifts the geocoding just a bit, but nothing major. But overall it does not seem that DSen 2 causes a fundamental change in the radiometry of the image when run on L2A data. It seems to work if you ask me. So why does you paper state that Dsen2 is only for L1C data? If you think of it, radiance values from atmospheric corrections are not implicitly different. Patterns often remain and the DN values of L1C data in a given scene can easily be the same as L2A values for another scene. Why should a CNN model that functions globally not be valid under the small (but significant and important) change caused by atmospheric correction? Have you actually tested Dsen2 with L2A data or was that just a statement in the paper to ward off negative reviews?

lanha commented 5 years ago

Hi Patrice. I agree with you that DSen2 could work on L2A data, although there is no experimental evidence about it. The idea of the paper was to give a proof of concept of how a neural network can be applied to super-resolve a Sentinel-2 image. I haven't done any testing with L2A data, as it was out of the scope of the paper. The weights are optimal for L1C, that's why the paper states it should be used for L1C. If there is a way to run DSen2 after the Sen2cor model, that's fine. Otherwise I could try to train a model that works best with L2A data.