SAR images with complex data need to have non-complex bands

[mortenwh]

Many packages depending on nansat expect some standard SAR bands like sigma0 in VV pol to be in real numbers. It will therefore be useful to adjust the SAR mappers to use the IntensityPixelFunc to provide sigma0 in magnitude with band names sigma0_VV, sigma0_HH, etc. I propose to add "_complex" to the suffix of bands with complex data.

[mortenwh]

I will start this by modifying the radarsat2 mapper. It should later be followed up by making a common class for all SAR data.

[mortenwh]

I suggest to leave this issue open until the other mappers have been updated. At least the ASAR mapper need an update...

[asumak]

Do you know which ASAR file have complex data?

[mortenwh]

https://earth.esa.int/handbooks/asar/CNTR6-2-9.htm

and

https://earth.esa.int/handbooks/asar/CNTR6-2-13.htm

On 29 November 2013 14:00, Asuka Yamakawa notifications@github.com wrote:

Do you know which ASAR file have complex data?

— Reply to this email directly or view it on GitHubhttps://github.com/nansencenter/nansat/issues/39#issuecomment-29514930 .

[asumak]

GDAL does not support 'ASA_IMS_1P'.

[asumak]

01eb5de : Create intensity pixelfunctions for Integer. Add it to asar mapper. As far as I know, ASAR complex data is CInt16. But if it is CFloat32 or CFloat64, the conventional intensity pixelfunction is chosen.

[asumak]

fixed in 1f3e7c5.

[asumak]

Created test function in test_mappers and tested. It works fine.

[akorosov]

Merged into develop in 56d5377308b60f11d820ffcba88001c447d40b85

[mortenwh]

It is not the raw counts that should be modified. We need extra band(s) of real valued sigma0 arrays.

[asumak]

If 'raw_counts' band is complex, it is necessary to set 'SourceTransferType' in VRT. If 'raw_counts' and 'incidence angle' bands are passed to the pixel functions, 'SourceTransferType' is applied to the both bands.It is impossible to set 'SourceTransferType' to each band. Therefore the extra bands, converted from complex to real and imaginary parts, have to be added.

[asumak]

Can you please check bb4bdbe, Morten? And please correct wkt, sphPass, other metadata. (line 204, 205, 238 etc...)

[mortenwh]

There is a lot of code changes in bb4bdbed9644d81e8c8b99712d2744221fea7f29 - have you written a test?

[asumak]

OK. I will do that.

[knutfrode]

The error message "NetCDF: One or more variable sizes violate format constraints" normally means that one tries to create a file larger than permitted for the file format, normally 2 GB for netCDF. http://www.unidata.ucar.edu/software/netcdf/faq-lfs.html

[asumak]

I made a test for ASAR sigma0 band in 5cf64e6. Although I can get complex data array, I get an error as " Error in `python': munmap_chunk(): invalid pointer: 0x0a17c6e8". It seems a memory problem of my virtual machine. Can you test it, Morten?

[mortenwh]

I have rewritten the test to prepare it for testing all SAR mappers. I also changed the standard_name of complex data in the asar mapper. The standard_name of the Normalized Radar Cross Section should be "surface_backwards_scattering_coefficient_of_radar_wave" regardless of whether it is complex or not. By using the standard_name, it is not so important if mappers don't follow the name-convention suggested above. In order to get the test returning all complex bands (not only one) I had to change the function Nansat._get_band_number. Please double-check that this is ok. Can you extend the test to also take Sentinel1 and others?

The test is failing on the ASAR example file ASA_IMS_1PNIPA20100411_101715_000000162088_00280_42418_0338.N1 at line 171 with "GDALError: Cannot read array from band 7", so there must be a bug somewhere.

[mortenwh]

See 38b17120dd9d5b5a4c84b534eed142f628fa5200 for latest commit

[asumak]

Regarding "GDALError: Cannot read array from band 7": Isn't it a memory problem? I have the same problem. But after resizeing with eResampleAlg=0, I can read the array although I get " Error in `python': munmap_chunk(): invalid pointer:.

[mortenwh]

I had forgotten to compile the pixel functions... The ASAR file now passes the test, but there is a radarsat-2 product that doesn't - can you check @asumak?

[asumak]

ok. I do.

[asumak]

The problem with RS2 was solved in a508744. I will continue for setinel1.

[asumak]

Added 'name' for getting a band number in 6fee462 because some seninel-1 data have some bands whose names are 'DN_XX'. e.g. 'S1A_IW_SLC__1SDV_20150317T163128_20150317T163156_005074_0065FB_2245.zip' has DN_VH, DN_VH_000 and DN_VH_001.

[asumak]

I think it is good to make tests for sentinel-1 when we modify the mapper for complex data.