r-barnes / ArcRasterRescue

Extract raster data from ArcGIS/ESRI formats
MIT License
84 stars 25 forks source link

"Could not map native type 'a' to GDAL type!" #9

Open jwilson8767 opened 3 years ago

jwilson8767 commented 3 years ago

Hello, I was eager to try out v1.0.0, but after successfully compiling I ran into an issue that I don't know how to troubleshoot.

OS: Ubuntu 18.04 (in WSL) Source gdb file: https://coast.noaa.gov/htdata/Inundation/SLR/FloodFreqData/Flood_Frequency_data_dist.zip Desired layer number: 22 Error as follows:

Projection: GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]]
Could not map native type 'a' to GDAL type! (Use `c++filt -t` to decode.)
terminate called after throwing an instance of 'std::runtime_error'
  what():  Could not map native data type to GDAL type!
Aborted
r-barnes commented 3 years ago

Next time please include the exact command you ran.

jwilson8767 commented 3 years ago

./build/arc_raster_rescue.exe FINAL_HTF_Minor.gdb 22 ./NC_2_HTF_MINOR.tif

r-barnes commented 3 years ago

@jwilson8767 : I've just pushed a fix that might work. It's on master.

r-barnes commented 3 years ago

Do you have any idea what the data should look like?

jwilson8767 commented 3 years ago

Wow crazy fast turn around! Thanks!

I realized I said layer 22, but later in the day I had found that later 25 (NC_5_HTF_MINOR) is what I'm really after for at this time.

Here's the result of manually exporting the data via ArcMap -> Copy Raster: https://drive.google.com/file/d/1K294VXRE8Kli_H5Aeo-Kx1vVCjHKjW7m/view?usp=sharing

And here's what I got from the latest change you made: https://drive.google.com/file/d/1E4HYuPSwvsHyDcwTaBXZjFReNGcdAKBv/view?usp=sharing

Notably, the NODATA value isn't being detected properly, and it looks like it's malprojected.

Here's a screenshot of the source info from QGIS from the version exported via ArcMap: image

And here's the same source info from the rescued version: image

Is it maybe just mis-encoding some of the CRS info and the NODATA value?

r-barnes commented 3 years ago

Are you sure the Arc exported data is in the unsigned byte type? It was encoded as a signed byte, which is what I've tried to export.

r-barnes commented 3 years ago

It looks as though the number of cells differ between Arc's export and the rescued version. That's probably an effect of no-data blocks.

r-barnes commented 3 years ago

I think the CRS is coming out right, at least on my end. QGIS might have reprojected yours? I'm getting:

GEOGCS["GCS_North_American_1983",DATUM["D_North_American_1983",SPHEROID["GRS_1980",6378137.0,298.257222101]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]]
r-barnes commented 3 years ago

The NODATA value is tricky. If you'd like to help with that, what I need is perhaps five separate small raster GDBs which are all the same except for the NODATA value. I also need to know what the value is. Having these for several different data types would probably also be useful.

jwilson8767 commented 3 years ago

Ok! This is by no means exhaustive, but I used ArcMap to draw a (vector) circle in a specific projection (EPSG 3631), then used "Copy Raster" to generate a series of raster versions of the circle based on several types (uint8, int8, and float32) and a couple NODATA values. The circle received the value 1 (based on its OBJECTID) in all rasters.

The circle: https://drive.google.com/file/d/1ec-YQ2Pg1l-NJ8EypbCypRJ40Qusbr24/view?usp=sharing The GDB: https://drive.google.com/file/d/1gjtsRM59LoC_LbB8O-22YAvtnb0w_V1M/view?usp=sharing

Using ArcRasterRescue on each of the raster layers produced differing results, some of which QGIS correctly detected (such as when NODATA is 0), but I'm not sure if the nodata value was actually being set properly on the raster or if QGIS just guessed.

Anyway, hopefully this is enough to figure out what's going on. I'm happy to generate other samples, if needed. ArcMap only crashed 5 times on me for this process, so I figure it's my lucky day :P

r-barnes commented 3 years ago

@jwilson8767 : Heads up: I've accepted a PR that should (hopefully) provide better affine projection transformations. I'm not sure if this will impact your results at all.