Closed Esteban82 closed 1 year ago
If I undertand the logic behing packing closer via scale/offsets, I think that in this case there is no case to do it.
So, I should use these values.
# DST_SCALE=1
# DST_OFFSET=0
Am I right?
But 31060 cannot be right since just googling what the peak is gives 21,229. It may have changed a tiny bit though. Should still fit with a scale of 0.5 but may need to adjust the offset so that value = (max - offset ) /scale and (min - offset) / scale both fit inside ±32767 m
Something is odd with my gmt convert to nc from TIFF anyway:
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: v_min: -6575 v_max: 0 name: z
Should still fit with a scale of 0.5 but may need to adjust the offset so that value = (max - offset ) /scale and (min - offset) / scale both fit inside ±32767 m
Are you sure? (31060 + 8528 ) * 0.5 = 79176.
I have to change the scale to a value greater than 0.6 to have less than 65534 values.
But where is 31060 coming from if Wikipedia says Olympus Mons is just over 20 km?
From the grid itself. I use gmt grdinfo
and I got: v_min: -8528 v_max: 31060 name: z
. I got the same result when I did it on the tiff file.
Well, here is a quote:
"The full range of topography on Mars is about 19 miles (30 kilometers), one and a half times the range of elevations found on Earth," noted Dr. David Smith of NASA's Goddard Space Flight Center, Greenbelt, MD, the principal investigator for MOLA
. S as long as the range is < 32767 m then we can use a scale of 0.5 and a suitable offset and store in units of 50 cm.
So, I guess that there is problem with the grid.
Or maybe I used the wrong one.
This is from the 11 GB tif.
$ gmt grdinfo Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: Title: Grid imported via GDAL
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: Command:
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: Remark:
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: Pixel node registration used [Geographic grid]
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: Grid file format: gd = Import/export through GDAL
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: x_min: -180 x_max: 179.998447904 x_inc: 0.00337412083064 name: x n_columns: 106694
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: y_min: -89.9992239522 y_max: 90 y_inc: 0.00337412083064 name: y n_rows: 53347
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: v_min: -8528 v_max: 31060 name: z
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: scale_factor: 1 add_offset: 0
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: Default CPT:
And this from the 3.4 GB NC grid:
$ gmt grdinfo Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Title: Grid imported via GDAL
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Command:
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Remark:
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Pixel node registration used [Geographic grid]
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Grid file format: nf = GMT netCDF format (32-bit float), CF-1.7
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: x_min: -180 x_max: 180 x_inc: 0.00337413537781 name: longitude n_columns: 106694
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: y_min: -90 y_max: 90 y_inc: 0.00337413537781 name: latitude n_rows: 53347
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: v_min: -8528 v_max: 31060 name: z
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: scale_factor: 1 add_offset: 0
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: format: netCDF-4 chunk_size: 129,129 shuffle: on deflation_level: 3
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Default CPT:
GEOGCS["unknown",
DATUM["unknown",
SPHEROID["unknown",3396190,0]],
PRIMEM["Reference meridian",0],
UNIT["degree",0.0174532925199433,
AUTHORITY["EPSG","9122"]],
AXIS["Longitude",EAST],
AXIS["Latitude",NORTH]]
Does gdalinfo
say the same thing?
Well, I made a map of Olympus and its top is barely over 21 km.
gmt grdimage Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc -R222/232/14/21 -png Olympus -JW15 -Baf
Does
gdalinfo
say the same thing?
Yes, the same.
actual_range={-8528,31060}
$ gdalinfo Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc
Driver: netCDF/Network Common Data Format
Files: Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc
Size is 106694, 53347
Coordinate System is:
GEOGCRS["unknown",
DATUM["unknown",
ELLIPSOID["unknown",3396190,0,
LENGTHUNIT["metre",1,
ID["EPSG",9001]]]],
PRIMEM["Reference meridian",0,
ANGLEUNIT["degree",0.0174532925199433,
ID["EPSG",9122]]],
CS[ellipsoidal,2],
AXIS["longitude",east,
ORDER[1],
ANGLEUNIT["degree",0.0174532925199433,
ID["EPSG",9122]]],
AXIS["latitude",north,
ORDER[2],
ANGLEUNIT["degree",0.0174532925199433,
ID["EPSG",9122]]]]
Data axis to CRS axis mapping: 1,2
Origin = (-180.000000000000000,90.000000000000000)
Pixel Size = (0.003374135377809,-0.003374135377809)
Metadata:
grid_mapping#spatial_ref=GEOGCS["unknown",
DATUM["unknown",
SPHEROID["unknown",3396190,0]],
PRIMEM["Reference meridian",0],
UNIT["degree",0.0174532925199433,
AUTHORITY["EPSG","9122"]],
AXIS["Longitude",EAST],
AXIS["Latitude",NORTH]]
lat#actual_range={-90,90}
lat#axis=Y
lat#long_name=latitude
lat#standard_name=latitude
lat#units=degrees_north
lon#actual_range={-180,180}
lon#axis=X
lon#long_name=longitude
lon#standard_name=longitude
lon#units=degrees_east
NC_GLOBAL#Conventions=CF-1.7
NC_GLOBAL#description=
NC_GLOBAL#GMT_version=6.4.0 [64-bit] [MP]
NC_GLOBAL#history=
NC_GLOBAL#node_offset=1
NC_GLOBAL#title=Grid imported via GDAL
z#actual_range={-8528,31060}
z#grid_mapping=grid_mapping
z#long_name=z
z#_FillValue=-nan
Corner Coordinates:
Upper Left (-180.0000000, 90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"N)
Lower Left (-180.0000000, -90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"S)
Upper Right ( 180.0000000, 90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"N)
Lower Right ( 180.0000000, -90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"S)
Center ( 0.0000000, 0.0000000) ( 0d 0' 0.01"E, 0d 0' 0.01"N)
Band 1 Block=129x129 Type=Float32, ColorInterp=Undefined
NoData Value=nan
Metadata:
actual_range={-8528,31060}
grid_mapping=grid_mapping
long_name=z
NETCDF_VARNAME=z
_FillValue=-nan
And what about?
gmt grdinfo Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc -L0
I got the same.
Later I will make a map to see where the high values are.
I would have swore I did run through Mars many months ago and got sensible results. I have not refreshed the original TIFF file nor nc file I created. So I was going to blame GDAL but not sure. How come I get max = 0 while @Esteban82 gets 31k?
Later I will make a map to see where the high values are.
grdinfo -M
tells you that.
gmt grdinfo Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc -M Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Title: Grid imported via GDAL Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Command: Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Remark: Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Pixel node registration used [Geographic grid] Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Grid file format: nf = GMT netCDF format (32-bit float), CF-1.7 Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: x_min: -180 x_max: 180 x_inc: 0.00337413537781 name: longitude n_columns: 106694 Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: y_min: -90 y_max: 90 y_inc: 0.00337413537781 name: latitude n_rows: 53347 Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: v_min: -6575 at x = -7.92078279941 y = 69.9154591636 v_max: 0 at x = -179.998312932 y = 68.1541604964 Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: scale_factor: 1 add_offset: 0 Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: 6474 nodes (0.0%) set to NaN
I found some strange values near the north pole!! They are also in the tiff file.
$ gmt grdinfo Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc -M
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Title: Grid imported via GDAL
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Command:
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Remark:
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Pixel node registration used [Geographic grid]
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Grid file format: nf = GMT netCDF format (32-bit float), CF-1.7
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: x_min: -180 x_max: 180 x_inc: 0.00337413537781 name: longitude n_columns: 106694
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: y_min: -90 y_max: 90 y_inc: 0.00337413537781 name: latitude n_rows: 53347
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: v_min: -8528 at x = 62.1397641854 y = -32.7999700077 v_max: 31060 at x = -133.779406527 y = 88.4360882524
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: scale_factor: 1 add_offset: 0
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: 39956 nodes (0.0%) set to NaN
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: format: netCDF-4 chunk_size: 129,129 shuffle: on deflation_level: 3
Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc: Default CPT:
Maybe we both had transmission issues or theirs has errors. I find it hard to believe theirs has errors given widespread usage?
@joa-quim any ideas?
Nope, but I guess I'll have to untie it. Where is that Mars tiff file?
Does it have a checksum?
Sorry. I checked the wrong file.
e98092a06bea74d5ec9a704dcd1fe318 Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif
So you can check if the checksum of the download file is the same.
Is it ok with this command md5sum Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif
?
If so, I got a different hash. b18c673dddd8c22a1ae1995f20c1540a *Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif
When I download the file, I had some issues. Do you have any type on how to download it? Any software to use?
I'm having troubles downloading that file with Firefox. Connection keeps breaking. Maybe try with wget
I am trying curl which is what our src_downloader_grid.sh script does. Good you are trying wget
I can't download it. Permanently breaking connection.
Yep:
curl: (18) transfer closed with 11023654838 bytes remaining to read
I am trying another source. It has an original GMT grid that is 43 Gb and the 11 Gb tiff.
This went very fast and gave this output which seems sane:
gmt grdinfo Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: Title: Grid imported via GDAL Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: Command: Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: Remark: Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: Pixel node registration used [Geographic grid] Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: Grid file format: gd = Import/export through GDAL Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: x_min: -180 x_max: 179.998447904 x_inc: 0.00337412083064 name: x n_columns: 106694 Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: y_min: -89.9992239522 y_max: 90 y_inc: 0.00337412083064 name: y n_rows: 53347 Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif: v_min: -8528 v_max: 21226 name: z
Any luck for you to download this file? If should I place it in my SOEST ftp just in case?
Sorry, I forgot to ask you if was going to do it. I can do it. I have no problem.
I can't donwload it. Yes, put it your SOEST, please.
Now I could enter. I will let you know if I have any problem.
I was unable to copy over the juge TIF file but running the grdedit command (see _marsrecipe) I was able to copy over the NC file. So, you can get that from my public ftp. File is Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.nc. If you place that in the staging area then the recipe should detect that and skip on to the conversions.
Thanks Paul. I manage to download it. I will try to process it on Monday.
I download the file. It is ok.
The cheksum is the same that USGS's.
md5sum staging/Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif
e98092a06bea74d5ec9a704dcd1fe318 *staging/Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif
USGS:
e98092a06bea74d5ec9a704dcd1fe318 Mars_HRSC_MOLA_BlendDEM_Global_200mp_v2.tif
I run this code and I got this image. So this issus can be closed.
gmt grdimage @mars_relief -Rg -Bf -Cgeo -png Mars
I run the code to process Mars MOLA grid (see below). The range of grid has values that do not match those of the recipe. As a result, the higher grid resolution weren't created (up to 30s included).
Info from
mars_relief.recipe
:# We use a precision of 0.5 m with a 6000 m offset to fit the range of -8528 to +21226 in 16-bit ints
Info from gmt grdinfo:v_min: -8528 v_max: 31060 name: z
I think the Mount Olimpus is higher with a higher resolution.
Solution
So, I think that I have to change these values so the script would work.
I will try to figure out the new values.
Full message when I run the code: