Having a little issue with my own cog geotiff similiar to the 500 error

[bmulcahy]

Very similar too issue 8

I've created my own COG geotiff and have it stored on a publicly available s3 bucket link to geotiff

here is what the meta data looks like from /tiles

{"bounds":[-93.58172838070541,38.45736064566228,-91.91929051352936,40.082810555216696],"center":[-92.7505094471174,39.27008560043949,13],"maxzoom":19,"minzoom":6,"name":"Untitled","tilejson":"2.1.0","tiles":["//tiles.rdnt.io/tiles/{z}/{x}/{y}?url=https%3A%2F%2Fs3.us-east-2.amazonaws.com%2Fusace-dev-tiles-iwrss%2Ftiles%2FFLOOD_DEPTH%2F45_37.tif"]}

here is what gdalinfo gives me on my image

$ gdalinfo ./45_37_cog.tif
Driver: GTiff/GeoTIFF
Files: ./45_37_cog.tif
Size is 55957, 54712
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433],
    AUTHORITY["EPSG","4326"]]
Origin = (-93.581728380705414,40.082810555216696)
Pixel Size = (0.000029709202909,-0.000029709202909)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  COMPRESSION=DEFLATE
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  ( -93.5817284,  40.0828106) ( 93d34'54.22"W, 40d 4'58.12"N)
Lower Left  ( -93.5817284,  38.4573606) ( 93d34'54.22"W, 38d27'26.50"N)
Upper Right ( -91.9192905,  40.0828106) ( 91d55' 9.45"W, 40d 4'58.12"N)
Lower Right ( -91.9192905,  38.4573606) ( 91d55' 9.45"W, 38d27'26.50"N)
Center      ( -92.7505094,  39.2700856) ( 92d45' 1.83"W, 39d16'12.31"N)
Band 1 Block=512x512 Type=Float32, ColorInterp=Gray
  Min=0.001 Max=158.545
  Minimum=0.001, Maximum=158.545, Mean=13.571, StdDev=10.970
  NoData Value=-9999
  Overviews: 27979x27356, 13990x13678, 6995x6839, 3498x3420, 1749x1710, 875x855, 438x428
  Metadata:
    STATISTICS_MAXIMUM=158.54547119141
    STATISTICS_MEAN=13.570900384542
    STATISTICS_MINIMUM=0.0009765625
    STATISTICS_STDDEV=10.970317641044

I'm not sure what is going on. when I tried it with my own lambda function I get a memory related error. Any thoughts to what is occurring?

[mojodna]

When I ran this locally, I got a stack trace that looked like this:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2309, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2295, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1741, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/marblecutter/virtual/web.py", line 120, in render_png
    scale=scale,
  File "/usr/local/lib/python2.7/dist-packages/marblecutter/tiling.py", line 47, in render_tile
    expand=expand,
  File "/usr/local/lib/python2.7/dist-packages/marblecutter/__init__.py", line 349, in render
    sources, bounds, shape, target_crs, expand
  File "/usr/local/lib/python2.7/dist-packages/marblecutter/mosaic.py", line 97, in composite
    source.recipes, window_data, source=source, expand=expand
  File "/usr/local/lib/python2.7/dist-packages/marblecutter/recipes.py", line 24, in apply
    dtype_min = np.iinfo(data.dtype).min
  File "/usr/local/lib/python2.7/dist-packages/numpy/core/getlimits.py", line 506, in __init__
    raise ValueError("Invalid integer data type %r." % (self.kind,))
ValueError: Invalid integer data type 'f'.

I fixed that in: https://github.com/mojodna/marblecutter/commit/29f75ecaed0d4fb15de6872fac2efac7f90842b6

After, it looks like:

(This is rolling out to tiles.rdnt.io now)

[mojodna]

It looks a bit cyclical, similar to what @fredliporace noted in https://github.com/mojodna/marblecutter/commit/0fdc263bc519503f81ff8db8ed6a62248cf079dd#commitcomment-32508381

I'm going to see if I can get it to render a more normal ramp.

[mojodna]

Reopening to deal with output weirdness.

[mojodna]

This is what I have locally now; I need to do some more cleanup before merging it:

[bmulcahy]

If you need any more information let me know.

[mojodna]

Much better:

CBERS also looks much better:

[bmulcahy]

That does look much better. Was it just an issue of the data ranges? Also are your docker scripts current which one should I use? This, marblecutter, or marble cutter virtual?Also for questions like these which is the best repo to ask against?

[mojodna]

No, it was how data was being rescaled (and the array types into which it was being updated). I think I'd missed it before because I was visualizing other float data (DEMs in that case) as hillshades, rather than using the translation to greyscale imagery.

The fix was in https://github.com/mojodna/marblecutter/commit/8331bf19825bd967870b29ebbde7be6946e1ba17

https://github.com/mojodna/marblecutter-virtual#running-locally is probably the best way to get things running yourself (that's what I use for working on tiles.rdnt.io), as it's the most generic method to render tiles. If you need to do something more custom, have a look at the various Catalog implementations.

Practically all of the rendering problems are issues with marblecutter, so that's likely the most sensible place to ask things.