Invalid meta tiles for DEM causes severe performance degradation in renderer

[h0od]

Version: mapproxy 1.35 (built on Jul 13 2018 12:57:52 at pomerol)

I'm encountering an issue during rendering (both desktop and javascript) that I think is related to how mapproxy generates tiling index or how it generates meta tiles. The rendering gets extremely sluggish (<1FPS). I debugged the vts-browser-cpp and noticed that when the problem occurs, the application spends several seconds when traversing the node tree each frame. The statistics show that it traverses of millions of higher LODs nodes even when the renderer is currently displaying a very low LOD.

I'm using a custom reference frame that covers entire Sweden. The DEM dataset covers most of Sweden and consists of several GeoTIFF files but I have combined them in a virtual raster. I can give you a link to the actual GeoTIFF files if required. There are a lot of NODATA parts outside Sweden's borders. I've generated the tiling index using mapproxy-tiling.

From my debugging, I've concluded that tiles that are completely in a NODATA area seem to have meta data in them that causes the renderer to continue traversing meta tiles very deep.

I made a quick workaround in vts-browser-cpp library that ignores the meta tile data and stops the traversal. Note, this is just a workaround not a fix! https://github.com/h0od/vts-browser-cpp/commit/beef90ebc0f1518b181e415e0f69b377352dafb7

SRS:

"sweref99tm": {
        "comment": "SWEREF99 TM",
        "srsDef": "+proj=utm +zone=33 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs",
        "srsModifiers": [ "adjustVertical" ],
        "type": "projected"
    },

Reference frame:

{
        "version" : 1,
        "id": "sweden",
        "description": "",
        "body": "Earth",

        "model": {
            "physicalSrs": "sweref99tm",
            "navigationSrs": "sweref99tm",
            "publicSrs": "geographic-wgs84"
        },

        "division": {
            "extents" : {
                "ll": [218128, 6126002, -500], "ur": [1083427, 7692850, 3000] },
            "heightRange": [-500,3000],
            "nodes": [
                {
                    "id": { "lod": 0, "position": [0,0] },
                    "srs": "sweref99tm",
                    "externalTexture": true,
                    "extents": {
                        "ll": [218128, 6126002], "ur": [1083427, 7692850] },
                    "partitioning": "bisection"
                }
            ]
        },

        "parameters": {
            "metaBinaryOrder": 5
        }
    }

freelayer.json

{
    "credits" : [],
    "lodRange" : [ 0, 15 ],
    "meshUrl" : "{lod}-{x}-{y}.bin?0",
    "metaUrl" : "{lod}-{x}-{y}.meta?0",
    "textureUrl" : "{lod}-{x}-{y}-{sub}.jpg?0",
    "tileRange" : 
    [
        [ 0, 0 ],
        [ 0, 0 ]
    ],
    "type" : "mesh-tiles"
}

[ladislavhorky]

Hi h0od,

thanks for reporting. I'll need some additional info before we look into it - how are you actually viewing the tileset?

The freelayer.json part seems suspicious, you should not need a free layer to do this. Are you viewing the tileset on mapproxy (simply browsing to it) or do you have it in the VTS storage? Or none of above?

Could you please send us the mapConfig.json - the first file browser downloads when rendering the tileset?

Thanks, Ladislav

[h0od]

Hi, thanks for your response!

I only included the freelayer.json because I thought it contained enough information about the surface. I'm not using it as a free layer though, I'm using it as a surface. Sorry if I had you confused. I've included the mapConfig.json at the bottom of this comment.

I've tried several ways of viewing it, both using web browser (by just browsing to it like you said) and also by using Unity with the vts-browser.dll. I've tried using the mapConfig.json generated by mapproxy and also by creating a storage view served by vtsd (using the remote surface from mapproxy in the VTS storage). I get the same result no matter what I try.

I have the same result no matter which or how many bound layers used. I've also tried using the sweden/tms/.system/tms-raster-patchwork/boundlayer.json as the bound layer. Same problem occurs, so I don't think it has to do with my bound layers.

If I remember correctly I did not have this issue when using the melown2015 reference frame.

Usually, I experience the issue immediately when starting the renderer. But sometimes, after my browser cache is cleared and the mapproxy docker container is restarted, it takes some time (seconds) before I experience the issue. But I think that's related to the fact that the renderer doesn't fully traverse the quadtree while the meta-files are being fetched, which can take some time.

I will gladly provide you with more information. I'm starting to grasp the internal architecture of mapproxy and the renderer, but I don't have the proper setup here to debug mapproxy.

Below is the mapConfig.json generated by mapproxy. Using only one bound layer:

{
    "bodies" : 
    {
        "Earth" : 
        {
            "atmosphere" : 
            {
                "colorHorizon" : [ 158, 206, 255, 255 ],
                "colorZenith" : [ 62, 120, 229, 255 ],
                "thickness" : 100000,
                "visibility" : 100000
            },
            "class" : "planet",
            "comment" : "Earth, Sol III, M-class planet.",
            "parent" : "Sun"
        },
        "Sun" : 
        {
            "class" : "star",
            "comment" : "Sol, G2V main-sequence star."
        }
    },
    "boundLayers" : 
    {
        "test1-sweden_50m_orto" : "../../../tms/test1/sweden_50m_orto/boundlayer.json"
    },
    "credits" : 
    {
        "goddard" : 
        {
            "id" : 202,
            "notice" : "Goddard Space Flight Center"
        }
    },
    "freeLayers" : {},
    "glue" : [],
    "namedViews" : {},
    "params" : {},
    "position" : 
    [
        "obj",
        725000.0625,
        7532500.0625,
        "float",
        0,
        0,
        -90,
        0,
        5413.1280850000003,
        45
    ],
    "referenceFrame" : 
    {
        "body" : "Earth",
        "description" : "",
        "division" : 
        {
            "extents" : 
            {
                "ll" : [ 218128, 6126002, -500 ],
                "ur" : [ 1083427, 7692850, 3000 ]
            },
            "heightRange" : [ -500, 3000 ],
            "nodes" : 
            [
                {
                    "extents" : 
                    {
                        "ll" : [ 218128, 6126002 ],
                        "ur" : [ 1083427, 7692850 ]
                    },
                    "externalTexture" : true,
                    "id" : 
                    {
                        "lod" : 0,
                        "position" : [ 0, 0 ]
                    },
                    "partitioning" : "bisection",
                    "srs" : "sweref99tm"
                }
            ]
        },
        "id" : "sweden",
        "model" : 
        {
            "navigationSrs" : "sweref99tm",
            "physicalSrs" : "sweref99tm",
            "publicSrs" : "geographic-wgs84"
        },
        "parameters" : 
        {
            "metaBinaryOrder" : 5
        },
        "version" : 1
    },
    "rois" : [],
    "services" : 
    {
        "atmdensity" : 
        {
            "url" : "atmdensity.png?def={param(0)}"
        }
    },
    "srses" : 
    {
        "geographic-wgs84" : 
        {
            "comment" : "Geographic, WGS84 (epsg:4326)",
            "srsDef" : "+proj=longlat +datum=WGS84 +no_defs",
            "type" : "geographic"
        },
        "sweref99tm" : 
        {
            "comment" : "SWEREF99 TM",
            "srsDef" : "+proj=utm +zone=33 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs",
            "srsModifiers" : [ "adjustVertical" ],
            "type" : "projected"
        }
    },
    "surfaces" : 
    [
        {
            "2d" : 
            {
                "creditsUrl" : "{lod}-{x}-{y}.credits?0",
                "maskUrl" : "{lod}-{x}-{y}.mask?0",
                "metaUrl" : "{lod}-{x}-{y}.2dmeta?0",
                "orthoUrl" : "{lod}-{x}-{y}-{sub}.ortho?0"
            },
            "id" : "test1-nh_riks_Sweref_99_TM_geotiff-dem",
            "lodRange" : [ 0, 15 ],
            "meshUrl" : "{lod}-{x}-{y}.bin?0",
            "metaUrl" : "{lod}-{x}-{y}.meta?0",
            "navUrl" : "{lod}-{x}-{y}.nav?0",
            "textureUrl" : "{lod}-{x}-{y}-{sub}.jpg?0",
            "tileRange" : 
            [
                [ 0, 0 ],
                [ 0, 0 ]
            ]
        }
    ],
    "textureAtlasReady" : false,
    "version" : 1,
    "view" : 
    {
        "freeLayers" : {},
        "surfaces" : 
        {
            "test1-nh_riks_Sweref_99_TM_geotiff-dem" : [ "test1-sweden_50m_orto" ]
        }
    },
    "virtualSurfaces" : []
}

[ladislavhorky]

Hi, h0od

ok, it really looks like some issue in generating the tiling or metatiles - could you please provide us with original GeoTiffs or link to them so we can replicate it?

Thanks, Ladislav

[h0od]

could you please provide us with original GeoTiffs or link to them so we can replicate it?

Absolutely! But I'd prefer if I could send you the link privately to avoid any copyright issues.

[ladislavhorky]

Hi, you can send the link to ladislav.horky (at) melown (dot) com.

[h0od]

Hi, you can send the link to ladislav.horky (at) melown (dot) com.

I've sent an email with a link. Hopefully your spamfilter won't flag it :)

[ladislavhorky]

Hi, we are downloading the data. We will let you know once we fix the issue. However, I cannot guarantee the time-frame.

[h0od]

Hi, we are downloading the data. We will let you know once we fix the issue. However, I cannot guarantee the time-frame.

Thank you! No problem, I have my workaround for now :) Right now I'm just evaluating the use of your VTS stack for an upcoming project. Looks promising by the way!

[vaclavblazek]

I've finally got time to dig in the issue. It was caused by resource being deeper than dataset's reference frame tiling. Fixed in 37b0d3dca3ff5c162d74966dd0b12c6140628e88. Will close once ubuntu package is built.

[ladislavhorky]

Closing as fixed.