Issue using mapproxy-setup-resource on vrt

[jrjdavidson]

Hi,

I tried using mapproxy-setup-resource on a vrt dataset and got the error below- it looks like vrts are a no go?

Jonathan

vts@darkstar:~/mapproxy/datasets$ mapproxy-setup-resource --referenceFrame melown2015 --id NZ8mfull --group DEM --dataset NZ8mDEM/NZ8mDEM.vrt --attribution "<a href=//www.linz.govt.nz/data/linz-data/linz-basemaps/data-attribution>LINZ CC BY 4.0 © Imagery Basemap contributors</a>"
2020-09-02 16:11:13 I3 [2436(main)]: Loaded configuration from "/etc/vts/mapproxy/mapproxy-setup-resource.conf". {program.cpp:configureImpl():540}
2020-09-02 16:11:13 I3 [2436(main)]: [mapproxy-setup-resource] Config:
mapproxy.dataRoot = "/var/vts/mapproxy/datasets"
mapproxy.definitionDir = "/etc/vts/mapproxy/auto.d"
mapproxy.ctrl = "/var/run/vts-backend-mapproxy.ctrl"
referenceFrame = melown2015
credits.firstNumericId = 32768
bottomLod = none
 {main.cpp:configure():213}
2020-09-02 16:11:13 I4 [2436(main)]: Using resource ID <melown2015/DEM/NZ8mfull>. {main.cpp:run():712}
2020-09-02 16:11:13 I4 [2436(main)]: Checking for resource existence. {main.cpp:run():716}
2020-09-02 16:11:13 I4 [2436(main)]: Measuring dataset. {main.cpp:run():736}
2020-09-02 16:11:13 I3 [2436(main)]: Using credit ID <auto-a-href-www-linz-govt-nz-data-linz-data-linz-basemaps-data-attribution-linz-cc-by-4-0-imagery-basemap-contributors-a> for attribution "<a href=//www.linz.govt.nz/data/linz-data/linz-basemaps/data-attribution>LINZ CC BY 4.0 © Imagery Basemap contributors</a>". {main.cpp:attribution2credit():443}
2020-09-02 16:11:13 I4 [2436(main)]: Copying dataset to destination. {main.cpp:operator()():780}
2020-09-02 16:11:13 E2 [2436(main)]: gdal error 4: /var/vts/mapproxy/datasets/NZ8mDEM.vrt.faaf24bf3b337161aa8a1f43a94cb216/original-dataset/NZ8mDEM.vrt: No such file or directory {geodataset.cpp:GDALErrorHandler():76}
2020-09-02 16:11:13 I4 [2436(main)]: Generating dataset overviews. {main.cpp:createVrtWO():572}
2020-09-02 16:11:13 I4 (ovr) [2436(main)]: Generating height overviews. {main.cpp:createVrtWO():577}
2020-09-02 16:11:13 I3 (dem) [2436(main)]: Creating dataset base in "/var/vts/mapproxy/datasets/NZ8mDEM.vrt.faaf24bf3b337161aa8a1f43a94cb216.tmp/vttwo/dem/dataset" from "/var/vts/mapproxy/datasets/NZ8mDEM.vrt.faaf24bf3b337161aa8a1f43a94cb216.tmp/vttwo/dem/./original". {generatevrtwo.cpp:buildDatasetBase():523}
2020-09-02 16:11:13 I3 (dem) [2436(main)]: About to generate 17 overviews with 934 tiles of size 4096x4096. {generatevrtwo.cpp:generate():1026}
2020-09-02 16:11:13 I3 (dem) [2436(main)]: Creating overview #0 of 682 tiles in "/var/vts/mapproxy/datasets/NZ8mDEM.vrt.faaf24bf3b337161aa8a1f43a94cb216.tmp/vttwo/dem/0/ovr.vrt" from "/var/vts/mapproxy/datasets/NZ8mDEM.vrt.faaf24bf3b337161aa8a1f43a94cb216.tmp/vttwo/dem/dataset". {generatevrtwo.cpp:createOverview():832}
2020-09-02 16:11:15 I3 (dem) [2436(tile:0-0-0)]: Processed tile #1/934 0-0-0 (size: 4096x4096, extents: 18461373.345500,-4108234.873550:18526909.345500,-4042698.873550) [empty]; duration: 00:00:01.422352. {generatevrtwo.cpp:createOverview():961}
2020-09-02 16:11:15 I3 (dem) [2436(tile:0-1-0)]: Processed tile #2/934 0-1-0 (size: 4096x4096, extents: 18526909.345500,-4108234.873550:18592445.345500,-4042698.873550) [empty]; duration: 00:00:01.447406. {generatevrtwo.cpp:createOverview():961}
2020-09-02 16:11:15 I3 (dem) [2436(tile:0-6-0)]: Processed tile #3/934 0-6-0 (size: 4096x4096, extents: 18854589.345500,-4108234.873550:18920125.345500,-4042698.873550) [empty]; duration: 00:00:01.457473. {generatevrtwo.cpp:createOverview():961}
2020-09-02 16:11:15 I3 (dem) [2436(tile:0-5-0)]: Processed tile #4/934 0-5-0 (size: 4096x4096, extents: 18789053.345500,-4108234.873550:18854589.345500,-4042698.873550) [empty]; duration: 00:00:01.474233. {generatevrtwo.cpp:createOverview():961}
2020-09-02 16:11:15 I3 (dem) [2436(tile:0-2-0)]: Processed tile #5/934 0-2-0 (size: 4096x4096, extents: 18592445.345500,-4108234.873550:18657981.345500,-4042698.873550) [empty]; duration: 00:00:01.491669. {generatevrtwo.cpp:createOverview():961}
2020-09-02 16:11:15 I3 (dem) [2436(tile:0-4-0)]: Processed tile #6/934 0-4-0 (size: 4096x4096, extents: 18723517.345500,-4108234.873550:18789053.345500,-4042698.873550) [empty]; duration: 00:00:01.508050. {generatevrtwo.cpp:createOverview():961}
2020-09-02 16:11:15 I3 (dem) [2436(tile:0-7-0)]: Processed tile #7/934 0-7-0 (size: 4096x4096, extents: 18920125.345500,-4108234.873550:18985661.345500,-4042698.873550) [empty]; duration: 00:00:01.523613. {generatevrtwo.cpp:createOverview():961}
2020-09-02 16:11:15 I3 (dem) [2436(tile:0-3-0)]: Processed tile #8/934 0-3-0 (size: 4096x4096, extents: 18657981.345500,-4108234.873550:18723517.345500,-4042698.873550) [empty]; duration: 00:00:01.593541. {generatevrtwo.cpp:createOverview():961}
2020-09-02 16:11:15 E2 (dem) [2436(tile:0-11-0)]: gdal error 4: /var/vts/mapproxy/datasets/NZ8mDEM.vrt.faaf24bf3b337161aa8a1f43a94cb216.tmp/vttwo/dem/./../../original-dataset/92LBP-92L38.tif: No such file or directory {geodataset.cpp:GDALErrorHandler():76}
terminate called after throwing an instance of 'geo::WarpError'
  what():  Warp failed: </var/vts/mapproxy/datasets/NZ8mDEM.vrt.faaf24bf3b337161aa8a1f43a94cb216.tmp/vttwo/dem/./../../original-dataset/92LBP-92L38.tif: No such file or directory> (err=3). @{geodataset.cpp:warpInto():1437}
/usr/bin/mapproxy-setup-resource: line 8:  2436 Aborted                 (core dumped) /opt/vts/bin/mapproxy-setup-resource "$@" -f "${CONFIG}"

[jrjdavidson]

it might be that the paths in the vrt file are relative paths, not absolute?

[ladislavhorky]

Hi,

Nested VRTs are surely a no-go currently (tested) - e.g. you cannot run mapproxy-setup-resource on the resulting DEM produced by the same tool because that contains nested VRTs.

The tool tries to copy the input DEM, there should be an option to just link it but it is not currently implemented. It is meant for simpler cases to be honest.

If gdal_translateing the VRT into simgle tiff is an option, it should work. Otherwise I'd fall back to the old approach (generatevrtwo, mapproxy-calipers, mapproxy-tiling, tutorial here ) - it is slightly more complex but you have a complete control over it.

[jrjdavidson]

ok thanks again. should i leave this open in case someone else tries this?

[ladislavhorky]

Yes, let's leave that open until there are some updates from either side.

[jrjdavidson]

You can now use --linkdataset with mapproxy-setup-resource on vrt files