sci-visus / OpenVisus

Open Source distribution of the ViSUS capabilities
Other
27 stars 18 forks source link

Older version raster images are now displayed as black #223

Closed bw4sz closed 1 year ago

bw4sz commented 1 year ago

I've been using OpenVisus for a number of years (thanks for a great tool). I moved our data from Azure to a local server and just rebooted the system. Everything looks good, but the files are now black. @spetruzza helped me get the original data together a couple years ago. Is it possible that there has been breaking changes to the openvisus server since then, do I need to regenerate the files? Anywhere I can go within the docker container to get logs?

image

Location: https://visualize.idtrees.org/

Docker command

sudo docker run --rm -d -p 8080:80 -v /pgsql/TreeDemo/openvisus/OpenVisus/:/converted -v /pgsql/NEON_crown_maps/App/config.js:/home/OpenVisus/dataportal/viewer/config.js -v /pgsql/NEON_crown_maps/App/visus.config:/home/OpenVisus/visus.config visus/dataportal

Files at that location look big and heavy as expected.

ABBY  BART  BLAN  BONA  CLBJ  CUPE  DEJU  DELA  DSNY  GRSM  GUAN  GUIL  HARV  HEAL  HOPB  JERC  JORN  KONZ  LAJA  LENO  MOAB  NIWO  NOGP  OAES  OSBS  PUUM  REDB  RMNP  SCBI  SERC  SJER  SOAP  SRER  TALL  TEAK  UKFS  WLOU  WOOD  WREF  YELL
ben@serenity:/pgsql/TreeDemo/openvisus/OpenVisus$ du -sh *
11G ABBY
11G BART
11G BLAN
5.1G    BONA
11G CLBJ
7.9G    CUPE
11G DEJU
11G DELA
11G DSNY
11G GRSM
11G GUAN
11G GUIL
11G HARV
11G HEAL
8.0G    HOPB
11G JERC
11G JORN
11G KONZ
11G LAJA
11G LENO
11G MOAB
11G NIWO
11G NOGP
11G OAES
11G OSBS
11G PUUM
11G REDB
11G RMNP
11G SCBI
11G SERC
11G SJER
11G SOAP
11G SRER
11G TALL
11G TEAK
11G UKFS
11G WLOU
11G WOOD
11G WREF
11G YELL
ben@serenity:/pgsql/NEON_crown_maps/App$ cat config.js
DEFAULT_SERVER='visualize.idtrees.org:8080/mod_visus?';
ben@serenity:/pgsql/NEON_crown_maps/App$ cat visus.config
<visus>
    <dataset name="ABBY - Oregon" url="file:///converted/ABBY/visus.idx" permissions="public"/>
    <dataset name="BART - New Hampshire" url="file:///converted/BART/visus.idx" permissions="public"/>
    <dataset name="BONA - Alaska" url="file:///converted/BONA/visus.idx" permissions="public"/>
    <dataset name="BLAN - Virginia" url="file:///converted/BLAN/visus.idx" permissions="public"/>
    <dataset name="CLBJ - Texas" url="file:///converted/CLBJ/visus.idx" permissions="public"/>
    <dataset name="CUPE - Puerto Rico" url="file:///converted/CUPE/visus.idx" permissions="public"/>
    <dataset name="DELA - Alabama" url="file:///converted/DELA/visus.idx" permissions="public"/>
    <dataset name="DEJU - Alaska" url="file:///converted/DEJU/visus.idx" permissions="public"/>
    <dataset name="DSNY - Florida" url="file:///converted/DSNY/visus.idx" permissions="public"/>
    <dataset name="GRSM - North Carolina" url="file:///converted/GRSM/visus.idx" permissions="public"/>
    <dataset name="GUAN - Puerto Rico" url="file:///converted/GUAN/visus.idx" permissions="public"/>
    <dataset name="GUIL - Puerto Rico" url="file:///converted/GUIL/visus.idx" permissions="public"/>
    <dataset name="HARV - Massachusetts" url="file:///converted/HARV/visus.idx" permissions="public"/>
    <dataset name="HEAL - Alaska" url="file:///converted/HEAL/visus.idx" permissions="public"/>
    <dataset name="HOPB - Massachusetts" url="file:///converted/HOPB/visus.idx" permissions="public"/>
    <dataset name="JERC -Georgia" url="file:///converted/JERC/visus.idx" permissions="public"/>
    <dataset name="JORN - New Mexico" url="file:///converted/JORN/visus.idx" permissions="public"/>
    <dataset name="KONZ - Kansas" url="file:///converted/KONZ/visus.idx" permissions="public"/>
    <dataset name="LAJA - Puerto Rico" url="file:///converted/LAJA/visus.idx" permissions="public"/>
    <dataset name="LENO - Alabama" url="file:///converted/LENO/visus.idx" permissions="public"/>
    <dataset name="MOAB - Utah" url="file:///converted/MOAB/visus.idx" permissions="public"/>
    <dataset name="NIWO - Colorado" url="file:///converted/NIWO/visus.idx" permissions="public"/>
    <dataset name="NOGP - North Dakota" url="file:///converted/NOGP/visus.idx" permissions="public"/>
    <dataset name="OSBS - Florida" url="file:///converted/OSBS/visus.idx" permissions="public"/>
    <dataset name="OAES - Oklahoma" url="file:///converted/OAES/visus.idx" permissions="public"/>
    <dataset name="PUUM - Hawaii" url="file:///converted/PUUM/visus.idx" permissions="public"/>
    <dataset name="REDB - Utah" url="file:///converted/REDB/visus.idx" permissions="public"/>
    <dataset name="RMNP - Colorado" url="file:///converted/RMNP/visus.idx" permissions="public"/>
    <dataset name="SOAP - California" url="file:///converted/SOAP/visus.idx" permissions="public"/>
    <dataset name="SCBI - Virginia" url="file:///converted/SCBI/visus.idx" permissions="public"/>
    <dataset name="SERC - Maryland" url="file:///converted/SERC/visus.idx" permissions="public"/>
    <dataset name="SJER - California" url="file:///converted/SJER/visus.idx" permissions="public"/>
    <dataset name="SRER - Arizona" url="file:///converted/SRER/visus.idx" permissions="public"/>
    <dataset name="TALL - Alabama" url="file:///converted/TALL/visus.idx" permissions="public"/>
    <dataset name="TEAK - California" url="file:///converted/TEAK/visus.idx" permissions="public"/>
    <dataset name="UKFS - Kentucky" url="file:///converted/UKFS/visus.idx" permissions="public"/>
    <dataset name="WLOU - Colorado" url="file:///converted/WLOU/visus.idx" permissions="public"/>
    <dataset name="WREF - Washington" url="file:///converted/WREF/visus.idx" permissions="public"/>
    <dataset name="WOOD - Washington" url="file:///converted/WOOD/visus.idx" permissions="public"/>
    <dataset name="YELL - Wyoming" url="file:///converted/YELL/visus.idx" permissions="public"/>
</visus>

Things i've tried

Header of the .midx file

ben@serenity:/pgsql/TreeDemo/openvisus/OpenVisus$ cd ABBY/
ben@serenity:/pgsql/TreeDemo/openvisus/OpenVisus/ABBY$ ls
ABBY  visus  visus.idx  visus.midx
ben@serenity:/pgsql/TreeDemo/openvisus/OpenVisus/ABBY$ head visus.midx
<dataset typename='IdxMultipleDataset'>
    <field name='voronoi'><code>output=voronoi()</code></field>
    <dataset url='./2019_ABBY_3_550000_5070000_image/visus.idx' name='2019_ABBY_3_550000_5070000_image' offset='0.0 80000.0'/>
    <dataset url='./2019_ABBY_3_550000_5071000_image/visus.idx' name='2019_ABBY_3_550000_5071000_image' offset='0.0 90000.0'/>
    <dataset url='./2019_ABBY_3_550000_5072000_image/visus.idx' name='2019_ABBY_3_550000_5072000_image' offset='0.0 100000.0'/>
    <dataset url='./2019_ABBY_3_551000_5062000_image/visus.idx' name='2019_ABBY_3_551000_5062000_image' offset='10000.0 0.0'/>
    <dataset url='./2019_ABBY_3_551000_5063000_image/visus.idx' name='2019_ABBY_3_551000_5063000_image' offset='10000.0 10000.0'/>
    <dataset url='./2019_ABBY_3_551000_5064000_image/visus.idx' name='2019_ABBY_3_551000_5064000_image' offset='10000.0 20000.0'/>
    <dataset url='./2019_ABBY_3_551000_5065000_image/visus.idx' name='2019_ABBY_3_551000_5065000_image' offset='10000.0 30000.0'/>
    <dataset url='./2019_ABBY_3_551000_5066000_image/visus.idx' name='2019_ABBY_3_551000_5066000_image' offset='10000.0 40000.0'/>

head of the .idx file

ben@serenity:/pgsql/TreeDemo/openvisus/OpenVisus/ABBY$ head visus.idx
(version)
6
(box)
0 29999 0 109999
(fields)
DATA uint8[3] default_compression(zip) default_layout(row_major) default_value(0) min(0 0 0) max(0 0 0)
(bits)
V11010101010101010101010101010101
(bitsperblock)
16
spetruzza commented 1 year ago

Hi Ben,

I see two issues from that webpage that are related to the connection to the visus server:

  1. the page is served over HTTPS but you are trying to point to a visus server via HTTP, this is generally something blocked by servers, you should try to use HTTPS for both (or http for both)
  2. You can test the connection to the server by changing the urs using the blue button on the top left and just type the url

Once (2) is working you should fix the url of the server that you are using for your viewer link. It does not seem like the server url is encoded correctly, you should have in the url something like this: viewer.html?server=https%3A%2F%2Fvisualize.idtrees.org%3A8080%2Fmod_visus%3F&dataset=NAME_OF_DATASET

Steve

On Dec 7, 2022, at 1:55 PM, Ben Weinstein @.***> wrote:

I've been using OpenVisus for a number of years (thanks for a great tool). I moved our data from Azure to a local server and just rebooted the system. Everything looks good, but the files are now black. @spetruzza https://github.com/spetruzza helped me get the original data together a couple years ago. Is it possible that there has been breaking changes to the openvisus server since then, do I need to regenerate the files? Anywhere I can go within the docker container to get logs?

https://user-images.githubusercontent.com/1208492/206292139-c5211863-ffc7-4e30-a660-a3cd5f15e6d5.png Location: https://visualize.idtrees.org/ https://visualize.idtrees.org/ @.:/pgsql/NEON_crown_maps/App$ cat config.js DEFAULT_SERVER='visualize.idtrees.org:8080/mod_visus?'; @.:/pgsql/NEON_crown_maps/App$ cat visus.config

Things i've tried

Changing the path to converted to include trailing slash/no trailing slash to see if the docker mount can successful locate the files. My current belief is that it knows where the files are, but not how to display them. Is this your impression as well?

Changing the location of the config.js, which was previously /home/OpenVisus/dataportal/ext/visus/config.js

Header of the .midx file

@.:/pgsql/TreeDemo/openvisus/OpenVisus$ cd ABBY/ @.:/pgsql/TreeDemo/openvisus/OpenVisus/ABBY$ ls ABBY visus visus.idx visus.midx @.***:/pgsql/TreeDemo/openvisus/OpenVisus/ABBY$ head visus.midx

output=voronoi() head of the .idx file ***@***.***:/pgsql/TreeDemo/openvisus/OpenVisus/ABBY$ head visus.idx (version) 6 (box) 0 29999 0 109999 (fields) DATA uint8[3] default_compression(zip) default_layout(row_major) default_value(0) min(0 0 0) max(0 0 0) (bits) V11010101010101010101010101010101 (bitsperblock) 16 — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you were mentioned.
bw4sz commented 1 year ago

Confirmed. Thanks for your time.

ben@serenity:/pgsql/NEON_crown_maps/App$ cat config.js
DEFAULT_SERVER='https:/visualize.idtrees.org:8080/mod_visus?';

solves the issue.