flickering and slowdown/memory issues

[falkamelung]

Hi @stackTom

I think I have figured tout the disk filling up issue. There were old docker images, sudo docker system prune -a removes them.

I installed it on two instances (tileserverDevelopment branch) to ckeckout the impact of memory:

149/165.168.186: 3GB RAM 1 CPU, 
149.165.153.50: 15GB 4 CPUs 

and ingested a big dataset (15GB):

149.165.168.186/start/39.5/118.4/11.0?flyToDatasetCenter=true&startDataset=S1_IW123_069_0126_0130_20141022_20231116_N39495_N39710_E118090_E118410_filtDel4DS
149.165.153.50/start/39.5/118.4/11.0?flyToDatasetCenter=true&startDataset=S1_IW123_069_0126_0130_20141022_20231116_N39495_N39710_E118090_E118410_filtDel4DS

and a smaller one: 800 MB

149.165.168.186/start/-8.5/123.6/11.0?flyToDatasetCenter=true&startDataset=S1_IW3_090_0622_0624_20170116_20240807_S08552_S08486_E123555_E123591_filtDel4DS
149.165.153.50/start/-8.5/123.6/11.0?flyToDatasetCenter=true&startDataset=S1_IW3_090_0622_0624_20170116_20240807_S08552_S08486_E123555_E123591_filtDel4DS

If I don’t do anything fancy it works fine. But for now I observed these problems>

1. If you change pixel size and then go to the layer button it starts flickering.

2. If you select a shorter time period and then move around and then select a point I see

For me this is easily reproducible. Go to :

149.165.168.186/start/-8.5/123.6/11.0?flyToDatasetCenter=true&startDataset=S1_IW3_090_0622_0624_20170116_20240807_S08552_S08486_E123555_E123591_filtDel4DS

Sometimes it get scrambles when I select the start date and other times you have to move around a bit.

3. When I select a limited time period and just move around without selecting a pixel and/or changing the color scale the "Recoloring in progress box" shows up all time. Maybe something is not efficient? The same happens when a reference point is selected manually. If we could speed this up that would be fantastic. (If nothing can be done about this I wonder add a feature to change the reference point. The workflow is, we first ingest, look at the data and then select the reference point accordingly. Once we know which is a good reference point we could run an additional json_2_mbtiles command to change the reference point).

4. I am trying to understand teh memory requirements. When I open the big data set on the small memory instance it sometimes goes very fast (even after emptying cache), and sometimes it take a few minutes. Why could that be?

   149.165.168.186/start/39.5/118.4/11.0?flyToDatasetCenter=true&startDataset=S1_IW123_069_0126_0130_20141022_20231116_N39495_N39710_E118090_E118410_filtDel4DS

   Please explain a bit what operations are done on the remote instance and which in your browser to understand memory. Given the memory available on insarmaps, should we allow datasets with limited size only?

5. Both, changing pixel size and selecting a reference point id currently a mess, but will probably work after fixing (1).

[stackTom]

Regarding 3 - we only recolor those points currently in view and hide the rest. For performance reasons. Otherwise the recoloring is impossibly slow. Whenever the map is changed, we must recolor the new points that are now in view.

Regarding the scrambling, it appears postgresql has changed how some commands work. We were on an old version, 9.2 in our insarmaps server. Now in docker, we are on 16. I will have to fix the sql commands I use. Or just roll back the Postgres on the docker to version 9.2.

[falkamelung]

Regarding 3, yes, we talked about that before. I just want to be sure we are on the same page regarding this. Even if I just more around without zooming it happens. See video. In this case there is no need to recolor anything, is there? https://github.com/user-attachments/assets/64380f24-9368-4db5-aa3a-70bf065c613a

It occasionally also happens just after displaying a dataset without selecting a time period (I think). In that case there is no reason to recolor? This is more difficult to reproduce, but I can try if needed.

And, finally, if nothing can be done abut this, would a big memory machine mitigate this? Jetstream also has some GPUs which we could try.

And regarding the scrambling, if you find the time to fix this, my preference would be stay with version 16

[disilvestro]

Hi everybody, I would like to address some more issues.

• [ ] Sometimes, once the Select Reference Point option is selected, the option remains active and there is no way to turn it off.
• [ ] When you show 2 timeseries together by choosing the option Second graph, once you deselect it, the second timeseries still runs in the background. What happens is that when you select a new point, the timeseries in the background(not visible) changes but not the visible one.
• [ ] Some swaths are not visible until you zoom in

[falkamelung]

Hi @stackTom and I tried to update the insarmaps.miami.edu. The repo is in docker/insarmaps.new, correct? I was not sure about the locations of the postgres_dir and mbtiles_dir. Which ones are we using? See below the /data listing.

Once we are finalized I probably will move or restart as /data/postgres_dir /data/mbtiles_dir for consistency with the other instances.

[insaradmin@insarweb insarmaps]$ ll /data
total 48
drwxr-xr-x  3 insaradmin root        4096 Apr 23 14:22 converting
drwx------  2 insaradmin root       16384 Jan 24  2017 lost+found
-rw-r--r--  1 insaradmin postgres   16374 Dec 21  2017 OUT.txt
drwx------ 19        101        104  4096 Sep 21 19:57 postgresDBData
drwxrwxr-x  2 root               33  4096 Sep 21 18:07 tileserver
drwxrwxr-x  3 insaradmin insaradmin  4096 May  9  2020 TILESERVERBACKUP