slow speed of URL-specified displacement map.

[falkamelung]

For the command beow it does three things: (1) the regular display, (2) the display of the displacement time series, and (3) the temporal zoom into the time period of interest. This takes literally 59 seconds on my laptop. Can't we skip the first two steps?

https://insarmaps.miami.edu/start/19.4289/-155.6005/10.2598?flyToDatasetCenter=false&startDataset=S1_IW12_087_0527_0531_20180902_XXXXXXXX&pointID=645557&minScale=-1&maxScale=1&startDate=20201003&endDate=20201208&colorscale=displacement

[stackTom]

Unfortunately, no. The way mapbox works and we have implemented on the fly coloring, we need the base dataset to display. Displacement is considered a type of on the fly coloring. So are custom start and end dates. It needed to be implemented this way due to how mapbox works with mbtiles.

As to why it takes so long, it might be just the sheer amount of points. Once one user uses a dataset, it goes into a cache on the database so it should be faster for other users until the dataset is purged from the cache. It could also be due to the fact that there are so many points. I bought one of the new apple silicon, m1 macbook airs, which is much faster than previous macs. This, along with the fact that the dataset is in the cache after you clicked on it, means it loads much quicker for me.

Basically, in order to properly do on the fly recoloring with only those points that are rendered, we need to show the datasets. But even if we didn't show the datasets, this isn't the slow part.

The slow part is the on the fly coloring. Postgresql (our database) can only handle so many points before it gets slow. The principle slowness has always been the database, and I am not sure this can be sped up besides just getting a faster server at this point, or making more sparse datasets. I remember both zishi and I did all we could to speed it up (at points even writing ugly code but which was fast). I could take a look again, but doubt I will be able to do much.

Here a video of it loading on my m1 macbook (after you and i have clicked on it so the data for on the fly is on cache in the database). It goes much quicker:

https://www.youtube.com/watch?v=IODTfPdfK9c

I used the below url. The one you posted doesn't work, as it is using the old pointID instead of pointLat and pointLon...

https://insarmaps.miami.edu/start/19.4289/-155.6005/10.2598?flyToDatasetCenter=false&startDataset=S1_IW12_087_0527_0531_20180902_XXXXXXXX&pointLat=19.52325&pointLon=-155.66817&startDate=20201009&endDate=20201208&colorscale=displacement

[stackTom]

Unfortunately, no. The way mapbox works and we have implemented on the fly coloring, we need the base dataset to display. Displacement is considered a type of on the fly coloring. So are custom start and end dates. It needed to be implemented this way due to how mapbox works with mbtiles.

As to why it takes so long, it might be just the sheer amount of points. Once one user uses a dataset, it goes into a cache on the database so it should be faster for other users until the dataset is purged from the cache. It could also be due to the fact that there are so many points. I bought one of the new apple silicon, m1 macbook airs, which is much faster than previous macs. This, along with the fact that the dataset is in the cache after you clicked on it, means it loads much quicker for me.

Basically, in order to properly do on the fly recoloring with only those points that are rendered, we need to show the datasets. But even if we didn't show the datasets, this isn't the slow part.

The slow part is the on the fly coloring. Postgresql (our database) can only handle so many points before it gets slow. The principle slowness has always been the database, and I am not sure this can be sped up besides just getting a faster server at this point, or making more sparse datasets. I remember both zishi and I did all we could to speed it up (at points even writing ugly code but which was fast). I could take a look again, but doubt I will be able to do much.

Here a video of it loading on my m1 macbook (after you and i have clicked on it so the data for on the fly is on cache in the database). It goes much quicker:

https://insarmaps.miami.edu/start/19.4289/-155.6005/10.2598?flyToDatasetCenter=false&startDataset=S1_IW12_087_0527_0531_20180902_XXXXXXXX&pointLat=19.52325&pointLon=-155.66817&startDate=20201009&endDate=20201208&colorscale=displacement

[stackTom]

I can try increasing postgres cache size. I don't remember how much ram our server has. I already increased this from default a long time ago. Let's see if increasing more makes a difference.

[falkamelung]

If there is nothing we can do on this one, we should consider to show more detailed messages when appropriate, e.g. " Recoloring YYY points which takes a while. Consider to reduce window size or to zoom out".

Another thought is to add a button to reduce the number of points (e.g. keeping the same zoom level but reducing the spatial resolution by a factor of 4). Could mess things up, though.

[falkamelung]

This dataset over Turkey takes 45 seconds to 2 minutes until the colors change (after having displayed it multiple times). In one post you said something on calculating the min and max at ingest. That is probably something we should do.

https://insarmaps.miami.edu/start/39.4793/35.0104/6.2346?flyToDatasetCenter=false&startDataset=S1_IW123_167_0452_0475_20170925_XXXXXXXX

A possible bug is that although it changes the colorscale to [-8 +8], it does not change the URL as you can see in the screenshot

[stackTom]

Closing this for now. On my computers, it takes around 30-40 seconds for the first recoloring, and 5-10 for subsequent re-colorings after updating to the latest mapbox (as seen here: https://github.com/geodesymiami/insarmaps/issues/53#issuecomment-1200326805). So now, it gets faster on subsequent re-colorings rather than slower (old version of mapbox had a significant memory leak).