search

geodesymiami / insarmaps

3 stars 0 forks source link

Very slow updating of colormap #12

Closed falkamelung closed 2 years ago

falkamelung commented 5 years ago

I know that we have changed the code so that the colormap updates itself if the [-2 2] interval is not appropriate. It works but it takes very long (10-20 seconds). The user gets the impression that the colormap changes on its own. Can this be accelerated? It should not show the [-2 2] colormap at all. Please try

http://insarmaps.miami.edu/start/-0.008/-78.0/9?startDataset=S1_IW3_021_0485_0489_20141007_20190219

falkamelung commented 5 years ago

This one is still in an issue. I just tried. It takes 10 seconds to update the color scale.

stackTom commented 3 years ago

Yes, I have noticed this is very slow for large datasets. I am looking at the code for this now, and the sql is not very good. I will try to speed it up.

Every time we select a dataset, the database goes through and finds the point with the highest velocity and the one with the lowest velocity. If my rewritten sql is still not fast enough, the solution will be to calculate these points when the dataset is being ingested and add this as another attribute to the dataset in the database. This will then require you to reingest all the datasets. Or for me to write another script which goes through the database and calculate the min and max velocities of all the points for each dataset.

stackTom commented 3 years ago

What I tried did not work to speed it up. We simply have to store the min and max in the database at the time of ingestion. This should not add any more time to ingestion but will make it so that obtaining the min and max so we can update to the appropriate interval is instantaneous.

I will update the python scripts, ideally when I have access to jetstream so i can have datasets (or feel free to send me a dataset to play with this weekend).

I will also have to write a script to update all current datasets with their min and max. This will take a while as it will take approximately 10 seconds per dataset. The other alternative is to simply start from scratch and reingest everything when I add this feature to the ingest script. If we choose to reingest, I won't need to write a script to update the current database.

falkamelung commented 2 years ago

not relevant anymore