search

asc-csa / Scisat-App

🛰️ Cette application permet aux utilisateurs de filtrer les données du satellite SCISAT et de visualiser les données extraites | 🛰️ This application allows users to filter data from the SCISAT satellite and visualize the extracted features.
https://donnees-data.asc-csa.gc.ca/scisat/
Other
6 stars 1 forks source link

Slow first load #146

Closed low-decarie closed 1 year ago

low-decarie commented 2 years ago

Same as https://github.com/asc-csa/AlouetteApp/issues/136

emilinecsa commented 1 year ago

SCISAT refreshes within 10-12 seconds now on the QA server. This is not perfect, but a great improvement. The source code contains TODO statements that identify the slowest parts.

  1. numpy.nan is slow
  2. Mapping to binned dataset

To improve the performance, I added a static variable - data_reader.page_df - in data_reader(). It makes sure the function will not be called 3 times with the same parameters. In addition, I speed up this part:

Before: for i in range (nbDays): date.append(dt.datetime(years[i],months[i],days[i])) Now: date = np.array([dt.datetime(years[i],months[i],days[i]) for i in range (nbDays)])

Bassically speaking, two methods are still slow:

  1. data_reader() (because of numpy.nan)
  2. generate_geo_map() (because of the bin mapping)
emilinecsa commented 1 year ago

The latest version of SCISAT was deployed on the production server on May 24th, 2023. This new version is much faster than the previous one. The Graphical User Interface (GUI) hasn’t changed. https://donnees-data.asc-csa.gc.ca:8888/scisat/