4. Performance improvement

[JohnShiuMK]

All groups are required to meet the following four performance specifications:

• [x] Use a binary format for loading data faster
  • [ ] Note that binary format is not required in reflection as per Joel
• [x] Have no redundant filtering/mutating of the data in the app (Completed)
  • Create a processed data file that is loaded in the app instead (stored in data/processed).
• [x] Have no excessively slow computation that could be easily sped up, e.g. no row-wise use of apply unless well justified and optimized.
• [x] #129

Outside of these baseline requirements, your app will also be assessed for how well it performs. This means that there might be additional things that you need to consider, such as using VegaFusion, aggregating large scatter plots, using indications that the data is loading, etc. We are aware that the servers have some slowdown, and you won't be penalized for that of course, but if your app is noticeably slower than many other apps, you are expected to use what we discussed to speed it up (and remember that you are also allowed to sample the data as long as what you are visualizing is meaningful).

Tasks:

• [x] #111
• [ ] (optional) Implement "Update" button to limit unnecessary callbacks
• [x] #128
  • for commodities in particular
• [x] #127
  • Function logic to skip computation and return existing chart

[srfrew]

Suggestions for performance improvements

• [x] Remove pd.DataFrame.apply() in data.py and plotting.py
• [ ] (optional) Implement "Update" button to limit unnecessary callbacks
• [x] Store previous widget / chart state so that only affected charts are modified
  • for commodities in particular
• [x] Reduce computation: Update numeric stats only when end-date is changed to reduce computation
  • Function logic to skip computation and return existing chart

[srfrew]

Write that pre-fetching the data is a viable solution, but not able to implement for this milestone

[JohnShiuMK]

It's done after the implementation of geo view