edwardchalstrey1
commented
2 months ago Time before changes implemented (commit 3f173e7).
| shapesData | Mb | Python time (local) | Template time (local) | Python time (Turing VM) | Template time (Turing VM) | Template time (CSH VM) |
|---|---|---|---|---|---|---|
| Load of all shapes | 111.69 | 0.52 | 6.10 | 3.72 | 20.42 | 34.43, 122.74, 32.52 |
| Load of all shapes with vars | 187.89 | 5.87 | 15.29 | 8.70 | 31.72 | 70.05, 182.29, 120.71 |
As of commit 60f2ba:
| shapesData | Mb | Python time (local) | Template time (local) | Python time (Turing VM) | Template time (Turing VM) | Template time (CSH VM) |
|---|---|---|---|---|---|---|
| Load of all shapes | 111.68 | 0.54 | 6.01 | 3.67 | 20.12 | 65.85 |
| Load of all shapes with vars | 80.12 | 5.03 | 9.06 | 6.63 | 12.94 | 39.60 |
Where VMs are connected to on internet speed of: 13Mbps download (note: in reality this varies a lot)
This suggests the following:
- Locally, loading the vars view takes about 3.5X the time of loading without vars
- Once you get to the site, the python time is however irrelevant since the vars view is only 2X the time, suggesting the main thing here is just the time it takes to load the shapes all again
- So the problems are that the loading 111-188 mb is slow and that we are doing so twice
TODO:
- [x] Double check that Python speed is not an issue on VM by checking the Turing VM
- [x] Make it so we don't load the shapes all over again in the vars view
- Could
assign_variables_to_shapesand the categorical func too work with the "shapes" sans geometries? Then once this gets passed through to the template, merge the shapes dict that includes variables with the shapes dict that includes geometries - For that to work, you'd still need to load or somehow get from before, a version of "shapes" that does not include the geometries.
- Could
Consider if there is anything else we can do to reduce the Mb of the shapes dataI tried using theGZipMiddlewarebut this seemed to make it slower not faster
Role
Developer
Desired Feature
Opening the world map, the variables dropdown is available faster
Benefit
Avoid long wait time for variables to be fully loaded
Acceptance Criteria
All data variables used to colour the World map should be loaded in under one minute
Dependencies
No response
Technical Notes
TODO
Need to figure out if the time the variables functions take to run, or the size of the data in Mb is causing the problem. Almost certainly it will be the latter, otherwise it would also be slower on faster internet connections or when running locally.
Could one way to refactor this be that there is a new view function that gets called each time the variables dropdown is changed, and this gets a data dict linking each shape id to the variable info, which updatesThis would be an enormous refactor, see the alternative below that has resulted in a significant speed upshapesDatain the world map template?