SEED-platform / seed

Standard Energy Efficiency Data (SEED) Platform™ is a web-based application that helps organizations easily manage data on the energy performance of large groups of buildings.
Other
106 stars 55 forks source link

Comprehensive performance improvements #4669

Closed axelstudios closed 1 month ago

axelstudios commented 1 month ago

Any background context you want to provide?

There was significant overhead to loading every django html request - e.g. just loading index.html on seedv2 takes 2.5 seconds before any data comes through or any other requests can be made, and this happens any time you load/refresh the page.

What's this PR do?

  1. Completely removes the aforementioned overhead via COMPRESS_CACHEABLE_PRECOMPILERS - the problem was the npx sass command was running and generating a new css file on every single load, resulting in hundreds of thousands of files in the CACHE directory and multiple gigabytes. Now it only runs on change (500x faster).

    Before After
    Overhead 2.5s 5ms
  2. Enables Django compression using brotli, bundles all individual static files into logical groups, and only loads the files necessary for the specific context (so instead of making 251 requests on load now it only makes 23). Compression is only used in production, but you can enable it in a development environment. This change should also permanently fix stale browser cache issues (8x faster):

    • Landing only loads landing.css
    • SEED only loads vendors.css, seed.css, vendors.js, and seed.js
    Before After
    Data Transferred 8.1 MB 1.9 MB
    Load Time 1.71s 224ms
  3. HUGELY improves performance when importing files with UBIDs into an org that already has records with UBIDs. For instance, if you import the 512-record covered-buildings-sample.csv to a new org, then upload it again with a single column changed and click Save Mappings (30x faster):

    Before After
    Queries 798,549 33,142
    Time 1h 57m 40s 3m 52s
  4. Switches AngularJS tests to use Puppeteer instead of the deprecated Protractor+WebDriver setup (just run npm test, but keep in mind that it's expecting SEED to be running on localhost:80). This PR also switches to automatic test paths instead of hardcoded paths

  5. Cleans up all landing files to remove unnecessary and (lots of) deprecated code

  6. Fixes serving the FAQ page on Windows

  7. Suppresses noisy Created ubid_model id logs

  8. Sorts org members on the Accounts page in ascending order by 'last_name', then 'first_name', or otherwise email (previously the order was arbitrary) image

  9. Renames inventory_settings and inventory_detail_settings to inventory_column_list_profiles and inventory_detail_column_list_profiles respectively, since the previous names were very ambiguous

  10. Makes the inventory list labels column pixel-perfect with borders consistent with the other fixed columns, and removes the broken sort button from the notes_count & meters_exist_indicator column headers

  11. Formats large numbers with thousand-separator commas throughout the app, which is especially useful for orgs with large amounts of data

  12. Makes clicking the SEED Platform text logo in the top left corner navigate instantly in all but one case

  13. Updates frontend dependencies

How should this be manually tested?

  1. Run pip install -r requirements/base.txt
  2. To test using compression locally, modify your local_untracked.py file with the following setting:
    COMPRESS_ENABLED = True
  3. Monkey test, ensure everything works as before

What are the relevant tickets?

Tangentially related to #4549