visualize-admin / visualization-tool

The tool for visualizing Swiss Open Government Data. Project ownership: Federal Office for the Environment FOEN
https://visualize.admin.ch
BSD 3-Clause "New" or "Revised" License
29 stars 3 forks source link

perf: Upgrade to latest cube iri once #1473

Closed bprusinowski closed 2 months ago

bprusinowski commented 3 months ago

Closes #1472

This goal of this PR is to make the server-side not aware of a "latest cube" concept and to always just pass cubeIri, without the latest property. Cube upgrade will happen once on the client side when initializing the chart (edit mode) or fetching the chart config (embed and published modes).

The rationale for it is that there is no need to constantly ask for latest cube version in every GQL query. This will improve performance and reduce the number of queries we send to LINDAS.

This change should also make it easier to control when an upgrade should happen by having centralized place to define that behavior, instead of having to pass the latest property in every GQL query (upgradeConfiguratorState that could be called conditionally if at some we introduce an ability to pin cube version).

Implementation logic

I introduced a new, publishIri property to chart config cube, which stores cube IRI at publish time. I decided to leave the current iri property to make it much easier to handle the change throughout the application without additional custom JS logic to keep the mapping somewhere and access upgraded iri in every query – because of this, majority of the code stays the same. We just keep the currently upgraded cube iri in the iri property.

This should make it easy to control the upgrade logic – if the cube is not meant to be upgraded, we assign publishIri to iri; if it's meant to be upgraded, we keep upgraded iri in iri, without affecting rest of the application.

Questions

vercel[bot] commented 3 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
visualization-tool ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 26, 2024 2:21pm
Rdataflow commented 3 months ago

@bprusinowski unfortunately the upgrade of published charts does not to happen anymore. This needs to be fixed.

repro case:

  1. open https://visualization-tool-git-perf-upgrade-cube-version-once-ixt1.vercel.app/v/vyJ1Kt0I9Wbb?dataSource=Test
  2. see version 5 is loaded while version 6 is published (nb: hence dateModified is from the past)
Rdataflow commented 3 months ago

... while in preview mode upgrade happens as expected repro case:

  1. open https://visualization-tool-git-perf-upgrade-cube-version-once-ixt1.vercel.app/browse?dataset=https%3A%2F%2Fenvironment.ld.admin.ch%2Ffoen%2Fnfi%2Fnfi_C-98%2Fcube%2F2023-1&dataSource=Prod
  2. see upgrade to version 2023.3 happens on the fly 👍

nit: there is a delay due to QGL queries being fired with old version and later relaunched with new version

bprusinowski commented 3 months ago

Thanks for catching a problem with published charts @Rdataflow, should be fixed now 😄