search

swisstopo / swissgeol-assets-suite

1 stars 0 forks source link

Update Elasticsearch-Index #71

Closed ga-ebp closed 6 months ago

ga-ebp commented 7 months ago

The Elasticsearch index should be able to be updated.

If changes are made in the asset database outside the asset app, it should be possible to update the Elasticsearch index.

Possible solutions

Documentation: https://ebpgroup.atlassian.net/wiki/spaces/223443/pages/2044755971/Elastic-Search+Assets

daniel-va commented 6 months ago

@stijnvermeeren-swisstopo Das Feature ist jetzt zum Testen auf dem dev bereit. Zum Ankicken der Synchronisation kann unter dem Reiter "Verwaltung" der Knopf "Synchronisation" verwendet werden.

stijnvermeeren-swisstopo commented 6 months ago

@daniel-va I'm currently not able to test on the dev environment, as there seem to be some technical issues (after logging in, the page is stuck and the heatmap never loads). Marcel told me that he also experienced and reported the same problem already. Testing will have to wait until this is fixed, or until this feature is deployed to the int environment.

vej-ananas commented 6 months ago

From PR: Resolves https://github.com/swisstopo/swissgeol-assets-suite/issues/71.

Adds a way to sync the Elasticsearch asset index with the PostgreSQL database. To achieve this, the following has been done:

The all_study database view has been added to schema.prisma. The view existed already, but has not been mapped with prisma until. CRUD operations on the asset table have been unified into the AssetRepo class. Excluded from this are lookups of the AssetDetail type, as these don't fully conform to how assets are structured everywhere else. Elasticsearch operations have been unified into the the SearchAssetService. Both the AssetRepo and SearchAssetService have been tested. The tests are not at 100% coverage, but they should cover all central operations. The SearchAssetService also contains the syncWithDatabase method that refills the asset index using the data found in the PostgreSQL database. Syncs can be started via POST /api/assets/sync. The sync progress can be monitored via GET /api/assets/sync. A button has been added to the asset admin panel that starts a sync and displays its progress. Note that the size of this PR is due multiple factors:

The CRUD operations on assets and the Elasticsearch operations are quite large. A sizeable portion of the changes stems from just moving them around. The tests required the setup of a test environment that is filled with specific data. An dump of the entire integration database has been added to the repo. It can be used to quickly set up your local development database with some actually usable data. See the README on how to use that dump.

stijnvermeeren-swisstopo commented 6 months ago

Ich konnte das Syncen mittlerweile mit Erfolg auf INT testen.