Some 500 HPs deleted during a BU append

eamena-project / eamena-arches-dev

development repository of the EAMENA Arches-powered database platform

https://eamena.org/home

GNU Affero General Public License v3.0

5 stars 3 forks source link

Some 500 HPs deleted during a BU append #42

Closed zoometh closed 9 months ago

zoometh commented 9 months ago

Running this BU append

python manage.py packages -o import_business_data -s ../data_temp/bu_append_hp_grid_20240116.csv -ow append -c "../data_temp/Heritage Place.mapping"

Most of the HP listed in bu_append_hp_grid_20240116.csv have been deleted from the DB at 16:00 UK time, the 2023-01-16.

UUID <-> EAMENA IDs are listed here

During the process, I had these warnings for these deleted HP :

Tue Jan 16 06:28:22 2024 WARNING: 2024-01-16 08:28:18.369285: WARNING: failed to save document in resource: 4dedcdfa-7938-46fc-9bb8-a75cb297ec68 on line 551

The tail of the resource_import.log file is here

Expected fix

restore the entire DB from yesterday (2024-01-15) backup

ads04r commented 9 months ago

Database restored, currently re-indexing. Also, I documented everything in case someone needs to do this when I'm on holiday or something...

https://github.com/eamena-project/eamena-arches-dev/blob/main/dbs/database.eamena/install/notes/Restoring%20DB%20from%20backup.md

zoometh commented 9 months ago

To give a bit of context about this "BU append" issue:

Some 255 HPs had misspelling in the name of the GS they belonged too. The overall idea was to calculate the correct GS by a spatial request (within). I removed these values in the GS field using the bad_grids listing. During this process, me and @yourvick gently slipped from this update to HP with no Grids (which is slightly different) see this SQL statement and now find 555 HPs. Then I tried to BU append Grid ID on these 555 HPs but it deleted them (see my Arches forum thread).

Related GitHub issues on duplicates and Grid ID typos

zoometh commented 9 months ago

Closed