search

KitWallace / AIDVIEW-DB

A repository, browser and API for IATI activities
1 stars 0 forks source link

Purging stale activities #130

Open KitWallace opened 10 years ago

KitWallace commented 10 years ago

Activities no longer inclded in the current downloads need to be purged. This corrupts aggregations and leads to reporting of problems already fixed.

sets of actuivities which no longer exist are marked as deleted and a separate task (needs to be included into the adin interface) delets the set and all its activities.

However for activities the task is more complex . We need a) a full set of activitiy ids in the current set of downloads (currently about 176k) - this task could be combined with the refresh from CKAn since this alos does a complet scan of all downloads.

a process to check if each activity is included in this list and if not to mark it as deleted or delete it.

KitWallace commented 10 years ago

Progress so far is that the code is in place but indexing the full activity list corrupedt the index whihc need sto be re-constructed.
Then this task needs to be incorporated into the admin panel