axfelix / frdr_harvest

MIT License
6 stars 0 forks source link

Update tags and subjects #121

Closed KellyStathis closed 4 years ago

KellyStathis commented 4 years ago

See FRDR subjects and tags document for details.

axfelix commented 4 years ago

Easy fix but will necessitate dropping some production data, let's try to include it with something else.

KellyStathis commented 4 years ago

Would we need to drop and reharvest Socrata repositories to switch the "category" mapping from series to subject--or can we just rely on the update to do this gradually?

For the renaming, I think we can combine with https://github.com/axfelix/frdr_harvest/issues/110 to group the gmeta changes together.

axfelix commented 4 years ago

We would have to drop them -- the entries in keyword tables would never get cleaned up otherwise and we'd have data duplicated in two fields unless I'm misunderstanding -- but that won't actually be that painful, just worth planning around.

KellyStathis commented 4 years ago

Does our update function remove previous metadata (i.e. if a subject or tag is removed, do we drop it)?

KellyStathis commented 4 years ago

For the Socrata case, we have their "category" in the series column of records now, so I assumed we would 1) update the records table entry and 2) add subject terms and corresponding records_x_subjects linkages.

axfelix commented 4 years ago

I would test this locally, I'm actually not sure how it would behave in this case.

KellyStathis commented 4 years ago

This is updated now on the bilingual branch. We'll be doing a full reharvest when we introduce those chagnes, so I think the question about how our metadata update works won't apply here.