[SHARE-1074] Drop unnecessary JSON fields on IngestJob

CenterForOpenScience / SHARE

SHARE is building a free, open, data set about research and scholarly activities across their life cycle.

http://share-research.readthedocs.io/en/latest/index.html

Apache License 2.0

100 stars 58 forks source link

[SHARE-1074] Drop unnecessary JSON fields on IngestJob #761

Closed aaxelb closed 6 years ago

aaxelb commented 6 years ago

Save disk space by deleting transformed_datum and regulated_datum from IngestJob.

transformed_datum gives a look at the intermediate state of a datum halfway through ingestion, but that could be generated on demand rather than stored forever.

regulated_datum is completely redundant, as it's saved as a NormalizedData.

Also add @linked_many admin helper, so IngestJobs in the admin link to their ingested normalized data:

coveralls commented 6 years ago

Coverage decreased (-0.04%) to 79.152% when pulling faa550c86a3bd8199d9dc6612529b6222b08073c on aaxelb:share-1074--disk-dog into 84f31b833c10a32015492a4f43480b97667c4770 on CenterForOpenScience:develop.