Comment quality report for new files

etalab / monitor-consolidation

Monitoring des jeux de données consolidés sur data.gouv.fr

MIT License

3 stars 2 forks source link

Comment quality report for new files #5

Closed AntoineAugusti closed 4 years ago

AntoineAugusti commented 4 years ago

Post a comment on the dataset page when a file is added or updated to report the quality against the schema.

AntoineAugusti commented 4 years ago

Is it okay to merge as is and run things on production? It's not easy to run this on demo because:

dataset IDs are not the same
files are uploaded or updated on production, not on demo
file_urls are taken from production

I've executed the script locally to check the detection of new files and the API call.

I'd be happy to have your opinion on the message before shipping

abulte commented 4 years ago

@AntoineAugusti ok for prod.

About the message, this is what I wrote for the DECP validation: https://demo.data.gouv.fr/fr/datasets/fichier-csv/#discussion-5c9b8847634f4178a4eca78a

I think the part about "why this check has been triggered" is important.

AntoineAugusti commented 4 years ago

@geoffreyaldebert Your points are all valid. This implementation is a quick and naive implementation to solve the current problem: producers want to know about their errors (they asked us) and we need to show that we're proactive at checking data quality.

That being said, this implementation has flaws:

it can be seen as spammy. If you publish files often, you'll get multiple messages. Not even in the same thread. That could be improved later if it's an issue
if the consolidation is done automatically, there is not much you can do to fix your errors
some producers could be annoyed to have messages on their dataset saying that their data is not clean
messages stick around on the dataset, even if the underlying problem is sometimes temporary

I think we're still figuring things out and we will learn from this feature in production.

AntoineAugusti commented 4 years ago

@abulte One last check before merging this, can you confirm that secrets have the right value for production on GH's settings? I can see that you've set them but I can't check their values.

geoffreyaldebert commented 4 years ago

I agree with your explanations. We'll see on the fly when it's in production.

abulte commented 4 years ago

@AntoineAugusti it should be ok now, but DOMAIN should be useless? I guess you can remove this env from your code.