okfn / opendataeditor

No-code application to explore and publish all kinds of data: datasets, tables, charts, maps, stories, and more. Forever free and open source project powered by open standards and generative AI.
http://opendataeditor.okfn.org
MIT License
150 stars 18 forks source link

Schema information created in Open Data Editor is only written to .opendataeditor/metadata.json and not to datapackage.json #306

Open daoane opened 5 months ago

daoane commented 5 months ago

I create a data package and add csv files. "Resources" information is created, including a schema for each file that is derived from the data. I can successfully add metadata on the package level, like contributors, license, etc. - it is written to datapackage.json.

Then I start to edit the metadata of individual files. I extend the automatically created information in the schema by filling fields for "Title" and "Description" and add constraints and foreign keys. However, this information is not written to the schema information on package level (datapackage.json), but saved to a more obscure place, .opendataeditor/metadata.json

Obviously, such information is of general importance for all applications and all users accessing the data package, not only for those using Open Data Editor. Why is the schema information in datapackage.json not updated accordingly? Is this by purpose, and if so, how can I propagate the changes from Open Data Editor level to the data package level? (other than copy/paste from one json document to the other)

For the user who inspects the source, it is really weird that these additional & changed information shows up every time the Open Data Editor is started while it remains totally unclear where the information is stored (this become clear only after looking for hidden files in the file system!). The user who saves the edits but does not inspect the source will distribute/publish a data package, believing that rich schema information is included while only basic information is actually readily available (as all edits are stored in the hidden files, for Open Data Editor only).

Is this a bug or do I miss an important point?

romicolman commented 3 months ago

Thanks @daoane for reporting this! @pdelboca this issue needs to be reviewed in connection with issue #285. Thanks!