search

OpenEnergyPlatform / organisation

Repository for discussing and documenting decisions about the Open Energy Family
https://openenergyplatform.github.io/organisation/
GNU Affero General Public License v3.0
10 stars 4 forks source link

The metadata organisation is confusing #26

Closed MGlauer closed 2 years ago

MGlauer commented 4 years ago

As @Bachibouzouk pointed out: Information regarding OEP-metadata is scattered across this organisation. We need to clean this up and here is my proposal:

OMI would be a good place for metadata in general (I think we had a recent discussion regarding that!?). BUT: Specification, documentation and implementation are three different things and should be separated accordingly. In order to clean this up, I would propose the following: Impementation -> OMI Documentation -> ReadTheDocs (i.e. omi/docs) Specification -> Its own repository (OpenEnergyPlatform/metadata)

Bachibouzouk commented 4 years ago

@MGlauer - thanks for creating this issue :)

What would we do with the OpenEnergyPlatform/examples ? We link to ReadTheDocs ? What would you think that OpenEnergyPlatform/metadata should contain? The json templates and jsonschema? Should we then make a package oep-metadata from which one could import the json templates from oep-metadata import TEMPLATE_v1_4 ? Why not defining everything in OMI and omi/docs? I am in favor of a single go-to place for the metadata in general.

What do you think @OpenEnergyPlatform/admin ?

MGlauer commented 4 years ago

@MGlauer - thanks for creating this issue :)

What would we do with the OpenEnergyPlatform/examples ? We link to ReadTheDocs ?

Regarding the examples for out tools: Yes. For the simple reason that it enables us to implement proper doctests in unison with travis etc. Therefore we can be sure that the examples in our documentation are always up-to-date. Unfortunately, I am still looking for a good sphinx-extension to test REST-APIs :/

What would you think that OpenEnergyPlatform/metadata should contain? The json templates and jsonschema?

Yes.

Should we then make a package oep-metadata from which one could import the json templates from oep-metadata import TEMPLATE_v1_4 ? Why not defining everything in OMI and omi/docs? I am in favor of a single go-to place for the metadata in general.

I would suggest to import the jsonschema-spec as a submodule in OMI. The specification is not a python-package but language independent. For the same reason I would like to separate OMI from the metadata spec. The metadata is language independent and is its own thing that can be used independently from OMI (e.g. in other languages)

What do you think @OpenEnergyPlatform/admin ?

Ludee commented 4 years ago

I think concentrating the metadata in one place is good idea. I'm not sure if creating two different repos for Specification and Impementation helps. I'd be in favour of a joint solution.

Bachibouzouk commented 4 years ago

I would suggest to import the jsonschema-spec as a submodule in OMI.

@MGlauer - I gave @4lm the task to start working on this, just that you know :)

4lm commented 4 years ago

@MGlauer : The specification is not a python-package but language independent. For the same reason I would like to separate OMI from the metadata spec.

@Ludee: I think concentrating the metadata in one place is good idea. I'm not sure if creating two different repos for Specification and Impementation helps. I'd be in favour of a joint solution.

@Bachibouzouk: Why not defining everything in OMI and omi/docs? I am in favor of a single go-to place for the metadata in general.

I agree with @MGlauer, spec is independent from a specific language or implementation and should be IMO treated as it's own single-source-of-truth. In JSON schema the $id field is reserved for the JSON schema resource location. From XML projects I know, that many resource endpoints are description of schema and schema combined, and the XML schema are then cloned into the specific project code. That's IMO clean and doesn't conflict with the single-source-of-truth paradigm, because they (like JSON schema) self-reference an URL to their origin. I also would like to have a more future proofed place (keyword archiving) for holding the schema and it's description then a simple GitHub repo. I would suggest something like zenodo.org which we could use as our JSON schema and description archive/store.

What do you think?

Edit: And on zenodo.org each JSON schema + description would get automatically it's own DOI (+ citation rules), which is good for citation work in scientific papers.

EditEdit: And yes, I would also want to add an repo like @MGlauer proposed

Specification -> Its own repository (OpenEnergyPlatform/metadata)

but with slight the difference, that I would use it to organise the metadata creation and publishing work (discussion, python code for creation and publishing of schemas and links to zenodo).

4lm commented 4 years ago

I just talked to @Bachibouzouk and decided to consolidate the metadata work from examples and omi in a repo called metadata. The repo will contain schemas, examples and templates. I think that aligns with the original idea of @MGlauer. We then do a pypi package called oep-metadata, which we then can import into omi, examples etc.

Edit: We then can still discuss, if this is a good way to go and or needs some change of direction. But right, now there are IMO to many places where the metadata stuff is spread around, so I feel the urge to consolidate that ...

4lm commented 4 years ago

Zenodo (https://github.com/OpenEnergyPlatform/organisation/issues/26#issuecomment-545397922), on a second thought, this makes stuff maybe too complicated. So I'm totally happy if we have the $id in the JSON schema pointing to the GitHub blob of the JSON schema in the metadata repo.

Edit: Like this: https://github.com/OpenEnergyPlatform/metadata/blob/develop/metadata/v140/schema.json#L3

What do you think?

Ludee commented 4 years ago

The restructure of the metadata is a good idea! We just discussed that awe need a OEP method for versioning the metadata. ATM this is done here on GitHub.