How to ease the burden of listing source and export files

[hoijui]

As it is now, people have to manually list all source and exported files. This is very cumbersome and I personally would never do it for a project.

How to solve this?

[hoijui]

1. allow to specify the URL to sourceList.txt and exportList.txt files, which simply contain one file per line.

These files could then be automatically generated in the CI job of the project with a script along the lines of:

find . -type f -iname "*.fcstd" > sourceList.txt
find . -type f -iname "*.pdf" > exportList.txt

[moedn]

as noted in the last update (relating to OPEN-NEXT/OKH-LOSH_copy#97), POSH files are found in specific folders, the collector automatically sorts them into source & export files (as included in the requirements here

[moedn]

Update:

• the approach linked above won't be followed since
  • repo structures are always different and no one will submit to our rules
  • we would strongly recommend having manually exported files in a git repository, which is also a bad idea
• the former approach (having individual POSH files including links to source & (external) export files) is way too much manual effort
• other assessed (semi-)automated approaches are dangerous/unstable/easy to break in practice especially when people are not fully aware of how to comply the policies in the metadata standard or how to use a git repository or not using a git repository at all (e.g. find source files by their name, allow custom rules (python code) in a JSON file,…)
• even for the MOSH assembly itself different approaches exist (e.g. assembly drawing/3D model possibly accompanied by assembly instructions in other files)

[moedn]

New approach:

short: packaging is excluded

• a repo contains
  • .okh.json (MOSH)
  • sBoM.csv
• a MOSH
  • is an item on Wikibase
  • has 'subitems', which are version-specific MOSHs, of course linked to the 'generic' MOSH
  • can have an "export package", which is a permanent URL linking to e.g. a GitHub package
• MOSHs from other repos (which are linked in the sBoM) are referenced via any permanent URL. The crawler/collector looks onto the URL in the sBoM of MOSH A and finds either:
  • repository and version information, so MOSH A can be linked to a specific other version of a MOSH B, which either
  • exists on Wikibase (identifiable via repo & version) or
  • must be created as new item (+ data is recursively loaded from the GitHub-API (license, licensor etc.))
  • repository information only, so MOSH A is linked to a generic MOSH B
  • no information apart from the URL itself → new item is created for MOSH B, just with it's name (from the sBoM) and it's URL and is linked to MOSH A
• a POSH
  • is no longer linked in Wikibase as an individual item (unless for a small extra set of data to show what would be possible)
  • is referenced in a sBoM by any feasible URL pointing to the place where development of that part is happening
• crawler identifies used data formats from the GitHub-API directly since source / export files are no longer linked in the JSON

Background:

• when version-specific information is lost in the Wikibase-connection, nothing bad happens since it's still included in the sBoM or rather in it's representation on Wikibase
  • so if we fail to get the version of MOSH B from a URL pointing to a specific commit, people would still get to this commit by clicking on it, but the connection between MOSH A & B would be only on a generic level, which is fine for v1.0 of a demonstrator; who knows if we actually need version-specific queries; for building however this information is essential

[moedn]

OSH-metadata.md shall be updated accordingly

[moedn]

I assume only a GUI (or any kind of semi-automated process) can solve this issue. For the moment, I'd recommend people to just copy & modify their POSH files &/ use our templates