osPlanning / omx

Open Matrix (OMX)
https://github.com/osPlanning/omx/wiki
Apache License 2.0
49 stars 18 forks source link

add test suite #34

Closed bstabler closed 4 years ago

bstabler commented 5 years ago

we need test cases to round trip validate OMX APIs. We can use this for the OMX APIs on GitHub, as well as for commercial third-party applications. We may also setup continuous integration.

bstabler commented 4 years ago

Comments from the Zephyr software badging committee:

...what is needed is a linter or file validation tool -- one tool that I can download and run against a file, which will tell me if the file is or is not a valid OMX file according to the specification, and (importantly) if not, why not. This would let any new developer who wants to support OMX as output be able to check if they are using the specification correctly. Checking that writing out OMX is correct is really the core of having an interoperability standard; if you are messing up reading the file you are hurting only your own application, but if you are messing up writing you are pushing your problems out to other tools.

I do not deem the missing “contribution guidelines” as an important flaw for this file format. Indeed, that there is no visible mechanism to change the specification at all may in some respects be considered a plus rather than a minus. Because it has already become so widely adopted, it is likely to be beneficial for many users that the format not be allowed to change very much. In the long term, there may be value in allowing some changes (at least additions to) in the format, but making that hard for this standard is not in my mind a bad thing, and not a reason to delay badging.

The file validation tool should ideally also have a “compare” option, to put one OMX file against another and check if they are content-identical, especially since the standard allows for optional cruft that could make content-identical not bytes-identical. And there should be some reasonably portable docker or similar environment that could be used to run the validator.

If a file validation tool is created, ideally something that can run on any of Windows-Linux-macOS, the OMX standard is worthy of a Zephyr software badge.

bstabler commented 4 years ago

I committed the validator, which you can run at https://colab.research.google.com/drive/1PUoNc1o-n1042F-xmTH9MEMxramvduHr. It isn't a test suite....but it essentially meets the needs above so I'm closing this issue for now.