search

IO500 / io500

IO500 Storage Benchmark source code
MIT License
98 stars 31 forks source link

Info editing tool for the schema. #21

Closed JulianKunkel closed 3 years ago

JulianKunkel commented 3 years ago

This branch includes a python tool allowing to edit a file inserting new system defaults from the command line. Thus a file created from the template or by the schema editor here: https://www.vi4io.org/hpsl/addpage-tmp can be exported to json, then modified with the tool, then read back.

A lightweight verification using the actual schema will be implemented.

JulianKunkel commented 3 years ago

Hi, I've now done enough for this project and will not update it further until it is discussed and there are specific requests. It works sufficiently well to be discussed during our meeting.

One can add information for example by calling: $ ./info-creator/io500_add_osinfo.py site.json

Example output on my system: Site.Supercomputer.Nodes.name = kunkel-ThinkPad-X1 Site.Supercomputer.Nodes.kernel version = 5.4.0-58-generic
Site.Supercomputer.Nodes.Processor.architecture = x86_64
Site.Supercomputer.Nodes.Processor.model = Intel Core i7-7500U
Site.Supercomputer.Nodes.Processor.frequency = 2.7 GHz
Site.Supercomputer.Nodes.Processor.cores = 4
Site.Supercomputer.Nodes.distribution = Ubuntu
Site.Supercomputer.Nodes.distribution version = 20.04
Site.nationality = GBR

The updated site.json can then be uploaded to the webpage and changed/edited there.

adilger commented 3 years ago

So the next logical step would be to add an info_editor/io500_add_lustre_info.py script that would extract available information from the client, and/or from the server if it was run there? My Python-fu is very weak, but I might be able to cargo-cult enough code from one of the other scripts to extract information from Lustre. Failing that, I could at least generate a list of "how to get this input -> schema field" mappings so that it could be added.

adilger commented 3 years ago

What is a bit strange is that it has marked me as "adilger approved these changes" but I only gave approvals to a subset of the patches. I don't use Github much, so I don't know if that is working as expected.

gflofst commented 3 years ago

Thanks @adilger for the improvements. It is better.

JulianKunkel commented 3 years ago

I think it is best to create a new repository and outsource these tools to be more agile and allow externals to collaborate. Given that this is a contribution by VI4IO relevant for IO500 and the CDCL, I think it is well placed there.