Closed vsoch closed 6 months ago
This is ready to go.
compspec extract
) and then generation of the JGF graph also looks good (comspec create nodes
)The current issues with the second are that node feature discovery is not giving us the topology of sockets -> cores, nor is it able to give us basic counts of virtual vs physical cores. We will need hwloc for that (I hope/think). I can't add the bindings yet because there is a bug, but I wrote the Makefile (added here) that will make that possible.
This should be enough to prototype with rainbow, so I'm going to merge. Will pick up more after dinner.
Problem: we need a extract metadata for nodes and then parse into a cluster graph Solution: create a compspec create nodes subcommand.
This is for the registration step of a cluster to rainbow. Information about the design / idea planning is here.
In this PR I am adding a ClusterGraph, which still needs work to improve the output to easily map into a JGF (right now it has elements that can support any type that need further parsing). I am also generalizing the idea of plugins more, so we will have extractors and converters (that run create) but I need to finalize the design for the latter, right now the create commands are very separate. I am opening the PR sooner than later in case my computer explodes. A few problems I have run into is that NFD does not have cpu counts, let along physical vs. logical. This information is in /proc/cpuinfo for x86 but not arm. We also do not have a way to get socket -> core mapping. So likely we do need to add the hwloc extractor, and provide an automated build for doing that since it requires hwloc on the system. I will put some thought into this.
Dinosaur TODO
Update for the above - hwloc won't be possible soon, there is a bug in the library. I've opened an issue, and I'll remove the bullets to #19
Update: JGF looks good! It was... a pointer bug!