Closed kityansiu closed 1 year ago
Currently we limit both model manifests and data manifests to a single model graph. To support this use case, I plan to make data manifests (but not model manifests) support multiple model graphs.
You mentioned that rack manifest import manifest.yaml
gives the desired behavior. Is it possible that you are using an old version of CLI? My understanding is that the CLI uses the same ingestion routine as the RACK UI, and therefore should give the same error. Further, I believe that rack manifest import
now only accepts ingestion packages (zip file), not yaml files.
Also - are you able to give me access to this ingestion package, for testing?
@glguy please comment if needed
In the CLI the model graphs field is defined to accept a list
So what Kit is asking for should work there.
The current version of the CLI shouldn't support loading manifest yaml files directly, but it can load single data.yamls
Thanks @glguy
@kityansiu I have completed the code changes for this feature, but I would like to test it with your ingestion package first before updating RACK. Please point me to it if/when you can. Thanks!
@kityansiu: this is fixed and pushed to RACK. Please test on your ingestion package when you get a chance.
@glguy: per discussion above, I noticed that the Wiki CLI “rack manifest import” instructions are out of date (still lists parameter as YAML file vs zip file) – you may want to update this
@weisenje: for future reference, keeping model manifest as single model as discussed here
I tested this and verified that it's working.
The current version of the CLI shouldn't support loading manifest yaml files directly, but it can load single data.yamls
You're right. I was using RACK v12.0 when I ran into this issue.
@glguy: per discussion above, I noticed that the Wiki CLI “rack manifest import” instructions are out of date (still lists parameter as YAML file vs zip file) – you may want to update this
@glguy, can you update the Wiki and the CLI help?
I've updated the wiki to match to documentation in README
usage: rack manifest import [-h] [--clear] [--optimize | --no-optimize]
[--optimize-url OPTIMIZE_URL]
config
positional arguments:
config Manifest YAML file
^^^^^^
How about this here?
Well then, I guess the wiki wasn't so much out of date but the tool was... fixing
We have a need to ingest data that spans multiple model-graphs. Right now the UI will give the following error:![image](https://github.com/ge-high-assurance/RACK/assets/12802455/d292e03d-e4bd-4565-9afa-e44e57a078bf)
Here's an example of the yaml files that are in this ingestion package. In ADEPT/InstanceData/import.yaml, I have the following:
And in ADEPT/manifest.yaml, I have the following:
As an example of desired behavior, I can run
rack manifest import manifest.yaml
which correctly ingests the data into the data-graph http://rack001/adeptdata while correctly referencing both model-graphs.