search

ge-high-assurance / RACK

DARPA's Automated Rapid Certification of Software (ARCOS) project called Rapid Assurance Curation Kit (RACK)
BSD 3-Clause "New" or "Revised" License
20 stars 6 forks source link

Migration scripts for CSVs and nodegroups #294

Closed Ptival closed 3 years ago

Ptival commented 3 years ago

It is tedious to update the CSVs and nodegroups when the ontology changes. Supposedly, the task could be automated to some extent, given a description of the ontology changes.

This issue will keep track of an effort to try and support an automated migration script, that we can ship to other performers so as to ease their transition path when we make changes.

Marking this as cli since I may make this a rack command.

cuddihyge commented 3 years ago

I've had similar feature request from other SemTK users so I'd like to push this as far upstream as possible into SemTK. I'm willing to help, of course.
One easy operation is to detect that a node's property doesn't exist in the model, and substitute another with a very similar name (sometimes it is exactly the same name but a different base URI).

Another thought is that this operation should not be fully automatic. I'm fairly certain we'll want the user involved. This introduces a bit of a user interface development problem. One easier way around this might be to do it in semtk-python3 and use a simple text-based UI (present a list of options and the user hits a number). ...just brainstorming here.

GregIthaca commented 3 years ago

I wasn't sure where to post this so apologies for the repetition. If there's a better or correct place, let me know.

I have a few questions about the migration tool.

  1. The documentation says it will only migrate v4.1 to v5.0. Is this still true, or has it been upgraded to handle v5.0 to v5.1?
  2. I've followed (I believe) the installation instructions (including reinstalling the RACK CLI at the v5.1 version), and I'm getting an error about a missing module 'rack.commits'. Any idea if this is an error on my part, or whether there is some piece that may not have made it into the installation? (Full error below.)
$ rack_migrate --log-level=DEBUG --from-folder v5.0 --to-folder v5.1
Traceback (most recent call last):
  File "/home/gnelson/testing/ARCOS/RACK/bin/rack_migrate", line 4, in <module>
    __import__('pkg_resources').run_script('rack-migrate==0.0.0', 'rack_migrate')
  File "/home/gnelson/testing/ARCOS/RACK/lib/python3.7/site-packages/pkg_resources/__init__.py", line 651, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/home/gnelson/testing/ARCOS/RACK/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1455, in run_script
    exec(script_code, namespace, namespace)
  File "/home/gnelson/testing/ARCOS/RACK/lib/python3.7/site-packages/rack_migrate-0.0.0-py3.7.egg/EGG-INFO/scripts/rack_migrate", line 23, in <module>
ModuleNotFoundError: No module named 'rack.commits'
Ptival commented 3 years ago

Sorry @GregIthaca I tried debugging your problem 2 and never came back here to reply.

For question 1, we try and keep the tool up to date with each release, so once I update it, it should be able to migrate from any version (>= 4.0) to any newer version.

For question 2, I'm not sure what happened. I can try and help debugging if you pasted the output of pip install . from the migration directory (after having sourced the Python virtual environment in that directory). And maybe the output of which rack_migrate, to make sure there aren't stale copies of the executable on your PATH.

Ptival commented 3 years ago

Closing, further trouble with the migration scripts should have their own issues.