search

crs4 / rocrate-validator

A Python package to validate RO-Crates
Apache License 2.0
7 stars 7 forks source link

Wrong profile used when using `--profile-identifier` #23

Closed elichad closed 2 weeks ago

elichad commented 1 month ago

I've made a profile called workflow-ro-crate-linkml for my LinkML experiments and put it in rocrate_validator/profiles/. I copied the profile.ttl from the workflow-ro-crate profile and changed the prof:hasToken at the end to workflow-ro-crate-linkml. The actual SHACL for the two profiles is different.

The package shows both workflow-ro-crate-1.0 and workflow-ro-crate-linkml-1.0 when I run rocrate_validator profiles list.

However, when I run something like 

rocrate-validator validate -v --profile-identifier workflow-ro-crate tests/data/crates/invalid/1_wroc_crate/no_mainentity/

it uses the workflow-ro-crate-linkml profile for validation.

If I run without --profile-identifier and select workflow-ro-crate when requested, it also misbehaves - it passes the crate that should fail and says [OK] RO-Crate is a valid workflow-ro-crate-linkml-1.0 !!! (wrong profile again). It also only runs 6/27 checks (27 is the right number of checks but I don't know why it only runs 6). When I select workflow-ro-crate-linkml it works as expected (running 25 checks as I haven't defined everything yet).

Finally, I tried changing the name workflow-ro-crate-linkml to wroc-linkml to see if making the names less similar would help, but it doesn't seem to.

I don't quite understand what's happening here - is the package matching the wrong name? Is there some strange caching happening?

kikkomep commented 1 month ago

The issue with the profile selection is that the validator doesn’t support multiple validation profiles per ROCrate profile. In other words, each version of a ROCrate profile can have at most one validation profile in the profile folder. So, to test your “linkml” validation profile for the Workflow ROCrate profile, you can try moving the original workflow-ro-crate validation profile out of the profiles folder.

The number of checks executed by the validate command depends on the validation mode. By default, a fail-fast mode is enabled, meaning the validator stops as soon as the first check fails. To run all checks, you can disable fail-fast mode by launching the validate command with the —no-fail-fast (or -nff) flag.

elichad commented 2 weeks ago

I figured out the problem - when copying the profile.ttl, I needed to make the subject ID unique compared to all other profiles (the first line after the @prefix lines). Before that I was just ensuring the prof:hasToken value was unique; I didn't realise the profiles could interfere with each other if they shared an overarching ID.

(This probably won't be an issue for anyone making brand new profiles, it only arose for me because I'm trying to make a second version of an existing profile)