zazuko / cube-creator

A tool to create RDF cubes from CSV files
GNU Affero General Public License v3.0
11 stars 2 forks source link

sh:minInclusive of cube constraint set at higher value than lowest in dataset #1510

Open AdMSilvan opened 2 months ago

AdMSilvan commented 2 months ago

Describe the bug When validating some datasets against their cube constraint, many errors are reported due to sh:minInclusive being set to a value higher than some of the values contained in the dataset.

Affected functionalities (all that apply)

Relevant links

https://culture.ld.admin.ch/sfa/StateAccounts_Office/5/observation/OC1-O3-1951 https://culture.ld.admin.ch/sfa/StateAccounts_Function/7/shape/ **To Reproduce** Steps to reproduce the behavior: validate cubes with schema:amount 0, where minInclusive is not set to 0 Presumably CubeCreator still creates cubes with these constraint shapes if given a dataset with values containing 0. This does not seem to happen in every case, since in this dataset the shape was set correctly: https://culture.ld.admin.ch/sfa/StateAccounts_Domain/11/shape/ **Expected behavior** sh:minInclusive should be set to 0 if this is the lowest value in the data. **Additional context** The cause is unclear. At first I assumed it could be caused by 0 not being taken as a proper value and setting the min to the next lowest number, but this example shows that the value set for the min is not the lowest: Constraint-Shape with min value for amount: https://culture.ld.admin.ch/sfa/StateAccounts_Function/7/shape/#blank-2 Lowest values in dataset: https://s.zazuko.com/27ykjNn
AdMSilvan commented 2 months ago

@rdataflow discovered this issue so he may be able to add more context.

Rdataflow commented 2 months ago

@AdMSilvan thank you for reporting this one

important amendment:

how to solve is already documented in https://cube.link/#example-literals-dimensions and thus would look like

[
    sh:path ex:literalDimension;
    sh:nodeKind sh:Literal;
    sh:or([
        sh:datatype xsd:string ; sh:minInclusive 0 ; sh:maxInclusive 100
    ], [
        sh:datatype cube:Undefined
    ])
]

@tpluscode how to test and DOD

AdMSilvan commented 2 months ago

@Rdataflow I believe the issue with cube:Undefined was already caught earlier and is documented in this bug report https://github.com/zazuko/cube-creator/issues/1470 which is why I left it out of this one. The cause also seems to be different.

Thank you for your addition, I should have marked them as related to begin with.

tpluscode commented 2 months ago

@AdMSilvan did we agree that, like #1470, we'll keep this issue out of current sprint?

AdMSilvan commented 2 months ago

@tpluscode in the visualize call today we were able to speak to Martina and we thought it would be best to give estimates for the three issues I added in the gitlab page, so that we could potentially fit them into this or the upcoming sprint as well, so we can avoid organizational overhead.

tpluscode commented 2 months ago

Understood

That said, I believe this is exactly the same as #1470