search

zazuko / cube-creator

A tool to create RDF cubes from CSV files
GNU Affero General Public License v3.0
11 stars 2 forks source link

Auto-fill Shared Dimension Mapping INCOMPLETE #1398

Closed FabianCretton closed 1 year ago

FabianCretton commented 1 year ago

Describe the bug

Running an auto-fill for municipalities, only part of the municipalities where mapped, not all. This could be a follow-up of: https://github.com/zazuko/cube-creator/issues/1372

Affected functionalities (all that apply)

Relevant links

See the project: https://int.cube-creator.lindas.admin.ch/app/cube-projects/cube-project!!fabdemosdmunicipality17032023-07pavj8mrj/metadata
AFoletti commented 1 year ago

Did you actually manage to link a shared dimension? We could not... See https://github.com/zazuko/cube-creator/issues/1393

FabianCretton commented 1 year ago

@tpluscode any timeline that we can expect a correction of this functionality ? Thanks for the update

tpluscode commented 1 year ago

While looking into this today I found a possible performance improvement but I also need to ask whether you unchecked the box labeled Only current terms?

I found that a lot of the municipalities in your cube are actually no longer current. That means that they have schema:validThrough property set to a past data and thus will be excluded from the default mapping. Example: https://s.zazuko.com/7622bK

tpluscode commented 1 year ago

That said, there are 2 values remaining which are not found in the municipalities dimension: 3112 and 5240

FabianCretton commented 1 year ago

Thank you @tpluscode, yes, I did not think about that dataset where the fusion of municipalities does create terms that are "not current". I tried it out unchecking 'Only current terms' and it worked.

As you said, two municipalities are not matched, but it seems that they exist, do you know why ? https://ld.admin.ch/municipality/3112 https://ld.admin.ch/municipality/5240

tpluscode commented 1 year ago

Apparently these two municipalities are present on PROD but not on INT

FabianCretton commented 1 year ago

Great, thank you, it also allows me to better understand how it works. I guess we can then close this issue, but you did merge something, did you improve the performance ?

tpluscode commented 1 year ago

I merged on TEST but should go to INT soon, nothing against

But please keep your expectations on a reasonable level. In this process getting the unmapped values is the bottleneck and just worse the larger the cube or dimension.