zazuko / cube-creator

A tool to create RDF cubes from CSV files
GNU Affero General Public License v3.0
12 stars 2 forks source link

Shared Dimension Import - Very strange "unknown error" - Blocker #1301

Closed FabianCretton closed 1 year ago

FabianCretton commented 1 year ago

I have to create a Shared Dimension for "Zolltarif". I did prepare the .trig file, but when I create the Shared Dimension with an import of that file, I get: image

I join the file "SD_ZollTarif.trig" debug SD creation.zip

Having this error, but needing the cube these days, I spent hours to do some trials as the error says nothing (which is really a pity, could you not show the real error in the interface ?)

I validated the .trig file online -> https://www.w3.org/2015/03/ShExValidata/ -> seems ok Here is the result of my last test where I have "localized" the problem:

Those files are just the same as the original "SD_ZollTarif.trig", but where I removed the skos:broader, and then took only the first lines of the file -> thus limiting the number of terms.

I was suspecting to have a strange caracter or something weird, but here is the last test: I copied "SD_ZollTarif_2850lignes.trig" that works, and did duplicate the last term -> unknown error.

So, very strange, or is there a limitation about the number of terms ?

Thank you for any help, and sorry in advance if there is a stupid mistake in that generated file.

l00mi commented 1 year ago

We need to discuss how to go forward here, you know that this import was not build to be a "upload" function. It should only work with exported .trig files.

Instead of optimising the wrong utilisation of this function, we should soon look into creating an upload function which does error checking and everything. Or as alternative make it possible to extract a dimension from a cube to become a shared dimension.

FabianCretton commented 1 year ago

Thank you @l00mi
You mean that this function is only there to transfer Shared Dimensions from one system to another, right ?

If so, is there a better way to create Shared Dimensions ?

I can either learn how to make .trig files that conform to what the Cube Creator expects (I am sure it is not a far outreach), or we need another way to create the Shared Dimensions. Of course inserting all terms manually is often not an option.

But you (Zazuko) also handle a lot of Shared Dimensions that can not be handled manually (Zefix, the municipalities, etc.), so there are possibilities for sure.

Knowing what is happening with my file, even sending me a log or telling me where to look at a log, would certainly allow me to do the job.

Thanks

l00mi commented 1 year ago

As said the functionality to do this in Cube-Creator as a badge is on the roadmap. But not yet scheduled. I showed this way as a current work-around, but we can't improve the work-around. If there is no need for individual changes in the Cube-Creator, the best way is to simply create the Shared Dimension in a pipeline or simply in a file for that matter, which then gets uploaded to Lindas.

FabianCretton commented 1 year ago

Ok thanks, I will find a way

l00mi commented 1 year ago

@tpluscode can you try with the files above locally and see if it runs in a size limitation?

tpluscode commented 1 year ago

This loads fine locally. I wanted to see what happens in the app... and it got imported successfully too 🙈

https://int.cube-creator.lindas.admin.ch/app/shared-dimensions/dimension!!aussenhandel_zolltarif_fab_test6

FabianCretton commented 1 year ago

@tpluscode from what I see, you have uploaded "SD_ZollTarif_3000lignes.trig", which fails for me, yes. But could you also try with the "real" file: SD_ZollTarif.trig

If it works for you, but not for me, what could be the cause ? I have Cube Creator 1.12.7, in Firefox 105.0.1 I try right now with the exact same file (i just replaced test6 by test7 in the URLs), on INT: image

Do you have any log to see what happens with my upload and "error unkown" ?

tpluscode commented 1 year ago

I do not seem to find any errors being caught by monitoring, which would suggest a problem happens in the browser.

I'd expect errors in the browser console. Are there any? I use mostly Chrome. Can you try that?

FabianCretton commented 1 year ago

Here is my feedback:

Here is the error with the developper output: image

Does it make any sense ? Thanks for your help

FabianCretton commented 1 year ago

Last update: I did now create the Shared Dimension from the BAFU Helpers pipeline. I took the same RDF from the .trig file, and converted to a .ttl file The upload works

It seems then that there is effectively a problem when importing .trig files

l00mi commented 1 year ago

As we have now a more robust option for automatically curated dimensions with the pipelines.

We close this issue for now as the reported usage of the import function was not intended.

Please open again if we have problems with imports of "exports" generated by the cube-creator itself.