Test FUSEKI instead of Virtuoso

tomas-knap commented 10 years ago

Instead of using Virtuoso, try fuseki. Where were the problems?

tomesj commented 10 years ago

No problems found using FUSEKI.

tomas-knap commented 10 years ago

Ok, so you just set up FUSEKI, adjust the configuration - ports etc. and everything was working?

Btw. as the data are merged from one data unit to another, what mechanism is used for merging, I mean which SPARQL query?

tomas-knap commented 10 years ago

For example, "DEFINE sql:log-enable 2 \n"

- "ADD <%s> TO <%s>", sourceGraphName, targetGraphName"

Will this work with Fuseki (if you try to use this query when manually querying fuseki)? Use arbitrary graphs

tomas-knap commented 10 years ago

in other words, this log_define(2) used in Virtuoso, is it working with Fuseki?

tomesj commented 10 years ago

Log_define(2) is Virtuoso specific, only merge using ADD X TO Y is OK (used SPARQL 1.1 Update syntax).

tomas-knap commented 10 years ago

Well, so how is Fuseki able to work with Define sql:log_enable? It is discarded when the query is executed?

tomas-knap commented 10 years ago

What will happen if you submit query: DEFINE sql:log-enable 2 "+ "ADD <%s> TO <%s> to fuseki?

tomesj commented 10 years ago

I get Error 400: Lexical error at line 1, column 7. Encountered: " " (32), after : "DEFINE"

But part ADD <something> TO <something2> passed.

tomas-knap commented 10 years ago

But if there is in the code: "DEFINE sql:log-enable 2 \n"

"ADD <%s> TO <%s>", sourceGraphName, targetGraphName"

What will happen if fuseki is used? It will throw an error during merging?

tomesj commented 10 years ago

Is returned this code

Error 400: Lexical error at line 1, column 7. Encountered: " " (32), after : "DEFINE"

Fuseki - version 1.0.0 (Build date: 2013-09-12T10:49:49+0100)

tomas-knap commented 10 years ago

So I would suggest to:

list all virtuoso specific expressions, such as this define:sql:log-enable. Is there any other? Please document every such place in the code.
add to config properties flag where user may set whether virtuoso rdf extensions should be used, e.g. virtuoso.rdf.virtuosoExtensions = yes/no.
if yes is defined, it behaves like now. If no is specifies, no virtuoso specific extension should be used in the core engine (merging data etc.). As a result, it should work without these extensions (you should comment in the code what the virtuoso extension brings, whether it is significatnt improvement or not, what the user will lose if the extension is not used).

Then try connection to fuseki instead of to virtuoso in virtuoso.rdf.* params in config properties. Try it on these 7 milions of triples, try [INTLIB] Decisions from nsoud.cz on odcs.xrg.cz:8080/odcleanstore.

tomesj commented 10 years ago

Add virtuosoExtension to code and config.properties. Prepare to close.

tomas-knap commented 10 years ago

Jirka, what is still missing:

list all virtuoso specific expressions, such as this define:sql:log-enable. Is there any other? Please document every such place in the code. Create a page for that in confluence.
did you test how does it work if useExtension is set to NO?

tomas-knap commented 10 years ago

Did you successfully use fuseki instead of virtuoso as RDF store? That means you try to run couple of pipelines, checking the log file that no suspicious messages occur.

tomesj commented 10 years ago

To questions:

No, there is only define: sql: log-enable. I add special comment to this.
yes, I tried you id useExtension is disable (set to false) and it work normally.
I tested on fuseki only list of methods - extraction, loading, merging. But no whole pipeline, because fuseki need special calling as s-update, s-query,...and it is fuseki specific syntax - our code is not written for this specific cause.

tomas-knap commented 10 years ago

Regarding the use of "useExtension", it should log a warning when user is using virtuoso but not using extension (useExtension = false), because actually, when not using extension, the performance or even functionality of virtuoso fro bigger graphs may be influenced

Regarding the testing of fuseki, it is not enough to use the individual methods, it either can be used instead of virtuoso for our tool and not. What is the problem with the calls? Fuseki does not support openRDF interface and does not provide an adapter as virtuoso, right?

tomesj commented 10 years ago

To VIRTUOSO: So when is used Virtuoso, but useExtension is false set, I add warning - you right that the performance for big graphs can cause problems.

To FUSEKI: Yes, you right

tomas-knap commented 10 years ago

openRDF would not work for fuseki

tomesj commented 10 years ago

No actuall yet - reopen again if case of need

tomas-knap commented 10 years ago

Fuseki cannot be used via openRDF. This should be check in the future

mff-uk / odcs

Test FUSEKI instead of Virtuoso #640