In memory RDF Virtuoso for unit testing sparql queries

[vinodreddyb]

Hi,

Is there any lightweight RDF store to write integration testing with Junit,

DataSource virtuosoDataSource = ../
VirtGraph virtGraph = VirtGraph(virtuosoDataSource);
Query sparql = QueryFactory.create("SELECT * FROM <http://test1> WHERE { ?s ?p ?o }");
QueryExecution vqe = VirtuosoQueryExecutionFactory.create(sparql, virtGraph))

I want to load test datasource when I am writing unit testing . Please provide solution

[TallTed]

I'm sorry; your need is not clear.

With a small-to-moderately-sized dataset, Virtuoso -- whether Open Source Edition or Enterprise Edition -- is quite small; it runs perfectly happily on most laptop or desktop workstations.

Our ballpark guideline is to give Virtuoso 10 bytes of RAM per triple/quad (or 10 KB per 1000 triples; 10 MB per 1,000,000 triples; etc.) in the RDF store. More RAM is always better, but not generally required for reasonable performance.

Is that sufficient information to answer your question? If not, please elaborate on your need....

[vinodreddyb]

I want to write a unit test code for my DAO layer where I implemented the Virtuoso related code to query RDF store

public void queryForData(java.sql.DataSource virtuosoDataSource) {
VirtGraph virtGraph = VirtGraph(virtuosoDataSource);
Query sparql = QueryFactory.create("SELECT * FROM <http://test1> WHERE { ?s ?p ?o }");
VirtuosoQueryExecution queryExecution= VirtuosoQueryExecutionFactory.create(sparql, virtGraph))
ResultSet results = queryExecution.execSelect();
while (results.hasNext()) {
        QuerySolution rs = results.nextSolution();
        RDFNode s = rs.get("s");
        RDFNode p= rs.get("p");
        RDFNode o = rs.get("o");
                System.out.println(" { " + s + " " +
                        p + " " + o + " . }");
    }
}

Now I want to write Unit test method for that. For unit testing any data layer, normally I inject in memory datasource (H2 or Derby) and load my test data when tests starts and execute my SQL queries to validate. Datasource will be closed after tests execution completes.

I need help on this. I came across apache jena in-memory https://jena.apache.org/documentation/jdbc/drivers.html#in-memory. Using this I am loading test datasource like below .

DriverManagerDataSource ds = new DriverManagerDataSource();
ds.setDriverClassName("org.apache.jena.jdbc.mem.MemDriver");
ds.setUrl("jdbc:jena:mem:dataset=sample.ttl");  //TTL file contains my data 
 queryForData(ds);

I am getting sparql lexical error like below.

16:55:22.239 [main] DEBUG org.springframework.jdbc.datasource.DriverManagerDataSource - Loaded JDBC driver: org.apache.jena.jdbc.mem.MemDriver
16:55:22.250 [main] DEBUG org.springframework.jdbc.datasource.DriverManagerDataSource - Creating new JDBC DriverManager Connection to [jdbc:jena:mem:dataset=c:/Vinod/data/sample.ttl]
16:55:22.261 [main] DEBUG org.apache.jena.shared.LockMRSW - Lock : main
16:55:22.263 [main] DEBUG org.apache.jena.shared.LockMRSW - Lock : main
16:55:22.546 [main] DEBUG org.apache.jena.riot.system.stream.StreamManager - Not mapped: c:/Vinod/data/sample.ttl
16:55:22.546 [main] DEBUG org.apache.jena.riot.system.stream.StreamManager - open(c:/Vinod/data/sample.ttl)
16:55:22.546 [main] DEBUG org.apache.jena.riot.system.stream.StreamManager - Not mapped: c:/Vinod/data/sample.ttl
16:55:22.554 [main] DEBUG org.apache.jena.riot.system.stream.StreamManager - Found: c:/Vinod/data/sample.ttl (LocatorFile)
16:55:24.192 [main] DEBUG org.springframework.jdbc.datasource.DriverManagerDataSource - Creating new JDBC DriverManager Connection to [jdbc:jena:mem:dataset=c:/Vinod/data/sample.ttl]
16:55:24.192 [main] DEBUG org.apache.jena.shared.LockMRSW - Lock : main
16:55:24.192 [main] DEBUG org.apache.jena.shared.LockMRSW - Lock : main
16:55:24.192 [main] DEBUG org.apache.jena.riot.system.stream.StreamManager - Not mapped: c:/Vinod/data/sample.ttl
16:55:24.192 [main] DEBUG org.apache.jena.riot.system.stream.StreamManager - open(c:/Vinod/data/sample.ttl)
16:55:24.192 [main] DEBUG org.apache.jena.riot.system.stream.StreamManager - Not mapped: c:/Vinod/data/sample.ttl
16:55:24.193 [main] DEBUG org.apache.jena.riot.system.stream.StreamManager - Found: c:/Vinod/data/sample.ttl (LocatorFile)
16:55:25.554 [main] INFO org.apache.jena.jdbc.statements.JenaStatement - Received input command text:
 sparql

 define input:default-graph-uri <virt:DEFAULT> 
SELECT  *
WHERE
  { ?s  ?p  ?o }

16:55:25.555 [main] INFO org.apache.jena.jdbc.statements.JenaStatement - Command text after pre-processing:
 sparql

 define input:default-graph-uri <virt:DEFAULT> 
SELECT  *
WHERE
  { ?s  ?p  ?o }

16:55:25.557 [main] ERROR org.apache.jena.jdbc.statements.JenaStatement - Invalid SPARQL query
org.apache.jena.query.QueryParseException: Lexical error at line 1, column 7.  Encountered: "\n" (10), after : "sparql"
    at org.apache.jena.sparql.lang.ParserSPARQL11.perform(ParserSPARQL11.java:109)
    at org.apache.jena.sparql.lang.ParserSPARQL11.parse$(ParserSPARQL11.java:52)
    at org.apache.jena.sparql.lang.SPARQLParser.parse(SPARQLParser.java:34)
    at org.apache.jena.query.QueryFactory.parse(QueryFactory.java:147)
    at org.apache.jena.query.QueryFactory.create(QueryFactory.java:79)
    at org.apache.jena.query.QueryFactory.create(QueryFactory.java:52)
    at org.apache.jena.query.QueryFactory.create(QueryFactory.java:40)
    at org.apache.jena.jdbc.statements.JenaStatement.executeQuery(JenaStatement.java:562)
    at virtuoso.jena.driver.VirtuosoQueryExecution.execSelect(VirtuosoQueryExecution.java:102)

[smalinin]

The class VirtuosoQueryExecution works only with Virtuoso JDBC driver and connections to Virtuoso DB. It doesn't work with another JDBC drivers like org.apache.jena.jdbc.mem.MemDriver

[JervenBolleman]

You can start a virtuoso-t with a process builder and a new virtuoso.ini file. something like this. This allows you to start a virtuoso server just for testing which you can demolish after your unit test has passed.

In your setup/'@Before` method

ProcessBuilder pb = new ProcessBuilder();

//You will want to write a new configFile with a new directory location for the data and listing port
File cf = new File(configFile);pb = new ProcessBuilder(virtuosoInstallLocation + "virtuoso-t", "-f", "+wait", "+configfile", cft.getAbsolutePath());
//p is a java process field.
p = pb.start();
//You need to make sure the server has started before running tests.
waitUntilServerIsListening();

Then for your tearDown/@After method.

//p being the virtuoso process. 
//you can also test this using 
p.destroyForcibly();
//remove all test data files etc. using the normal methods.

One of the methods you can use to see if your virtuoso server is ready for testing.

/**
   * This method checks to see if the virtuoso server is started and waiting
   * for a connection.
   */
   private void waitUntilServerIsListening(){
        while (true)
        {
            if (testIfServerIsListenting())
                return;
            try
            {
                Thread.sleep(TimeUnit.SECONDS.toMillis(5));
            } catch (InterruptedException e1)
            {
                Thread.interrupted();
            }
        }
    }

    private boolean testIfServerIsListenting()
    {
        try (Socket sock = new Socket("localhost", Integer.parseInt(port)))
        {
            return true;
        } catch (Exception e)
        {
            return false;
        }
    }