slorber
commented
10 years ago I think there is a concurrency issue when loading multiple graphs in the quad backend because when I run the loadings sequentially it works fine in all cases
Open slorber opened 10 years ago
slorber
commented
10 years ago I think there is a concurrency issue when loading multiple graphs in the quad backend because when I run the loadings sequentially it works fine in all cases
slorber
commented
10 years ago This is a problem during the loading I think.
I replaced a part of my previous code:
this.loadGraph = function loadGraph(graphUri,graphData,graphMimeType) {
var deferred = $q.defer();
RdfStore.load(graphMimeType,graphData,graphUri,function(success, results) {
if ( success ) {
deferred.resolve(results);
} else {
deferred.reject("Can't LOAD graph with GraphURI="+graphUri+" and graphMimeType="+graphMimeType+" -> "+results);
}
});
return deferred.promise;
}has become:
this.loadGraph = function loadGraph(graphUri,graphData,graphMimeType) {
var deferred = $q.defer();
setTimeout(function() {
RdfStore.load(graphMimeType,graphData,graphUri,function(success, results) {
if ( success ) {
deferred.resolve(results);
} else {
deferred.reject("Can't LOAD graph with GraphURI="+graphUri+" and graphMimeType="+graphMimeType+" -> "+results);
}
});
},getNextDelay());
return deferred.promise;
}
// TODO temporary: to delete
var delay = 0;
function getNextDelay() {
delay += 1000;
return delay;
}By spacing the graph loadings of 1 second, it works perfectly fine. If I put delay += 0; then the problem appears again.
It is not a problem during the SPARQL Construct query used by store.graph(...).
After loading all the graphs in store (spaced by 1sec which takes some time), I finally run a concurrent call of store.graph on all successfully loaded graphUris and it retrieves the correct graph. So it seems it's really a loading problem.
Here's a fiinal debug output with loadings not spaced and run concurrently:
Got graph http://melvincarvalho.com/#me with size=0 services.js:127
Got graph http://www.w3.org/People/Berners-Lee/card#i with size=146 services.js:127
Got graph http://mmt.me.uk/foaf.rdf#mischa with size=0 services.js:127
Got graph http://www.dfki.uni-kl.de/~grimnes/foaf.rdf#ggrimnes with size=0 services.js:127
Got graph http://danbri.org/foaf.rdf#danbri with size=0 services.js:127
Got graph http://www.discoveringidentity.com/foaf.rdf#me with size=704 services.js:127
Got graph https://my-profile.eu/people/tim/card#me with size=0 services.js:127
Got graph http://dbpedia.org/resource/Roy_Fielding with size=0 services.js:127
Got graph https://dr.jones.dk/me/#me with size=0 services.js:127
Got graph http://bigasterisk.com/foaf.rdf#drewp with size=0 services.js:127
Got graph http://crschmidt.net/foaf.rdf#crschmidt with size=0 services.js:127
Got graph http://presbrey.mit.edu/foaf#presbrey with size=0 services.js:127
Got graph http://www.anjeve.de/foaf.rdf#AnjaJentzsch with size=0 services.js:127
Got graph http://id.myopenlink.net/dataspace/person/KingsleyUyiIdehen#this with size=0 services.js:127
Got graph http://axel.deri.ie/~axepol/foaf.rdf#me with size=0 services.js:127
Got graph http://www.informatik.uni-leipzig.de/~auer/foaf.rdf#me with size=0 services.js:127
Got graph http://richard.cyganiak.de/foaf.rdf#cygri with size=0 services.js:127
Got graph http://www.ibiblio.org/hhalpin/foaf.rdf#me with size=0 services.js:127
Got graph http://b4mad.net/FOAF/goern.rdf#goern with size=0 services.js:127
Got graph http://sebastian.tramp.name with size=0 services.js:127
Got graph https://my-profile.eu/people/deiu/card#me with size=0 services.js:127
Got graph http://dbpedia.org/resource/James_Gosling with size=3175 And here the result when loadings are spaced by 1 second:
Got graph https://my-profile.eu/people/deiu/card#me with size=55 services.js:127
Got graph http://id.myopenlink.net/dataspace/person/KingsleyUyiIdehen#this with size=202 services.js:127
Got graph http://mmt.me.uk/foaf.rdf#mischa with size=301 services.js:127
Got graph http://presbrey.mit.edu/foaf#presbrey with size=35 services.js:127
Got graph http://melvincarvalho.com/#me with size=60 services.js:127
Got graph http://www.dfki.uni-kl.de/~grimnes/foaf.rdf#ggrimnes with size=101 services.js:127
Got graph http://danbri.org/foaf.rdf#danbri with size=278 services.js:127
Got graph http://www.discoveringidentity.com/foaf.rdf#me with size=26 services.js:127
Got graph http://dbpedia.org/resource/James_Gosling with size=237 services.js:127
Got graph https://my-profile.eu/people/tim/card#me with size=19 services.js:127
Got graph http://axel.deri.ie/~axepol/foaf.rdf#me with size=262 services.js:127
Got graph http://www.informatik.uni-leipzig.de/~auer/foaf.rdf#me with size=421 services.js:127
Got graph http://richard.cyganiak.de/foaf.rdf#cygri with size=296 services.js:127
Got graph http://dbpedia.org/resource/Roy_Fielding with size=70 services.js:127
Got graph http://www.ibiblio.org/hhalpin/foaf.rdf#me with size=63 services.js:127
Got graph http://b4mad.net/FOAF/goern.rdf#goern with size=231 services.js:127
Got graph https://dr.jones.dk/me/#me with size=554 services.js:127
Got graph http://bigasterisk.com/foaf.rdf#drewp with size=62 services.js:127
Got graph http://crschmidt.net/foaf.rdf#crschmidt with size=374 services.js:127
Got graph http://sebastian.tramp.name with size=58 services.js:127
Got graph http://www.anjeve.de/foaf.rdf#AnjaJentzsch with size=247 services.js:127
Got graph http://www.w3.org/People/Berners-Lee/card#i with size=86 As you can see it seems in the concurrent version that the James Gosling uri graph name seems to get all the triples while the others doesn't get any triple.
I use Chrome browser and do not use RDFStoreClient but rdfstore.create. I removed the RDFStoreClient code and it still work so I don't think I am using WebWorkers. I don't really know yet how javascript concurrency works or what could be the problem.
slorber
commented
10 years ago As I do not need a real store but just want to retrieve a RDF Interface api Graph, I tried to create a new rdfstore for each request.
this.fetchGraph = function fetchGraph(uri) {
var uriToFetch = maybeCorsProxifiedUri(uri);
return $http.get(uriToFetch,config)
.then(function(response) {
var mimeType = parseMimeType(response);
console.debug("It seems we got a response with mimeType "+mimeType + " from "+ uri+" - will try to parse it as a graph");
return parseGraph(uri, response.data, mimeType);
})
};
function parseGraph(graphUri, graphData, graphMimeType) {
var deferred = $q.defer();
var temporaryRdfStore = rdfstore.create();
temporaryRdfStore.load(graphMimeType, graphData, graphUri, function(success, results) {
if ( success ) {
temporaryRdfStore.graph(graphUri,function(success,graph){
if ( success ) {
deferred.resolve(graph);
} else {
deferred.reject("Can't GET graph with GraphURI="+graphUri);
}
});
} else {
deferred.reject("Can't LOAD graph with GraphURI="+graphUri+" and graphMimeType="+graphMimeType+" -> "+results);
}
});
return deferred.promise;
}The following does not work when. So there may be an isolation problem in the lib between different instances of RDFStore.
But a workaround that seems to work is to use the default graph:
this.fetchGraph = function fetchGraph(uri) {
var uriToFetch = maybeCorsProxifiedUri(uri);
return $http.get(uriToFetch,config)
.then(function(response) {
var mimeType = parseMimeType(response);
console.debug("It seems we got a response with mimeType "+mimeType + " from "+ uri+" - will try to parse it as a graph");
return parseGraph(uri, response.data, mimeType);
})
};
function parseGraph(graphUri, graphData, graphMimeType) {
var deferred = $q.defer();
var temporaryRdfStore = rdfstore.create();
temporaryRdfStore.load(graphMimeType, graphData, function(success, results) {
if ( success ) {
temporaryRdfStore.graph(function(success,graph){
if ( success ) {
deferred.resolve(graph);
} else {
deferred.reject("Can't GET graph with GraphURI="+graphUri);
}
});
} else {
deferred.reject("Can't LOAD graph with GraphURI="+graphUri+" and graphMimeType="+graphMimeType+" -> "+results);
}
});
return deferred.promise;
}Funny right? :)
I have the following code:
As part of an AngularJS prototype, I use Q to handle future results instead of callbacks.
Steps:
Most of the time, the graph returned by store.graph(graphUri,callback) is actually the appropriate graph, but it seems it randomly returns empty graphs as well as other graphs (graphs that do not correspond to the provided graphUri).
Here's a sample of logs to illustrate the problem:
As you can see the graph for this uri has 62 triples so it seems it's not the loading that cause troubles (or at least the callback returns the appropriate quad number, I don't know...)
But when fetching the graph it seems it find the triples of another graph as well, a graph I also loaded in the store, but with a diifferent uri/graphname, so it is not expected for me to return these triples as a result right?
I ask for the graph: http://bigasterisk.com/foaf.rdf And it seems to return me the triples of some other graphs I added before like http://b4mad.net/FOAF/goern.rdf#goern
My code works fine for the first graphs to load, and then doesn't work anymore.
Note that I call store.load concurrently to load 20 graphs so it may be a concurrency issue during the loading, perhaps multiple graphs are loaded to the same graphName (which also explain why sometimes I retrieve big graphs, and sometimes I retrieve empty graphs)
Generally, I find it quite hard to use RDFStore to parse a graph in a random media type and get the appropriate parsed graph as RDFInterface. I don't need yet SPARQL in my app so I only use this low level interface.
Wouldn't it be easier to retrieve the RDF interface graph after loading in the store, instead of calling the callback with a count of the triples loaded?