Create the first version of memory crawler

[GoogleCodeExporter]

Please create the first version of memory crawler and push it into git.

Original issue reported on code.google.com by hbi...@gmail.com on 26 Nov 2014 at 6:38

[GoogleCodeExporter]

Data crawler vs scraper: Data crawlers get page links and follow linkage 
however data scrapers are used to download data.

The scraper will use SPARQL query language 
(http://www.w3.org/TR/rdf-sparql-query/) for linked data.

There is a vast pool of online linked data resources that supplies a SPARQL 
endpoint, complete list can be found here 
“http://www.w3.org/wiki/SparqlEndpoints”.

In order to visualise and/or parse data, the Sgvizler SPARQL Javascript library 
will be used (http://dev.data2000.no/sgvizler/)

Please check /semantic folder in source dir

Original comment by kazim.oz...@gmail.com on 1 Dec 2014 at 3:32

[GoogleCodeExporter]

Example queries with http://dbpedia.org/sparql

Musicians born in Berlin
========================
PREFIX dbo: <http://dbpedia.org/ontology/>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX : <http://dbpedia.org/resource/>
PREFIX dbpedia2: <http://dbpedia.org/property/>
PREFIX dbpedia: <http://dbpedia.org/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>

SELECT ?name ?birth ?description ?person WHERE {
     ?person dbo:birthPlace :Berlin .
     ?person <http://purl.org/dc/terms/subject> <http://dbpedia.org/resource/Category:German_musicians> .
     ?person dbo:birthDate ?birth .
     ?person foaf:name ?name .
     ?person rdfs:comment ?description .
     FILTER (LANG(?description) = 'en') .
}
ORDER BY ?name

People born in Berlin before 1900
=================================
PREFIX dbo: <http://dbpedia.org/ontology/>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX dc: <http://purl.org/dc/elements/1.1/>
PREFIX : <http://dbpedia.org/resource/>
PREFIX dbpedia2: <http://dbpedia.org/property/>
PREFIX dbpedia: <http://dbpedia.org/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>

PREFIX dbo: <http://dbpedia.org/ontology/>

SELECT ?name ?birth ?death ?person WHERE {
     ?person dbo:birthPlace :Berlin .
     ?person dbo:birthDate ?birth .
     ?person foaf:name ?name .
     ?person dbo:deathDate ?death .
     FILTER (?birth < "1900-01-01"^^xsd:date) .
}
ORDER BY ?name

Original comment by ka...@monolytic.com on 1 Dec 2014 at 5:19

[GoogleCodeExporter]

added query.txt under semantic/query for db feed

Original comment by kazim.oz...@gmail.com on 8 Dec 2014 at 2:43

• Changed state: Done