search

ruby-rdf / sparql

Ruby SPARQL library
http://rubygems.org/gems/sparql
The Unlicense
88 stars 14 forks source link

Random errors when parsing #31

Closed anthony-o closed 6 years ago

anthony-o commented 6 years ago

I'm trying to create a Logstash filter that parses SPARQL queries.

So I'm using sparql v2.2.2 to parse those queries.

I'm testing 3 queries using rspec tests that correctly pass.

Nevertheless, when running Logstash with this filter, I have some random error when parsing (parsed_query = SPARQL::Grammar.parse(sparql_query, {:resolve_iris => TRUE}).

For example, for this one:

SELECT ?subject ?p0 ?p1 ?p2 WHERE {{?subject <http://purl.org/dc/terms/title> ?p0 . VALUES (?subject) { (<http://identifiers.org/ncbigene/6093>) }}UNION{?subject <http://www.w3.org/2000/01/rdf-schema#label> ?p1 . VALUES (?subject) { (<http://identifiers.org/ncbigene/6093>) }}UNION{?subject <http://www.w3.org/2004/02/skos/core#prefLabel> ?p2 . VALUES (?subject) { (<http://identifiers.org/ncbigene/6093>) }}}

I can have this error

'#<NoMethodError: undefined method `first' for nil:NilClass>' at ["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:589:in `block in Parser'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:600:in `block in onFinish'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `block in eval_with_binding'", "org/jruby/RubyBasicObject.java:1691:in `instance_eval'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `eval_with_binding'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:599:in `onFinish'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:402:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:1525:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar.rb:179:in `parse'", "/usr/share/logstash/vendor/local_gems/65c66494/logstash-filter-sparql-0.1.0/lib/logstash/filters/sparql.rb:44:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164:in `block in multi_filter'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:48:in `multi_filter'", "(eval):46:in `block in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:455:in `filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:434:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:393:in `block in start_workers'"]

or this one

'#<NoMethodError: undefined method `+' for nil:NilClass>' at ["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:455:in `add_prod_data'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:125:in `method_missing'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:725:in `block in Parser'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:720:in `block in Parser'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:600:in `block in onFinish'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `block in eval_with_binding'", "org/jruby/RubyBasicObject.java:1691:in `instance_eval'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `eval_with_binding'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:599:in `onFinish'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:402:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:1525:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar.rb:179:in `parse'", "/usr/share/logstash/vendor/local_gems/65c66494/logstash-filter-sparql-0.1.0/lib/logstash/filters/sparql.rb:44:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164:in `block in multi_filter'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:48:in `multi_filter'", "(eval):46:in `block in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:455:in `filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:434:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:393:in `block in start_workers'"]

For this query:

PREFIX : <http://rdf.example.com/>
SELECT ?type WHERE { <http://www.metaphacts.com/resource/admin/Templates> a ?type }

I can have the error

'#<TypeError: invalid SPARQL::Algebra::Expression operand: nil>' at ["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/algebra/expression.rb:118:in `block in new'", "org/jruby/RubyArray.java:2486:in `map'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/algebra/expression.rb:109:in `new'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/algebra/expression.rb:76:in `for'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/algebra.rb:406:in `Expression'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:944:in `block in Parser'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:600:in `block in onFinish'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `block in eval_with_binding'", "org/jruby/RubyBasicObject.java:1691:in `instance_eval'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `eval_with_binding'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:599:in `onFinish'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:402:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:1525:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar.rb:179:in `parse'", "/usr/share/logstash/vendor/local_gems/65c66494/logstash-filter-sparql-0.1.0/lib/logstash/filters/sparql.rb:44:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164:in `block in multi_filter'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:48:in `multi_filter'", "(eval):46:in `block in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:455:in `filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:434:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:393:in `block in start_workers'"]

or

'#<NoMethodError: undefined method `first' for nil:NilClass>' at ["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:589:in `block in Parser'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:600:in `block in onFinish'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `block in eval_with_binding'", "org/jruby/RubyBasicObject.java:1691:in `instance_eval'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `eval_with_binding'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:599:in `onFinish'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:402:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:1525:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar.rb:179:in `parse'", "/usr/share/logstash/vendor/local_gems/65c66494/logstash-filter-sparql-0.1.0/lib/logstash/filters/sparql.rb:44:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164:in `block in multi_filter'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:48:in `multi_filter'", "(eval):46:in `block in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:455:in `filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:434:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:393:in `block in start_workers'"]

At last, for this one

PREFIX : <http://rdf.example.com/>
SELECT ?isoform ?isoformLabel ?length ?weight ?isoelectricPoint ?canonicalIsoform WHERE {
  <http://identifiers.org/ncbigene/207> <http://rdf.example.com/biomart/encodesProtein> ?uniprot.
  ?entry <http://www.w3.org/2004/02/skos/core#exactMatch> ?uniprot.
  GRAPH ?graph {
    ?entry <http://nextprot.org/rdf#isoform> ?isoform.
    ?isoform <http://nextprot.org/rdf#sequence> ?sequence.
    ?isoform <http://nextprot.org/rdf#canonicalIsoform> ?canonicalIsoform.
    BIND(STRAFTER(STR(?isoform), "http://nextprot.org/rdf/isoform/") AS ?isoformLabel)
    ?sequence <http://nextprot.org/rdf#length> ?length.
    ?sequence <http://nextprot.org/rdf#molecularWeight> ?weight.
    ?sequence <http://nextprot.org/rdf#isoelectricPoint> ?isoelectricPoint.
  }
  FILTER(?graph = <http://rdf.example.com/nextprot>)
}

I can have this error

'#<NoMethodError: undefined method `<<' for nil:NilClass>' at ["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:446:in `add_prod_datum'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:125:in `method_missing'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:146:in `block in Parser'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:636:in `block in onTerminal'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `block in eval_with_binding'", "org/jruby/RubyBasicObject.java:1691:in `instance_eval'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `eval_with_binding'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:635:in `onTerminal'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:309:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:1525:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar.rb:179:in `parse'", "/usr/share/logstash/vendor/local_gems/65c66494/logstash-filter-sparql-0.1.0/lib/logstash/filters/sparql.rb:44:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164:in `block in multi_filter'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:48:in `multi_filter'", "(eval):46:in `block in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:455:in `filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:434:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:393:in `block in start_workers'"]

or this one

'#<NoMethodError: undefined method `first' for nil:NilClass>' at ["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:622:in `block in Parser'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:600:in `block in onFinish'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `block in eval_with_binding'", "org/jruby/RubyBasicObject.java:1691:in `instance_eval'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:118:in `eval_with_binding'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:599:in `onFinish'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/ebnf-1.1.2/lib/ebnf/ll1/parser.rb:402:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar/parser11.rb:1525:in `parse'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/sparql-2.2.2/lib/sparql/grammar.rb:179:in `parse'", "/usr/share/logstash/vendor/local_gems/65c66494/logstash-filter-sparql-0.1.0/lib/logstash/filters/sparql.rb:44:in `filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:145:in `do_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:164:in `block in multi_filter'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:161:in `multi_filter'", "/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:48:in `multi_filter'", "(eval):46:in `block in filter_func'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:455:in `filter_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:434:in `worker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:393:in `block in start_workers'"]

Do you know if SPARQL::Grammar.parse(sparql_query, {:resolve_iris => TRUE}) depends on some various environment settings on runtime or is those error can be explained differently?

I asked to the Elasticsearch community the same issue and specify how to reproduce this (Docker and docker-compose are needed):

git clone https://github.com/anthony-o/docker-elk_logstash-filter-sparql.git
cd docker-elk_logstash-filter-sparql
docker-compose up -d

Then connect to http://localhost:5602, create the index pattern * and go to the "Discover" tab. If none of the events contain the field exception, restart the ELK stack with ocker-compose down --volumes && docker-compose build && docker-compose up -d

gkellogg commented 6 years ago

I have some other SPARQL issues to investigate, and I'll through this into the list. Note that JRuby has had issues with testing across many gems in the RDF.rb ecosystem, which could be a contributing factor. The JRuby issues could use a champion who is vested in that platform. (Same with Erubis).

gkellogg commented 6 years ago

Recent changes seem to have fixed this on the develop branch. It will be released as 3.0 shortly.