search

sul-dlss-deprecated / rialto-etl

ETL tools for RIALTO, Stanford Libraries' research intelligence project
https://library.stanford.edu/projects/rialto
Apache License 2.0
3 stars 0 forks source link

Error handling invalid grant identifiers #341

Closed justinlittman closed 5 years ago

justinlittman commented 5 years ago

From time exe/transform call WebOfScience -i data/wos/wos_2012-01-01+2012-06-30.ndj > data/wos/wos_2012-01-01+2012-06-30.sparql:

2019-01-11T05:40:59-08:00 [ WARN] 12859 : retrying connection (5 remaining) to https://rialto-resolver.sul.stanford.edu/grant?identifier=%5C21173004: (Faraday::RetriableResponse)  500
2019-01-11T05:41:04-08:00 [ WARN] 12859 : retrying connection (4 remaining) to https://rialto-resolver.sul.stanford.edu/grant?identifier=%5C21173004: (Faraday::RetriableResponse)  500
2019-01-11T05:41:15-08:00 [ WARN] 12859 : retrying connection (3 remaining) to https://rialto-resolver.sul.stanford.edu/grant?identifier=%5C21173004: (Faraday::RetriableResponse)  500
2019-01-11T05:41:35-08:00 [ WARN] 12859 : retrying connection (2 remaining) to https://rialto-resolver.sul.stanford.edu/grant?identifier=%5C21173004: (Faraday::RetriableResponse)  500
2019-01-11T05:42:15-08:00 [ WARN] 12859 : retrying connection (1 remaining) to https://rialto-resolver.sul.stanford.edu/grant?identifier=%5C21173004: (Faraday::RetriableResponse)  500
2019-01-11T05:43:35-08:00 [ WARN] 12859 : retrying connection (0 remaining) to https://rialto-resolver.sul.stanford.edu/grant?identifier=%5C21173004: (Faraday::RetriableResponse)  500
Traceback (most recent call last):
        21: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/concurrent-ruby-1.1.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'
        20: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/concurrent-ruby-1.1.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
        19: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/concurrent-ruby-1.1.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
        18: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/concurrent-ruby-1.1.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
        17: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/concurrent-ruby-1.1.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
        16: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/concurrent-ruby-1.1.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
        15: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/traject-3.0.0/lib/traject/thread_pool.rb:123:in `block in maybe_in_thread_pool'
        14: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/traject-3.0.0/lib/traject/indexer.rb:554:in `block (3 levels) in process'
        13: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/traject-3.0.0/lib/traject/indexer.rb:429:in `map_to_context!'
        12: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/traject-3.0.0/lib/traject/indexer.rb:429:in `each'
        11: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/traject-3.0.0/lib/traject/indexer.rb:435:in `block in map_to_context!'
        10: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/traject-3.0.0/lib/traject/indexer.rb:476:in `handle_mapping_errors'
         9: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/traject-3.0.0/lib/traject/indexer.rb:436:in `block (2 levels) in map_to_context!'
         8: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/traject-3.0.0/lib/traject/indexer/step.rb:135:in `execute'
         7: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/traject-3.0.0/lib/traject/indexer/step.rb:135:in `each'
         6: from /opt/app/rialto/rialto/shared/bundle/ruby/2.5.0/gems/traject-3.0.0/lib/traject/indexer/step.rb:138:in `block in execute'
         5: from lib/rialto/etl/configs/wos_to_sparql_statements.rb:216:in `block (2 levels) in load_config_file'
         4: from lib/rialto/etl/configs/wos_to_sparql_statements.rb:216:in `map'
         3: from lib/rialto/etl/configs/wos_to_sparql_statements.rb:217:in `block (3 levels) in load_config_file'
         2: from /opt/app/rialto/rialto/releases/20190110192655/lib/rialto/etl/transformers/grants.rb:11:in `resolve_grant'
         1: from /opt/app/rialto/rialto/releases/20190110192655/lib/rialto/etl/service_client/entity_resolver.rb:17:in `resolve'
/opt/app/rialto/rialto/releases/20190110192655/lib/rialto/etl/service_client/entity_resolver.rb:40:in `resolve': Entity resolver returned 500 for grant type and {"identifier"=>"\\21173004"} params. (RuntimeError)
justinlittman commented 5 years ago

Which produces a malformed SPARQL query in entity resolver:

entity-resolver_1         | 2019/01/11 14:12:27 http: panic serving 172.20.0.1:34594: Query: SPARQL request failed: 400 Bad Request. Response body: 
entity-resolver_1         | SPARQL-QUERY: queryStr=SELECT ?id
entity-resolver_1         |             WHERE {
entity-resolver_1         |                 ?id a <http://vivoweb.org/ontology/core#Grant> .
entity-resolver_1         |                 ?id <http://purl.org/dc/terms/identifier> "\21173004" .
entity-resolver_1         |             }
entity-resolver_1         | java.util.concurrent.ExecutionException: org.openrdf.query.MalformedQueryException: Lexical error at line 4, column 49.  Encountered: "2" (50), after : "\"\\"
entity-resolver_1         |     at java.util.concurrent.FutureTask.report(FutureTask.java:122)
entity-resolver_1         |     at java.util.concurrent.FutureTask.get(FutureTask.java:192)
entity-resolver_1         |     at com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:281)
entity-resolver_1         |     at com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:653)
entity-resolver_1         |     at com.bigdata.rdf.sail.webapp.QueryServlet.doPost(QueryServlet.java:273)
entity-resolver_1         |     at com.bigdata.rdf.sail.webapp.RESTServlet.doPost(RESTServlet.java:269)
entity-resolver_1         |     at com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doPost(MultiTenancyServlet.java:193)
entity-resolver_1         |     at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
entity-resolver_1         |     at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
entity-resolver_1         |     at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)
entity-resolver_1         |     at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
entity-resolver_1         |     at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
entity-resolver_1         |     at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
entity-resolver_1         |     at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
entity-resolver_1         |     at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)
entity-resolver_1         |     at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
entity-resolver_1         |     at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
entity-resolver_1         |     at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)
entity-resolver_1         |     at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
entity-resolver_1         |     at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
entity-resolver_1         |     at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
entity-resolver_1         |     at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
entity-resolver_1         |     at org.eclipse.jetty.server.Server.handle(Server.java:497)
entity-resolver_1         |     at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
entity-resolver_1         |     at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)
entity-resolver_1         |     at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
entity-resolver_1         |     at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)
entity-resolver_1         |     at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)
entity-resolver_1         |     at java.lang.Thread.run(Thread.java:748)
jcoyne commented 5 years ago

Should this be moved to https://github.com/sul-dlss/rialto-entity-resolver ?

justinlittman commented 5 years ago

Agree @jcoyne

justinlittman commented 5 years ago

Moved to https://github.com/sul-dlss/rialto-entity-resolver/issues/49