Search returning REXML::ParseException error

ianlin0126 commented 11 years ago

I found that the search function for some queries frequently returns REXML::ParseException error. The issue is especially severe on Heroku Cedar stack (for unknown reason). And it only happens for some queries. I'm suspecting that there are some issues with parsing the XML response.

The following is my code:

client = ASIN::Client.instance

client.search(:Keywords => "Shark Tale", :SearchIndex => :UnboxVideo, :ResponseGroup => :ItemAttributes)

And the following is the error I'm seeing. It rarely fails on my local machine, but it shows up frequently on heroku:

REXML::ParseException: #<NoMethodError: undefined method `[]' for nil:NilClass>
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/rexml/parsers/baseparser.rb:340:in `pull_event'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/rexml/parsers/baseparser.rb:183:in `pull'
/app/vendor/bundle/ruby/2.0.0/gems/crack-0.4.0/lib/crack/xml.rb:203:in `parse'
/app/vendor/bundle/ruby/2.0.0/gems/crack-0.4.0/lib/crack/xml.rb:235:in `parse'
/app/vendor/bundle/ruby/2.0.0/gems/asin-1.1.2/lib/asin/client.rb:375:in `call'
/app/vendor/bundle/ruby/2.0.0/gems/asin-1.1.2/lib/asin/client.rb:197:in `search'
(irb):223:in `irb_binding'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb/workspace.rb:86:in `eval'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb/workspace.rb:86:in `evaluate'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb/context.rb:380:in `evaluate'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb.rb:492:in `block (2 levels) in eval_input'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb.rb:624:in `signal_status'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb.rb:489:in `block in eval_input'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb/ruby-lex.rb:247:in `block (2 levels) in each_top_level_statement'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb/ruby-lex.rb:233:in `loop'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb/ruby-lex.rb:233:in `block in each_top_level_statement'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb/ruby-lex.rb:232:in `catch'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb/ruby-lex.rb:232:in `each_top_level_statement'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb.rb:488:in `eval_input'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb.rb:397:in `block in start'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb.rb:396:in `catch'
/app/vendor/ruby-2.0.0/lib/ruby/2.0.0/irb.rb:396:in `start'
/app/vendor/bundle/ruby/2.0.0/gems/railties-3.2.8/lib/rails/commands/console.rb:47:in `start'
/app/vendor/bundle/ruby/2.0.0/gems/railties-3.2.8/lib/rails/commands/console.rb:8:in `start'
/app/vendor/bundle/ruby/2.0.0/gems/railties-3.2.8/lib/rails/commands.rb:41:in `<top (required)>'
script/rails:6:in `require'
script/rails:6:in `<main>'
...
Exception parsing
Line: 1
Position: 30315
Last 80 unconsumed characters:
Actor><Actor>Matthew Broderick</Actor><Actor>Patrick Warburton</Actor><Actor>J
from /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/rexml/parsers/baseparser.rb:439:in `rescue in pull_event'
from /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/rexml/parsers/baseparser.rb:331:in `pull_event'
from /app/vendor/ruby-2.0.0/lib/ruby/2.0.0/rexml/parsers/baseparser.rb:183:in `pull'
from /app/vendor/bundle/ruby/2.0.0/gems/crack-0.4.0/lib/crack/xml.rb:203:in `parse'
from /app/vendor/bundle/ruby/2.0.0/gems/crack-0.4.0/lib/crack/xml.rb:235:in `parse'
from /app/vendor/bundle/ruby/2.0.0/gems/asin-1.1.2/lib/asin/client.rb:375:in `call'
from /app/vendor/bundle/ruby/2.0.0/gems/asin-1.1.2/lib/asin/client.rb:197:in `search'
from (irb):223
from /app/vendor/bundle/ruby/2.0.0/gems/railties-3.2.8/lib/rails/commands/console.rb:47:in `start'
from /app/vendor/bundle/ruby/2.0.0/gems/railties-3.2.8/lib/rails/commands/console.rb:8:in `start'
from /app/vendor/bundle/ruby/2.0.0/gems/railties-3.2.8/lib/rails/commands.rb:41:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'

An example where this issue doesn't occur is with a slightly different query

client.search(:Keywords => "Shark Tales", :SearchIndex => :UnboxVideo, :ResponseGroup => :ItemAttributes)

phoet commented 11 years ago

from what i can see you are using ruby 2.0.0. are you also using HTTPClient? than this might bite you: https://github.com/nahi/httpclient/issues/153

as a workaround i can recommend switching to CURB

i did that with the example-project on heroku: http://asin.herokuapp.com/

ianlin0126 commented 11 years ago

Thanks phoet! The ruby version is the exact reason of the discrepancy, your example project with CURB fixes the problem :)

phoet / asin

Search returning REXML::ParseException error #25