Running into JSON::NestingError

RalfLackinger commented 6 years ago

Hello,

I encounter the following error when running my log file with dexter:

/opt/dexter/vendor/ruby-2.3.4/lib/ruby/2.3.0/json/common.rb:156:in `parse': nesting of 101 is too deep (JSON::NestingError)
    from /opt/dexter/vendor/ruby-2.3.4/lib/ruby/2.3.0/json/common.rb:156:in `parse'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/indexer.rb:514:in `plan'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/indexer.rb:164:in `block in calculate_plan'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/indexer.rb:158:in `each'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/indexer.rb:158:in `calculate_plan'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/indexer.rb:182:in `create_hypothetical_indexes'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/indexer.rb:92:in `process_queries'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/processor.rb:72:in `process_queries_without_lock'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/processor.rb:56:in `block in process_queries'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/processor.rb:55:in `synchronize'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/processor.rb:55:in `process_queries'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/processor.rb:49:in `perform'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/lib/dexter/client.rb:23:in `perform'
    from /opt/dexter/vendor/bundle/ruby/2.3.0/gems/pgdexter-0.3.3/exe/dexter:5:in `<top (required)>'
    from /opt/dexter/bin/dexter:17:in `load'
    from /opt/dexter/bin/dexter:17:in `<main>'

This is happening when dexter is processing a very large query:

SQL: EXPLAIN (FORMAT JSON) SELECT ...
...
very large query with 64 joins  
...

Setting --log-level debug2 didn't show any additional output.

The error looks to me like there is a limitation in a Ruby library you are using. Is there anything I can do about this issue? Please let me know if I can provide you with more information.

ankane commented 6 years ago

Hey @RalfLackinger, thanks for the detailed report. I've bump max nesting from 100 to 1000 (didn't even realize it had a limit), and it now handles the error more gracefully. Can you try master or the latest package to see if that fixes it?

RalfLackinger commented 6 years ago

@ankane Thanks for the quick response and patch. I didn't get any more errors after using the new version from master. Worked just fine.

I only ran into some problems with installing the new version, I'm not really familiar with Ruby and it's environment :)

RalfLackinger commented 6 years ago

Sorry, don't know if I should close it or you want to add this patch first to a release and then close it yourself. You can at least few this as resolved from my side with your code change.

ankane commented 6 years ago

Sweet, we can close it out.

ankane / dexter

Running into JSON::NestingError #25