search

apache / drill

Apache Drill is a distributed MPP query layer for self describing data
https://drill.apache.org/
Apache License 2.0
1.93k stars 979 forks source link

HTTP Storage Plugin OversizedAllocationException when field present in page 1 is absent in page 2 #2896

Open hyperbolix opened 6 months ago

hyperbolix commented 6 months ago

Describe the bug When Apache Drill is configured with the HTTP storage plugin, and INDEX pagination mode is enabled, and the HTTP service configured with that plugin includes a field in all records from page 1, but that field is absent in page 2, a query that spans into page 2 results in an OversizedAllocationException.

To Reproduce Steps to reproduce the behavior:

  1. Start Drill.
  2. Start the Python3 HTTP server that demonstrates the issue: https://gist.github.com/hyperbolix/24b1696dbb03c6032fb7af1a06b75145
  3. Enable the http storage plugin and configure it using this configuration: https://gist.github.com/hyperbolix/7df108c8169cf54bcc72fdc499be1dce
  4. Open the Drill query interface and issue this query: select * from http.a_bug_test where total_results_val=300 and page_size_val=10 limit 15
  5. See OversizedAllocationException error. Here is an example error: https://gist.github.com/hyperbolix/5e2d1e4e10eb92d5a6fa9604acd5cac8
  6. Modify the Python3 HTTP server, commenting out line 46 and fixing indentation on line 47.
  7. Restart the Python3 HTTP server and issue the query again from step 4.
  8. Observe that now the error does not occur and result span into page 2.

Expected behavior Drill is supposed to be resilient to flexible schemas with certain limitations. It should be reasonable for a field to be absent on some pages and present on others, or documentation should indicate the contrary. This would result in the response including the data from page 2 and not producing an exception. If this will not be supported the exception should indicate the issue with the result schema, not indicate that a buffer cannot be expanded.

Error detail, log output or screenshots https://gist.github.com/hyperbolix/5e2d1e4e10eb92d5a6fa9604acd5cac8 org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: OversizedAllocationException: Unable to expand the buffer. Max allowed buffer size is reached.

Drill version Observed in 1.21.1 and also in the latest commit as of 2024-03-26: 749772cb0bd83c1a8fe455410ec80b1e5a9bf239

hyperbolix commented 6 months ago

Additional notes: Omitting the limit 15 clause results in the query succeeding (reading all pages). Adding an order by foo_id clause causes the query to fail in the same way as it failed with the limit 15 clause.

cgivre commented 6 months ago

Ok... this is really weird. This query only seems to fail if the limit is between 13 and 19. It works if the limit is 12 or 20-22.