uber / aresdb

A GPU-powered real-time analytics storage and query engine.
https://eng.uber.com/aresdb/
Apache License 2.0
3.02k stars 234 forks source link

use GetVectorParty for safe vp request #334

Closed jshencode closed 4 years ago

jshencode commented 4 years ago

panic on vector party request when columnID >= len(batch.Columns) this could happen when new column is created in table but no ingestion happen on the column before query. currently we only expand batch.Columns during ingestion using GetOrCreateVectorParty call

panic: runtime error: index out of range

goroutine 1666532 [running]:
github.com/uber/aresdb/query.(*AQLQueryContext).estimateLiveBatchMemoryUsage(0xc03694b800, 0xc001b58300, 0xc001b58300)
        /home/shz/gocode/pkg/mod/github.com/uber/aresdb@v0.0.3-0.20191028200138-affd4a9f8540/query/aql_processor.go:1046 +0x2c7
github.com/uber/aresdb/query.(*AQLQueryContext).calculateMemoryRequirement(0xc03694b800, 0x16d5d20, 0xc000f9c120, 0x14f575f)
        /home/shz/gocode/pkg/mod/github.com/uber/aresdb@v0.0.3-0.20191028200138-affd4a9f8540/query/aql_processor.go:1006 +0xf1
github.com/uber/aresdb/query.(*AQLQueryContext).FindDeviceForQuery(0xc03694b800, 0x16d5d20, 0xc000f9c120, 0xffffffffffffffff, 0xc000780b40, 0xffffffffffffffff)
        /home/shz/gocode/pkg/mod/github.com/uber/aresdb@v0.0.3-0.20191028200138-affd4a9f8540/query/aql_processor.go:1300 +0x49
github.com/uber/aresdb/api.handleQuery(0x16d5d20, 0xc000f9c120, 0x16a7560, 0xc000af8440, 0xc000780b40, 0xffffffffffffffff, 0x0, 0x0, 0x0, 0x0, ...)
codecov[bot] commented 4 years ago

Codecov Report

Merging #334 into master will increase coverage by 0.05%. The diff coverage is 80%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #334      +/-   ##
==========================================
+ Coverage   71.99%   72.04%   +0.05%     
==========================================
  Files         167      167              
  Lines       23637    23637              
==========================================
+ Hits        17017    17030      +13     
+ Misses       5319     5304      -15     
- Partials     1301     1303       +2
Impacted Files Coverage Δ
query/aql_processor.go 80.76% <80%> (ø) :arrow_up:
subscriber/common/job/driver.go 67.73% <0%> (+1.41%) :arrow_up:
controller/mutators/etcd/schema_mutator.go 63.58% <0%> (+1.73%) :arrow_up:
memstore/cuckoo_index.go 93.28% <0%> (+2.01%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update affd4a9...dc61ddd. Read the comment docs.