search

BlazingDB / blazingsql

BlazingSQL is a lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS cuDF.
https://blazingsql.com
Apache License 2.0
1.92k stars 181 forks source link

[BUG] max_bytes_chunk_read parameter make crash the engine with csv files #1518

Closed Christian8491 closed 3 years ago

Christian8491 commented 3 years ago

Describe the bug when using max_bytes_chunk_read parameter the engine crashes.

Steps/Code to reproduce bug Minimal reproducer

from blazingsql import BlazingContext
bc = BlazingContext()

name_region = ['r_regionkey', 'r_name', 'r_comment']
type_region = ['int32', 'str', 'str']
data_dir = "/dataset/region.psv"

# max_bytes_chunk_read: 4 MBs (enough for the region.psv data)
bc.create_table('region', data_dir, names=name_region, dtype=type_region, delimiter='|', max_bytes_chunk_read=4194304)

result = bc.sql( "select * from region")

Expected behavior Should not crash the engine using this max_bytes_chunk_read size.

Environment overview (please complete the following information)

Environment details Nightly

Additional context

region.psv data:

0|AFRICA|lar deposits. blithely final
1|AMERICA|hs use ironic, even reques
2|ASIA|ges. thinly even pinto beans c
3|EUROPE|ly final courts cajole furi
4|MIDDLE EAST|uickly special account

Error: 

[christiandb:10487:0:10487] Caught signal 11 (Segmentation fault: address not mapped to object at address (nil))
==== backtrace (tid:  10487) ====
 0  /home/christian/miniconda3/envs/bsql20/lib/./libucs.so.0(ucs_handle_error+0x115) [0x7fae64d9aee5]
 1  /home/christian/miniconda3/envs/bsql20/lib/./libucs.so.0(+0x26281) [0x7fae64d9b281]
 2  /home/christian/miniconda3/envs/bsql20/lib/./libucs.so.0(+0x26452) [0x7fae64d9b452]
 3  /home/christian/miniconda3/envs/bsql20/jre/lib/amd64/server/libjvm.so(+0x93c9b4) [0x7fae63cdd9b4]
 4  /home/christian/miniconda3/envs/bsql20/jre/lib/amd64/server/libjvm.so(JVM_handle_linux_signal+0x9d) [0x7fae63ce584d]
 5  /home/christian/miniconda3/envs/bsql20/jre/lib/amd64/server/libjvm.so(+0x939233) [0x7fae63cda233]
 6  /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7faf4ad08980]
 7  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZSt4swapIPiENSt9enable_ifIXsrSt6__and_IJSt6__not_ISt15__is_tuple_likeIT_EESt21is_move_constructibleIS5_ESt18is_move_assignableIS5_EEE5valueEvE4typeERS5_SF_+0x2b) [0x7fae65f21b81]
 8  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZNSt12_Vector_baseIiSaIiEE12_Vector_impl12_M_swap_dataERS2_+0x23) [0x7fae65f1e2bf]
 9  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZNSt6vectorIiSaIiEE14_M_move_assignEOS1_St17integral_constantIbLb1EE+0x65) [0x7fae66021933]
10  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZNSt6vectorIiSaIiEEaSEOS1_+0x47) [0x7fae660203b5]
11  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZN3ral5batch17BindableTableScanC2EmRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS_2io13data_providerEESA_INSB_11data_parserEERNSB_6SchemaESA_IN9blazingdb7manager7ContextEESA_INS_5cache5graphEE+0x4d6) [0x7fae6607db40]
12  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZN9__gnu_cxx13new_allocatorIN3ral5batch17BindableTableScanEE9constructIS3_JRmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS1_2io13data_providerEESE_INSF_11data_parserEERNSF_6SchemaERSE_IN9blazingdb7manager7ContextEERSE_INS1_5cache5graphEEEEEvPT_DpOT0_+0x190) [0x7fae6615fa90]
13  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZNSt16allocator_traitsISaIN3ral5batch17BindableTableScanEEE9constructIS2_JRmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS0_2io13data_providerEESE_INSF_11data_parserEERNSF_6SchemaERSE_IN9blazingdb7manager7ContextEERSE_INS0_5cache5graphEEEEEvRS3_PT_DpOT0_+0xb3) [0x7fae6615b635]
14  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZNSt23_Sp_counted_ptr_inplaceIN3ral5batch17BindableTableScanESaIS2_ELN9__gnu_cxx12_Lock_policyE2EEC2IJRmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS0_2io13data_providerEESG_INSH_11data_parserEERNSH_6SchemaERSG_IN9blazingdb7manager7ContextEERSG_INS0_5cache5graphEEEEES3_DpOT_+0x153) [0x7fae66155a61]
15  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EEC1IN3ral5batch17BindableTableScanESaIS6_EJRmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS4_2io13data_providerEESG_INSH_11data_parserEERNSH_6SchemaERSG_IN9blazingdb7manager7ContextEERSG_INS4_5cache5graphEEEEESt19_Sp_make_shared_tagPT_RKT0_DpOT1_+0x1a8) [0x7fae66150080]
16  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZNSt12__shared_ptrIN3ral5batch17BindableTableScanELN9__gnu_cxx12_Lock_policyE2EEC2ISaIS2_EJRmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS0_2io13data_providerEESG_INSH_11data_parserEERNSH_6SchemaERSG_IN9blazingdb7manager7ContextEERSG_INS0_5cache5graphEEEEESt19_Sp_make_shared_tagRKT_DpOT0_+0xda) [0x7fae6614b470]
17  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZNSt10shared_ptrIN3ral5batch17BindableTableScanEEC2ISaIS2_EJRmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES_INS0_2io13data_providerEES_INSE_11data_parserEERNSE_6SchemaERS_IN9blazingdb7manager7ContextEERS_INS0_5cache5graphEEEEESt19_Sp_make_shared_tagRKT_DpOT0_+0xbd) [0x7fae6614691b]
18  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZSt15allocate_sharedIN3ral5batch17BindableTableScanESaIS2_EJRmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS0_2io13data_providerEESC_INSD_11data_parserEERNSD_6SchemaERSC_IN9blazingdb7manager7ContextEERSC_INS0_5cache5graphEEEESC_IT_ERKT0_DpOT1_+0xb4) [0x7fae66142397]
19  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZSt11make_sharedIN3ral5batch17BindableTableScanEJRmRNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS0_2io13data_providerEESB_INSC_11data_parserEERNSC_6SchemaERSB_IN9blazingdb7manager7ContextEERSB_INS0_5cache5graphEEEESB_IT_EDpOT0_+0xf2) [0x7fae6613e4a0]
20  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZN3ral5batch14tree_processor11make_kernelEmNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10shared_ptrINS_5cache5graphEE+0x6ba) [0x7fae6613507a]
21  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZN3ral5batch14tree_processor19expr_tree_from_jsonEmRKN5boost13property_tree11basic_ptreeINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESA_St4lessISA_EEEPNS0_4nodeEiSt10shared_ptrINS_5cache5graphEE+0x15a) [0x7fae66135ef4]
22  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_ZN3ral5batch14tree_processor17build_batch_graphENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xfb) [0x7fae6613aa1d]
23  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_Z14generate_graphSt6vectorIN3ral2io11data_loaderESaIS2_EES_INS1_6SchemaESaIS5_EES_INSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaISD_EESF_SD_RN9blazingdb7manager7ContextERKSD_+0x2bb) [0x7fae6612e00c]
24  /home/christian/miniconda3/envs/bsql20/lib/libblazingsql-engine.so(_Z16runGenerateGraphjSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS5_EES7_S7_S_I11TableSchemaSaIS8_EES_IS7_SaIS7_EESC_SC_S_IiSaIiEEiS5_S_IS_ISt3mapIS5_S5_St4lessIS5_ESaISt4pairIKS5_S5_EEESaISM_EESaISO_EESM_S5_S5_+0x3e4) [0x7fae661b5b5a]
25  /home/christian/miniconda3/envs/bsql20/lib/python3.8/site-packages/cio.cpython-38-x86_64-linux-gnu.so(+0x6daea) [0x7fae67949aea]
26  python(PyCFunction_Call+0x58) [0x55ec515f2348]
27  python(_PyObject_MakeTpCall+0x23c) [0x55ec515e1dbc]
28  python(_PyEval_EvalFrameDefault+0x4596) [0x55ec5166d666]
29  python(_PyEval_EvalCodeWithName+0x8b1) [0x55ec51637821]
30  python(_PyFunction_Vectorcall+0x1e3) [0x55ec516380a3]
31  python(+0x103a40) [0x55ec515a1a40]
32  python(_PyEval_EvalCodeWithName+0x300) [0x55ec51637270]
33  python(PyEval_EvalCode+0x23) [0x55ec516cc543]
34  python(+0x22e5e4) [0x55ec516cc5e4]
35  python(+0x254854) [0x55ec516f2854]
36  python(+0x115390) [0x55ec515b3390]
37  python(PyRun_SimpleFileExFlags+0x384) [0x55ec515b60d2]
38  python(+0x118bf0) [0x55ec515b6bf0]
39  python(Py_BytesMain+0x39) [0x55ec516f5a09]
40  /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7faf4a926bf7]
41  python(+0x1e6fe5) [0x55ec51684fe5]
=================================
Segmentation fault (core dumped)
Christian8491 commented 3 years ago

It could be related to #1512 .