Open redink opened 9 years ago
Hi,
In my product system, I will fetch very large data from MySQL sometimes, and I using Emysql. In this case, I always got long_gc warnings, just like:
long_gc
monitor long_gc <0.15110.2> [{initial_call,{erlang,apply,2}},{almost_current_function,{io_lib_fread,fread_skip_white,2}},{message_queue_len,0}] [{timeout,59},{old_heap_block_size,1439468},{heap_block_size,1199557},{mbuf_size,0},{stack_size,49},{old_heap_size,79145},{heap_size,15269}] monitor long_gc <0.15106.2> [{initial_call,{erlang,apply,2}},{almost_current_function,{emysql_tcp,type_cast_row_data,2}},{message_queue_len,0}] [{timeout,63},{old_heap_block_size,1439468},{heap_block_size,1199557},{mbuf_size,0},{stack_size,53},{old_heap_size,16491},{heap_size,16069}]
So far, I guess the reason of this case: https://github.com/Eonblast/Emysql/blob/master/src/emysql_tcp.erl#L253
In function to_date, to_float Data will be converted into list from binary, then using io_lib module. And, what do you think about ?
to_date
to_float
Data
list
binary
io_lib
In addition, I am testing "plan b", https://github.com/redink/redink_toys/tree/master/research_for_fun/type_cv
Thanks.
This is probably going to be slow yes. Speeding up the conversion here is definitely possible.
Hi,
In my product system, I will fetch very large data from MySQL sometimes, and I using Emysql. In this case, I always got
long_gc
warnings, just like:So far, I guess the reason of this case: https://github.com/Eonblast/Emysql/blob/master/src/emysql_tcp.erl#L253
In function
to_date
,to_float
Data
will be converted intolist
frombinary
, then usingio_lib
module. And, what do you think about ?In addition, I am testing "plan b", https://github.com/redink/redink_toys/tree/master/research_for_fun/type_cv
Thanks.