When running parallel parsers, data gets lost

[IlanZuckerman]

I took 3 csv and I executed them in two matters (obviously not at the same time):

• In parallels /root/csv-to-influxdb -s http://localhost:8086 -d test -m weekend2 -ts timeStamp -b 1 DB_Monitor_Memory.csv & /root/csv-to-influxdb -s http://localhost:8086 -d test -m weekend3 -ts timeStamp -b 1 DB_Monitor_CPU.csv &

• One by one (the same as above, just without the &)

I connected to DB and checked the amount of rows written in every run.

select count(*) from weekend2 name: weekend2

time count_Latency count_allThreads count_bytes count_elapsed count_grpThreads count_label count_success count_timeStamp 0 4535141 4535141 4535141 4535141 4535141 4535141 4535141 4535141

select count(*) from weekend3 name: weekend3

time count_Latency count_allThreads count_bytes count_elapsed count_grpThreads count_label count_success count_timeStamp 0 5022860 5022860 5022860 5022860 5022860 5022860 5022860 5022860

Conclusion: 9.7% of points were lost during parallel execution (!) Total amount of rows in CSVs: 5022863 Total amount of points in parallel run: 4535141 Total amount of points in sequential run: 5022860 (Exactly matches the amount of rows in CSVs minus 3 headers) 4535141 / 5022860 = 90% meaning that 10% were lost.