xiaohe27 / LogReader

The log reader which can read log files
0 stars 0 forks source link

test the time to output huge amount of data to file #32

Closed xiaohe27 closed 9 years ago

xiaohe27 commented 9 years ago

compare two approaches: 1) BufferedWriter 2) Files.write bytes

Also observe the perf diff when using different charset

xiaohe27 commented 9 years ago

third approach: use a stringbuilder to accumulate the output, if it is full then use Files.wrtie to write it to output file and then empty it and repeat the process. At the end, if something is still in stringbuilder, then write it to output file.

xiaohe27 commented 9 years ago

Not the second approach, too many io ops involved...

xiaohe27 commented 9 years ago

Idea: for output to std.out, use stringbuilder for output to file, do not encode to string, just write bytes to file

xiaohe27 commented 9 years ago

use byte buffer to gather output bytes

xiaohe27 commented 9 years ago

Size of output file is unknown, not good idea to use mbb etc. Currently, buffered writer is the most efficient way