Only two places where this would be a problem, sync and async version of the same method. TCompactOutputProtocol writes to the inner writer, which might or not might not be a buffered writer. Either way, performs better!
There are additional places where flushes are performed, specifically after writing the footer, but I could not see a good reason to remove that. Posssibly if you were sending multiple files over some type of transport where you wanted to optimize things? But even then, you would have to have quite small files for this to even remotely matter. Either way, I did note that this flush was only performed on the async implementation, so I added it to the sync version as well for consistencies sake.
jorgecarleitao
commented
1 year ago
As per: https://github.com/jorgecarleitao/parquet2/issues/199
Only two places where this would be a problem, sync and async version of the same method. TCompactOutputProtocol writes to the inner writer, which might or not might not be a buffered writer. Either way, performs better!
There are additional places where flushes are performed, specifically after writing the footer, but I could not see a good reason to remove that. Posssibly if you were sending multiple files over some type of transport where you wanted to optimize things? But even then, you would have to have quite small files for this to even remotely matter. Either way, I did note that this flush was only performed on the async implementation, so I added it to the sync version as well for consistencies sake.