All the points are (0, 0, 0) when I read a huge .ply file, both on Windows and Linux.
I seem to have figured out why:
I pre-load the entire file upfront (preload_into_memory = true), just like in example.cpp and example-utils.hpp. (It seems to be prepared for files smaller than 1GB. It's already stated in the notes.)
In my experiment, this occurs when the .ply file > 2.01GB (2160000275 bytes) on Windows, and > 2.48GB (2685605907 bytes) on Ubuntu.
But the memoryStream capacity is automatically expanded according to the reference
[@ SaiKishor-MSFT]
The maximum size of a MemoryStream object that can be handled by the System.IO.MemoryStream class is determined by the amount of available memory on the system. The maximum size of a MemoryStream object is 2 gigabytes (GB) by default.
My questions are:
Do others get an all-zero error when reading a large file? (Does the capacity of MemoryStream automatically expand?)
If all-zero errors are common, do you need to add a judgment: If the file size is too large, preload is not allowed? like this (in example.cpp, line 64):
// For most files < 1gb, pre-loading the entire file upfront and wrapping it into a
// stream is a net win for parsing speed, about 40% faster.
std::ifstream file_helper(filepath, std::ios::binary);
file_helper.seekg(0, std::ios::end);
std::streamsize byte_size = file_helper.tellg(); // get the byte_size of the file to be read
file_helper.close();
if (preload_into_memory && byte_size < 1.9e9)
{
byte_buffer = read_file_binary(filepath);
file_stream.reset(new memory_stream((char*)byte_buffer.data(), byte_buffer.size()));
}
else
{
file_stream.reset(new std::ifstream(filepath, std::ios::binary));
}
All the points are (0, 0, 0) when I read a huge .ply file, both on Windows and Linux. I seem to have figured out why: I pre-load the entire file upfront (preload_into_memory = true), just like in example.cpp and example-utils.hpp. (It seems to be prepared for files smaller than 1GB. It's already stated in the notes.) In my experiment, this occurs when the .ply file > 2.01GB (2160000275 bytes) on Windows, and > 2.48GB (2685605907 bytes) on Ubuntu.
But the memoryStream capacity is automatically expanded according to the reference
My questions are:
Do others get an all-zero error when reading a large file? (Does the capacity of MemoryStream automatically expand?)
If all-zero errors are common, do you need to add a judgment: If the file size is too large, preload is not allowed? like this (in example.cpp, line 64):