This PR extends the capabilities CsvReader class, introduced in #2403.
Goals:
Extend testing, verify efficient use of memory
Add Parser capability to allow filtering and processing of (very) large files streamed via network, in files, etc.
Add seeking support to allow indexing, bookmarking, etc.
Add iterator support
The code has been moved into a separate library.
Changes:
Fix String move to avoid de-allocating buffers
Use longer of two buffers for result. Example:
String s1 = "Greater than SSO buffer length";
String s2;
s1 = ""; // Buffer remains allocated
s2 = std::move(s1); // The move we need to fix
At present this move will result in s1's buffer being de-allocated.
This can lead to performance degratation due to subsequent memory reallocations
where a String is being passed around as a general buffer.
This change uses the larger of the two buffers when deciding how to behave.
Checks added to HostTests to enforce predictable behaviour.
Add CStringArray::release method
Allows efficient conversion to a regular String object for manipulation.
Fix CStringArray operator+=
Must take reference, not copy - inefficient and fails when used with FlashString.
Move CsvReader into separate library
The CsvReader class has been moved out of Core/Data and into CsvReader
which has additional capabilities. Changes to existing code:
Add CsvReader to your project's :cpp:envvar:COMPONENT_DEPENDS
Change #include <Data/CsvReader> to #include <CSV/Reader.h>
Change CsvReader class to :cpp:class:CSV::Reader
TODO:
[x] Check documentation
[x] Add sample to demonstrate iteration
[x] Is there a better way to deprecate current CsvReader.h other than an error? Yes: static_assert.
This PR extends the capabilities
CsvReader
class, introduced in #2403.Goals:
The code has been moved into a separate library.
Changes:
Fix String move to avoid de-allocating buffers
Use longer of two buffers for result. Example:
At present this move will result in s1's buffer being de-allocated. This can lead to performance degratation due to subsequent memory reallocations where a String is being passed around as a general buffer.
This change uses the larger of the two buffers when deciding how to behave. Checks added to HostTests to enforce predictable behaviour.
Add CStringArray::release method
Allows efficient conversion to a regular
String
object for manipulation.Fix CStringArray operator+=
Must take reference, not copy - inefficient and fails when used with FlashString.
Move CsvReader into separate library
The
CsvReader
class has been moved out ofCore/Data
and intoCsvReader
which has additional capabilities. Changes to existing code:CsvReader
to your project's :cpp:envvar:COMPONENT_DEPENDS
#include <Data/CsvReader>
to#include <CSV/Reader.h>
CsvReader
class to :cpp:class:CSV::Reader
TODO:
CsvReader.h
other than an error? Yes: static_assert.