brianmhess / cassandra-loader

Delimited file loader for Cassandra
Apache License 2.0
197 stars 93 forks source link

Issue using Delimiter #36

Closed kdp8791 closed 8 years ago

kdp8791 commented 8 years ago

Is there a reason why the delimiter is only allowed to be one character? Running into an issue where data from an unload also has the same characters as the delimiter. Getting invalid data type exceptions as the parser is splitting up the data in the wrong places. Is there a recommended way to handle this type of scenario?

brianmhess commented 8 years ago

For now, that is a limitation. Performance can be optimized a bit if we have this limitation (versus the general case). We can consider lifting this limitation - which may make sense when revisiting the parsing code. In the meantime, you could consider replacing those delimiters with a single delimiter (using sed). Since you can read from stdin, folks have had success with running: cat file | sed | cassandra-loader.