[x] Appropriate branch selected (all PRs must first be merged to dev before they can be merged to master)
[x] Any modified or new methods or classes have helpful JavaDoc and code is thoroughly commented
[x] The description lists all applicable issues this PR seeks to resolve
[x] The description lists any configuration setting(s) that differ from the default settings
[x] All tests and CI builds passing
[x] The description lists all relevant PRs included in this release (remove this if not merging to master)
[x] e2e tests are all passing (remove this if not merging to master)
Description
The append transform wasn't handling newlines correctly. Newlines in the middle of a file would cause datatools to stop processing right there, causing problems.
I'm not entirely sold on the solution I've found here of creating a second file to store the non-newlined text contents. My thinking was this was more stable than relying on memory (imagine an >100mb text file, which happens!)
Checklist
devbefore they can be merged tomaster)Description
The append transform wasn't handling newlines correctly. Newlines in the middle of a file would cause datatools to stop processing right there, causing problems.
I'm not entirely sold on the solution I've found here of creating a second file to store the non-newlined text contents. My thinking was this was more stable than relying on memory (imagine an >100mb text file, which happens!)