search

Appendium / flatpack

CSV/Tab Delimited and Fixed Length Parser and Writer
http://flatpack.sf.net
Apache License 2.0
57 stars 20 forks source link

Inconsistent raw data in Row returned from BuffReaderFixedParser vs BuffReaderDelimParser #18

Closed icemanltd closed 8 years ago

icemanltd commented 9 years ago

If a user chooses to keep a copy of the raw data in the returned row, both BuffReaderDelimParser and BuffReaderFixedParser add the parsed line to the row that is returned. There is a difference though in that if the line is longer than the metadata indicates, the line is first truncated to the specified length.

In BuffReaderDelimParser, the line is first parsed to columns, then if there are too many columns the extra columns are removed from the array but the orignal line is left unchanged and set in the returned Row.

Would it be possible to return the original line in it's entirety in both cases? I realize that this might create an issue with users already relying on the current behavior. Perhaps it would be possible for the Row to (optionally) have both the original, unmodified line as well as the substring of the line that was used to create the row?

benoitx commented 9 years ago

I'd say that it is a bug, I'd have expected the Row to be the full one and not truncated for the FixedParser. Do you have a pull request with a test for this? many thanks

icemanltd commented 9 years ago

I have made what I think are appropriate changes and added test cases to support both the delimited and fixed parsers.

I have committed and generated a pull request.

benoitx commented 9 years ago

hi

You seem to have cloned an old repo (ks-appendium) rather than appendium/flatpack.

Any chance you could cancel the PR and redo this against master in https://github.com/Appendium/flatpack ?

If not, I'll try manually...

Many thanks

Benoit

On 6 August 2015 at 05:51, icemanltd notifications@github.com wrote:

I have made what I think are appropriate changes and added test cases to support both the delimited and fixed parsers.

I have committed and generated a pull request.

— Reply to this email directly or view it on GitHub https://github.com/Appendium/flatpack/issues/18#issuecomment-128238427.

Follow us on Twitter @appendium http://twitter.com/appendium and

http://www.appendium.com

IMPORTANT NOTICE This communication contains information that is considered confidential and may also be privileged . It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) please note that any form of distribution, copying or use of this communication or the information in it is strictly prohibited and may be unlawful. If you have received this communication in error please return it to the sender and delete the original.

icemanltd commented 9 years ago

I apologize, I see my mistake now. I will cancel those pull requests and do it over against the correct source.

benoitx commented 9 years ago

Any chance of adding a test case for these 2 scenarios? Many thanks in advance.

icemanltd commented 9 years ago

Again, I apologize. I have not used Git before now. The tests were there but I guess they did not get added. I have added them and it appears the pull request was automatically updated?

Date: Sun, 9 Aug 2015 06:09:13 -0700 From: notifications@github.com To: flatpack@noreply.github.com CC: icemanltd@hotmail.com Subject: Re: [flatpack] Inconsistent raw data in Row returned from BuffReaderFixedParser vs BuffReaderDelimParser (#18)

Any chance of adding a test case for these 2 scenarios?

Many thanks in advance.

— Reply to this email directly or view it on GitHub.

benoitx commented 8 years ago

Merged the Pull Request fixing this issue