search

jehugaleahsa / FlatFiles

Reads and writes CSV, fixed-length and other flat file formats with a focus on schema definition, configuration and speed.
The Unlicense
357 stars 64 forks source link

Multi-line records #11

Closed flytzen closed 8 years ago

flytzen commented 9 years ago

I have to interface with an old mainframe and they have some rather interesting file formats in there. One, not too difficult, has records that are 45 lines long, with each line having one value. That's relatively straight forward to handle. Another file, however, has each record on three lines, with fixed format data within each line. Is this something you would consider adding support for - or is it just too esoteric?

jehugaleahsa commented 9 years ago

I'm assuming it's a different fixed format on each line. On Oct 6, 2015 5:18 PM, "Frans Lytzen" notifications@github.com wrote:

I have to interface with an old mainframe and they have some rather interesting file formats in there. One, not too difficult, has records that are 45 lines long, with each line having one value. That's relatively straight forward to handle. Another file, however, has each record on three lines, with fixed format data within each line. Is this something you would consider adding support for - or is it just too esoteric?

— Reply to this email directly or view it on GitHub https://github.com/jehugaleahsa/FlatFiles/issues/11.

flytzen commented 9 years ago

Of course :) they don't want to make it easy.

Sent by Outlook for Android

On Tue, Oct 6, 2015 at 2:35 PM -0700, "Travis Parks" notifications@github.com wrote: I'm assuming it's a different fixed format on each line. On Oct 6, 2015 5:18 PM, "Frans Lytzen" notifications@github.com wrote:

I have to interface with an old mainframe and they have some rather interesting file formats in there. One, not too difficult, has records that are 45 lines long, with each line having one value. That's relatively straight forward to handle. Another file, however, has each record on three lines, with fixed format data within each line. Is this something you would consider adding support for - or is it just too esoteric?

— Reply to this email directly or view it on GitHub https://github.com/jehugaleahsa/FlatFiles/issues/11.

Reply to this email directly or view it on GitHub: https://github.com/jehugaleahsa/FlatFiles/issues/11#issuecomment-146008694

jehugaleahsa commented 9 years ago

Just thinking. I was wondering if you pass the same stream to the readers and then read them in the same order over and over. The underlying stream should keep track of its position. The issue is probably the underlying stream reader does buffering and reads more than what's needed for each line. On Oct 6, 2015 5:37 PM, "Frans Lytzen" notifications@github.com wrote:

Of course :) they don't want to make it easy.

Sent by Outlook for Android

On Tue, Oct 6, 2015 at 2:35 PM -0700, "Travis Parks" < notifications@github.com> wrote: I'm assuming it's a different fixed format on each line. On Oct 6, 2015 5:18 PM, "Frans Lytzen" notifications@github.com wrote:

I have to interface with an old mainframe and they have some rather interesting file formats in there. One, not too difficult, has records that are 45 lines long, with each line having one value. That's relatively straight forward to handle. Another file, however, has each record on three lines, with fixed format data within each line. Is this something you would consider adding support for - or is it just too esoteric?

— Reply to this email directly or view it on GitHub https://github.com/jehugaleahsa/FlatFiles/issues/11.

Reply to this email directly or view it on GitHub: https://github.com/jehugaleahsa/FlatFiles/issues/11#issuecomment-146008694

— Reply to this email directly or view it on GitHub https://github.com/jehugaleahsa/FlatFiles/issues/11#issuecomment-146009013 .

flytzen commented 9 years ago

I'll have a play with it, thank you.

Sent by Outlook for Android

On Tue, Oct 6, 2015 at 2:46 PM -0700, "Travis Parks" notifications@github.com wrote: Just thinking. I was wondering if you pass the same stream to the readers and then read them in the same order over and over. The underlying stream should keep track of its position. The issue is probably the underlying stream reader does buffering and reads more than what's needed for each line. On Oct 6, 2015 5:37 PM, "Frans Lytzen" notifications@github.com wrote:

Of course :) they don't want to make it easy.

Sent by Outlook for Android

On Tue, Oct 6, 2015 at 2:35 PM -0700, "Travis Parks" < notifications@github.com> wrote: I'm assuming it's a different fixed format on each line. On Oct 6, 2015 5:18 PM, "Frans Lytzen" notifications@github.com wrote:

I have to interface with an old mainframe and they have some rather interesting file formats in there. One, not too difficult, has records that are 45 lines long, with each line having one value. That's relatively straight forward to handle. Another file, however, has each record on three lines, with fixed format data within each line. Is this something you would consider adding support for - or is it just too esoteric?

— Reply to this email directly or view it on GitHub https://github.com/jehugaleahsa/FlatFiles/issues/11.

Reply to this email directly or view it on GitHub: https://github.com/jehugaleahsa/FlatFiles/issues/11#issuecomment-146008694

— Reply to this email directly or view it on GitHub https://github.com/jehugaleahsa/FlatFiles/issues/11#issuecomment-146009013 .

Reply to this email directly or view it on GitHub: https://github.com/jehugaleahsa/FlatFiles/issues/11#issuecomment-146010968