What exactly is the point of streaming?

[meelash]

First of all, thank you very much for this excellent package!!

However- what exactly is the point of the streaming parser if you have to gather all the segments in memory and build an interchange from them to use them for anything?

Or am I missing something?

I am proposing an option on X12Parser to stream entire transactions instead of individual segments. That way I am consuming this stream in order to extract some data from transactions, I'll operate on the transactions as they come and then discard them, making it possible to run on arbitrarily large files with a small, constant memory load.

[meelash]

From my understanding of the src, it should be fairly trivial to implement since the parser is doing all the work of keeping track of transactions, groups, interchange even when using streaming, although it appears all that information is thrown away because the segment is pushed with none of that information.

In the readme example, then you take the segments array and redo all that work that was done previously, from my understanding?

I'll be working on this in the next day or two, push request should be forthcoming. Feedback on this idea and suggestions very welcome!

[ahuggins-nhs]

That sounds like an excellent idea. Our use case was dealing with a very large, single transaction behind an AS2 endpoint, but I'm seeing where this would improve our handling of large sets of transactions. I'm looking forward to the pull request!

Will definitely get you feedback as I'm available to do so and when reviewing the request.

[ahuggins-nhs]

From my understanding of the src, it should be fairly trivial to implement since the parser is doing all the work of keeping track of transactions, groups, interchange even when using streaming, although it appears all that information is thrown away because the segment is pushed with none of that information.

In the readme example, then you take the segments array and redo all that work that was done previously, from my understanding?

I'll be working on this in the next day or two, push request should be forthcoming. Feedback on this idea and suggestions very welcome!

To your question of how it works when streaming, we treat the segments without prejudice and just pump them out while streaming. We do not try to perform any logic about the object model while streaming. That said, it should be easy to provide the option of doing a minimal check of ST and SE segments, and constructing a X12Transaction object to push into the stream instead of individual segments. I think the ISA, IEA, GS, and GE segments should still come out in order. It will be easy enough since they are not of class X12Transaction to ignore those segments if desired, or perform optional business logic on them.

[ahuggins-nhs]

Closing, as work will eventually continue in the JS-EDI project.

Specifically, there will be eventing so that listening for transactions, groups, segments, or whatever should be trivial.