CsvParser.stream()

[PawelPacholek]

It would be very useful, if CsvParser can be converted to Stream (java 8). One could then easily use it for functional coding (with lazy evaluation). For example instead of this code:

private void parseCsv(CsvParser parser) {
  String[] row;
  while ((row = parser.parseNext()) != null)
    if (isAppropriate(row))
       doSomethingWith(row);
}

one could have that one:

private void parseCsv(CsvParser parser) {
  parser.stream()
    .filter(this::isAppropriate)
    .forEach(this::doSomethingWith);
}

[jbax]

Will be implemented on a future 3.0.0 version, which will have support for Java 8

[lgoldstein]

Several things to remember though

• A Java 8 Stream does not expect any IOException(s) so need to figure out how to propagate such exception that may occur during the while ((row=parser.nextRow()) != null) loop.
• How to implement single result terminators - e.g., findFirst/findAny - that need to stop the parsing loop
• Parallel streams and I/O require extra care - synchronized access while reading a row ?

[pharmazone]

what is estimations date for 3.0.0 release?

[jbax]

We will start working on it after our HTML parser is released (by the end on June).

Version 3 will possibly come out by the end of July, but keep in mind we don't have fixed release dates as things can change.

On Fri, 25 May 2018, 7:21 am Alexey S. Mirniy, notifications@github.com wrote:

what is estimations date for 3.0.0 release?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/uniVocity/univocity-parsers/issues/78#issuecomment-392010063, or mute the thread https://github.com/notifications/unsubscribe-auth/AGxYPsV0BZ9oc2Zg08De9WHAxUj2FdPBks5t19sygaJpZM4H1K_s .

[pharmazone]

awesome, thanx!

[carbotaniuman]

I'll throw my 2 cents into the issue. I think that following the Files.lines does it for exceptions is a good bet. Same for synchronization.

I think that keeping the CsvParserBuilder before the .stream() is a good call, and I think this would make the library feel a lot more modern.

[carbotaniuman]

@jbax I tried forking your code but I get 19 test failures as of this commit (https://github.com/carbotaniuman/univocity-parsers/commit/f2f866b9b943e2cd4996968472134c754e11a482) when running test. Is there a config error on my side?

[jbax]

Are you on windows? I'm on vacation with a mac and this change might have revealed test cases that need updating or will fail under windows because of line ending differences.

Should work on mac/linux and that's why I didn't spot any error (yet)

On Tue, 8 Jan. 2019, 5:38 am carbotaniuman <notifications@github.com wrote:

@jbax https://github.com/jbax I tried forking your code but I get 19 test failures as of this commit (carbotaniuman@f2f866b https://github.com/carbotaniuman/univocity-parsers/commit/f2f866b9b943e2cd4996968472134c754e11a482) when running test. Is there a config error on my side?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/uniVocity/univocity-parsers/issues/78#issuecomment-452045951, or mute the thread https://github.com/notifications/unsubscribe-auth/AGxYPpaiM-aBvAo_qUsZfGtgrNNQggSLks5vA5sRgaJpZM4H1K_s .

[carbotaniuman]

Yes I am. Here's the complete list of test failures for when you get back (what fun).

https://gist.github.com/carbotaniuman/3e4a9cb584025c82f71749d833ea5b5f

[jbax]

@camerondavison fixed. The tests failed because they use \n as the line ending and on windows the default is \r\n.

[carbotaniuman]

@jbax I still get errors on the tests as of 356ce43.

https://gist.github.com/carbotaniuman/3e4a9cb584025c82f71749d833ea5b5f

[jbax]

@carbotaniuman now the errors happened because you ran the tests on Java 8+ and a couple of them relied on how the HashMap behaves (recent versions are slightly different from Java 6). I've updated the offending tests so it should not be a problem anymore.

[carbotaniuman]

@jbax I think adopting the Files.readLines() method of wrapping a spliterator with StreamSupport is a good call. Is progress going to be on 3.0.0 now?

[skyhirider]

Found this open issue when searching for OpenCsv alternatives. Looking for a parser with stream and seems OpenCsv supports it already so will have to use that library instead.