search

sadikovi / spark-netflow

NetFlow data source for Spark SQL and DataFrames
Apache License 2.0
18 stars 11 forks source link

Add closeable iterator #48

Closed sadikovi closed 7 years ago

sadikovi commented 7 years ago

PR adds CloseableIterator similar to Spark's NextIterator, also updates returned Iterator[Row] to InterruptibleIterator.

sadikovi commented 7 years ago

I think it is worth to run benchmarks to make sure this change does not affect performance.

sadikovi commented 7 years ago

Closes #45.

codecov-io commented 7 years ago

Current coverage is 95.92% (diff: 93.33%)

Merging #48 into master will decrease coverage by 0.10%

@@             master        #48   diff @@
==========================================
  Files            20         21     +1   
  Lines           880        907    +27   
  Methods         752        769    +17   
  Messages          0          0          
  Branches        128        138    +10   
==========================================
+ Hits            845        870    +25   
- Misses           35         37     +2   
  Partials          0          0

Powered by Codecov. Last update 3e2eb5d...4788b6d

sadikovi commented 7 years ago

Looks alright, tested on my machine, no significant performance regression.

sadikovi commented 7 years ago

Going to merge this, and do some benchmarking before releasing 1.2.0.