sadikovi / spark-netflow

NetFlow data source for Spark SQL and DataFrames

Apache License 2.0

18 stars 11 forks source link

Report file for exceptions #68

Closed sadikovi closed 7 years ago

sadikovi commented 7 years ago

This PR addresses issue #67, and adds reporting of corrupt file when read fails either during reading of a header or record. Patch handles issue in DefaultSource directly, not in NetFlowReader - reason is that reader is stream based and introducing path attribute does not make sense and would involve fair amount of changes to either add it and propagate into buffers or convert reader to work on file system and path.

codecov-io commented 7 years ago

Codecov Report

Merging #68 into master will increase coverage by 0.05%. The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #68      +/-   ##
==========================================
+ Coverage   93.93%   93.98%   +0.05%     
==========================================
  Files          12       12              
  Lines         445      449       +4     
  Branches       35       35              
==========================================
+ Hits          418      422       +4     
  Misses         27       27