search

uber / petastorm

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Apache License 2.0
1.78k stars 285 forks source link

Remove driver param for hdfs.connect when using pyarrow 0.17 and above #556

Closed tgaddair closed 4 years ago

tgaddair commented 4 years ago

See: https://github.com/apache/arrow/pull/6432

Related Horovod issue: https://github.com/horovod/horovod/issues/2000

codecov[bot] commented 4 years ago

Codecov Report

Merging #556 into master will decrease coverage by 0.01%. The diff coverage is 75.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #556      +/-   ##
==========================================
- Coverage   86.09%   86.08%   -0.02%     
==========================================
  Files          87       87              
  Lines        4969     4972       +3     
  Branches      792      793       +1     
==========================================
+ Hits         4278     4280       +2     
  Misses        563      563              
- Partials      128      129       +1
Impacted Files Coverage Δ
petastorm/hdfs/namenode.py 97.03% <75.00%> (-0.70%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 444d3cb...6579ce8. Read the comment docs.