Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Apache License 2.0
1.8k
stars
284
forks
source link
Implement detailed logging for hdfs nameservice/namenode resolution #489
In order to simplify solution of HDFS name resolution issues, we should add better logging.
Alternatively: can we fully rely on pyarrow/libhdfs/libhdfs3 for the name service resolution?
In order to simplify solution of HDFS name resolution issues, we should add better logging. Alternatively: can we fully rely on pyarrow/libhdfs/libhdfs3 for the name service resolution?