Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Apache License 2.0
1.8k
stars
284
forks
source link
Fix missing hostname attribute in ParseResult #460
According to #279 an issue with pickling ParseResult accross pyspark
driver-executor border may result in a corrupt ParseResult instance
that is lacking hostname attribute.
We try to fix the issue by marshalling url string instead of the
ParseResult instance.
According to #279 an issue with pickling
ParseResultaccross pyspark driver-executor border may result in a corruptParseResultinstance that is lackinghostnameattribute.We try to fix the issue by marshalling url string instead of the
ParseResultinstance.