search

uber / petastorm

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Apache License 2.0
1.76k stars 281 forks source link

Newer pyarrow versions? #760

Closed winding-lines closed 1 year ago

winding-lines commented 1 year ago

I see that in your workflow file you mention newer versions of pyarrow. Are these supported and if so is there a way to download a pypi package referencing them?

Many thanks!

selitvin commented 1 year ago

Not sure I understand the question. Petastorm's setup.py specifies only lower bound on pyarrow package: 'pyarrow>=0.17.1',