search

LSSTDESC / DC2-production

Configuration, production, validation specifications and tools for the DC2 Data Set.
BSD 3-Clause "New" or "Revised" License
11 stars 7 forks source link

update to recent pyarrow #384

Closed johannct closed 4 years ago

johannct commented 4 years ago

change write_gcr_to_parquet.py so that 2 arguments now obsolete to drive pyarrow are removed

wmwv commented 4 years ago

now obsolete to drive pyarrow

I don't think I quite follow this. What do you mean by "obsolete"?

wmwv commented 4 years ago

Do you mean: "pyarrow has been adopted by Project as the standard Parquet interface. pyarrow doesn't support these options so they are now obsolete for write_gcr_to_parquet.py." ?

heather999 commented 4 years ago

@wmwv I think the issue was that our use of append was not compatible with the "newer" pyarrow 0.13.0 that DM now provides as part of its distribution: #381

wmwv commented 4 years ago

@johannct Do you need anything further to merge this?