Closed cjackal closed 3 days ago
I think we can do this. There was a major bug (caused a crash) in PyArrow involving tensor columns. But everything else should be useable. We can enable that, and just disable our unit test for that version of PyArrow.
I'm totally new to the lance world so not really sure about the cost, but it seems
pylance
has rather strong version pinning onpyarrow
(pyarrow<15.0.1
).It didn't use to matter for a while but now that
numpy
2.0 is out andpyarrow
supports it frompyarrow>=16.0.0
, there arises a version conflict - there's no workingpyarrow
version for bothnumpy>=2.0
andpylance
. Another concern is thatpyarrow
is pretty widespread as a mandatory dependency in ML field, so resolving a version conflict is quite tough with some newer libraries.