Open asfimport opened 2 years ago
Roee Shlomo / @roee88: I assume that backward compatibility is not required for internal use methods (i.e., starting with an underscore). What about jvm_buffer, should it just be kept as is?
Antoine Pitrou / @pitrou: jvm_buffer should probably be kept, yes. We may also want to deprecate it (it's not obvious it's useful in isolation).
As for your other question: indeed, methods starting with an underscore do not enter into backward compatibility concerns.
Roee Shlomo / @roee88: I suspect that a better approach would be to create a new module and keep pyarrow.jvm as is:
Antoine Pitrou / @pitrou: cc @xhochy
Roee Shlomo / @roee88: @pitrou feel free to reuse code from my attempt the other day https://gist.github.com/roee88/4aa7dfeceb2d8c3d8868ed8465ebf561 if that helps. It's based on the java-python integration tests code for ARROW-14374 (with the original test_jvm.py tests updated).
Antoine Pitrou / @pitrou: @amol- ^^
Todd Farmer / @toddfarmer: This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned per project policy. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.
@jorisvandenbossche is there an ongoing effort to integrate C Data interface to pyarrow.jvm
?
I am not aware of someone actually working on this, except for this issue tracking that we should at some point do that.
Would it be okay if I work on this?
Certainly!
take
@jorisvandenbossche I am removing my assignment since focus has been changed and I couldn't attend to this issue timely.
The
pyarrow.jvm
is currently a custom-written bridge between PyArrow and Arrow Java, with limited datatype support. Now that Java implements the C data interface (see ARROW-12965), we should be able to simplify the code while making it more general.Also, we should reenable the
conda-python-jpype
build somewhere, for example in the Crossbow nightly builds.Reporter: Antoine Pitrou / @pitrou
Related issues:
Note: This issue was originally created as ARROW-14319. Please see the migration documentation for further details.