search

apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
https://arrow.apache.org/
Apache License 2.0
14.48k stars 3.52k forks source link

[JS] What is best practice for connecting to an arrow instance in javascript? #36625

Closed rhettjay closed 9 months ago

rhettjay commented 1 year ago

Describe the usage question you have. Please include as many useful details as possible.

In the javascript examples, it's mostly exhibited how one might connect if the arrow files are on fs or directly accessible. What is best practice for connecting to a remote arrow flight instance instance such as connecting a js app to use dremio?

Component(s)

JavaScript

westonpace commented 1 year ago

JS does not have an Arrow Flight implementation that I am aware of. So the best solution would probably be to add one :) .

Without that you might be able to open up some kind of connection and stream results using the streaming version of the arrow IPC format.

There is a bit of dicussion on this issue here: https://github.com/apache/arrow/issues/17325

domoritz commented 9 months ago

Closing as there is no action item besides what we already have covered in other issues.

rhettjay commented 9 months ago

I guess I should have been more specific. It would be nice to see a node library for this. I don't have the bandwidth to get that going and I'm not sure the community is asking for it (other than myself).

domoritz commented 9 months ago

You want an arrow flight server in node? That would be covered by https://github.com/apache/arrow/issues/17325

rhettjay commented 9 months ago

I'm not disagreeing that this should be closed, mind you. I've abandoned direct use of arrow flight altogether. I use dremio (built on arrow flight) and instead of calling the arrow flight db directly the implementation goes through an ETL to a PostgresDB where I can use typescript without issue. Thanks for following up. I'll follow up in the other thread if need be.