search

apache / incubator-graphar

An open source, standard data file format for graph data storage and retrieval.
https://graphar.apache.org/
Apache License 2.0
211 stars 46 forks source link

[Feat] Provide libraries for other languages #72

Open acezen opened 1 year ago

acezen commented 1 year ago

Is your feature request related to a problem? Please describe. Currently the libraries for GraphAr are only available for C++ and Spark. But many graph processing systems are implemented by other programming languages (like Neo4j by java). We need to provide libraries for more programming languages.

Describe the solution you'd like Implement library with

kesavkolla commented 1 year ago

Any movement on this? Maybe support a webassembly binding so that we can use it from any other programming language. WASI is now a days is the default binding.

acezen commented 1 year ago

Any movement on this? Maybe support a webassembly binding so that we can use it from any other programming language. WASI is now a days is the default binding.

Hi, @kesavkolla , thanks you for your interest in GraphAr. The work of implement libraries for languages is not start yet but it would be our next major work of development.

I'm not very familiar with webassembly binding but according to the document it seems to be a great solution to make GraphAr accessible for use with programming languages other than C++ and Spark. It would be very helpful if you can provide some information about how do you want to use with GraphAr and idea or proposal about webassembly binding with GraphAr?

kesavkolla commented 1 year ago

@acezen I am seeing this as a trend with many open source projects where the base SDK is developed in one language either C or rust and other language SDKs are derived from WASI. WASI has good compatible layer with many implementations like wasmer, wasmtime etc... it will be so easy to use in any programming language. Investing in each programming language as a separate SDK requires lot of effort in each programming language. So IMO wasm is the right choice to go with.

kesavkolla commented 1 year ago

I am actually looking for Rust client so that I can load GraphAr in rust and start leveraging Gremlin path traversal.

kesavkolla commented 1 year ago

@acezen I have asked a question in hugegraph community about support for GraphAr https://github.com/apache/incubator-hugegraph/discussions/2149

They also say GraphAR to support WASM binding. IMO This is really a huge thing so that many programming languages can start using GraphAR.

sighingnow commented 1 year ago

Hi @kesavkolla,

I am actually looking for Rust client so that I can load GraphAr in rust and start leveraging Gremlin path traversal.

Yes, we also have the plan to add rust support to read and build graph data in GraphAr format. We currently have a on-going refactor (#116) to have a clear repo layout for multi-languages supported. We could launch the Rust SDK after that PR merged and it would nice if we could have some collaboration to develop the Rust SDK.

I have asked a question in hugegraph community about support for GraphAr

Thanks for introducing. I have left a comment on that thread to discuss the detail.

base SDK is developed in one language either C or rust and other language SDKs are derived from WASI.

May I know more details about this idea? If I understand it correctly, WASM is usually used for porting/migrating existing software developing in C++/Rust to browser or edge computing.

kesavkolla commented 1 year ago

May I know more details about this idea? If I understand it correctly, WASM is usually used for porting/migrating existing software developing in C++/Rust to browser or edge computing.

WASM is for running things in browser but there is WASI which makes thing to run on server side. There are many WASI compatible hosts are avaialble eg: (wasmtime, wasmer, wasmedge etc...). All these support cross language bindings. If we have WASI compatible assembly it can be used in any programming language. WASI makes that portability. To my knowledge WASI has support for file systems, environment variables etc... in addition to standard WASM. There is also experimental proposal for sockets and HTTP too. WASI is growing rapidly.

https://github.com/WebAssembly/WASI

sighingnow commented 1 year ago

All these support cross language bindings. If we have WASI compatible assembly it can be used in any programming language. WASI makes that portability.

It won't be hard to compile current C++ implementation to WASM with the help of WASI (we only requires apache-arrow and a small YAML parser library). However to make it friend to users the APIs may need carefully design to expose to WASM.

For multiple language support, I think we prefer to implement the SDKs in these languages rather than via WASM to keep a nice language-specific interfaces and deliver good run time performance. There's currently no concrete proposal and needs further discussion, though.