apache / opendal

Apache OpenDAL: access data freely.
https://opendal.apache.org
Apache License 2.0
3.27k stars 455 forks source link

feat: Support NebulaGraph #5116

Open GG2002 opened 2 weeks ago

GG2002 commented 2 weeks ago

Which issue does this PR close?

Closes #4553.

Rationale for this change

What changes are included in this PR?

Support read, write, list, delete ops on NebulaGraph.

To access NebulaGraph, users must provide the username and password for NebulaGraph, as well as specify the space, tag, and the key field and value field of the tag.

Since each vertex in NebulaGraph must have a unique ID(called VID), but NebulaGraph lacks a self-incrementing ID, the unique IDs are generated using the Snowflake algorithm. Additionally, the set operation involves first delete existing original vertex, followed by inserting a new vertex.

NebulaGraph does not support the Blob type, hence Base64+string type is employed for file access. If it's supported in the future, just remove Base64.

Are there any user-facing changes?

No(?)

GG2002 commented 1 week ago

Test results are as follows:

$ OPENDAL_TEST=nebulagraph cargo test behavior --features tests -- --nocapture --test-threads=20
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.32s
     Running unittests src/lib.rs (target/debug/deps/opendal-20c211e669cef355)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 128 filtered out; finished in 0.00s

     Running tests/behavior/main.rs (target/debug/deps/behavior-419dea746c216121)

running 81 tests
test behavior::test_check                                     ... ok
test behavior::test_list_with_start_after                     ... ok
test behavior::test_delete_not_existing                       ... ok
test behavior::test_list_non_exist_dir                        ... ok
test behavior::test_delete_empty_dir                          ... ok
test behavior::test_create_dir                                ... ok
test behavior::test_create_dir_existing                       ... ok
test behavior::test_list_root_with_recursive                  ... ok
test behavior::test_list_sub_dir                              ... ok
test behavior::test_list_file_with_recursive                  ... ok
test behavior::test_list_empty_dir                            ... ok
test behavior::test_read_with_if_match                        ... ok
test behavior::test_read_with_if_none_match                   ... ok
test behavior::test_read_not_exist                            ... ok
test behavior::test_read_with_dir_path                        ... ok
test behavior::test_read_with_override_cache_control          ... ok
test behavior::test_read_with_override_content_disposition    ... ok
test behavior::test_read_with_override_content_type           ... ok
test behavior::test_list_dir_with_recursive                   ... ok
test behavior::test_list_dir_with_recursive_no_trailing_slash ... ok
test behavior::test_list_nested_dir                           ... ok
test behavior::test_stat_dir                                  ... ok
test behavior::test_stat_file                                 ... ok
test behavior::test_stat_not_exist                            ... ok
test behavior::test_stat_with_if_match                        ... ok
test behavior::test_stat_with_if_none_match                   ... ok
test behavior::test_stat_with_override_cache_control          ... ok
test behavior::test_stat_with_override_content_disposition    ... ok
test behavior::test_stat_with_override_content_type           ... ok
test behavior::test_stat_root                                 ... ok
test behavior::test_remove_all                                ... ok
test behavior::test_write_with_empty_content                  ... ok
test behavior::test_write_with_dir_path                       ... ok
test behavior::test_write_only                                ... ok
test behavior::test_write_with_cache_control                  ... ok
test behavior::test_write_with_content_type                   ... ok
test behavior::test_write_with_content_disposition            ... ok
test behavior::test_write_with_user_metadata                  ... ok
test behavior::test_writer_write                              ... ok
test behavior::test_write_with_special_chars                  ... ok
test behavior::test_writer_write_with_concurrent              ... ok
test behavior::test_writer_sink                               ... ok
test behavior::test_writer_sink_with_concurrent               ... ok
test behavior::test_list_rich_dir                             ... ok
test behavior::test_writer_abort                              ... ok
test behavior::test_writer_futures_copy                       ... ok
test behavior::test_writer_futures_copy_with_concurrent       ... ok
test behavior::test_stat_nested_parent_dir                    ... ok
test behavior::test_blocking_create_dir                       ... ok
test behavior::test_blocking_create_dir_existing              ... ok
test behavior::test_writer_abort_with_concurrent              ... ok
test behavior::test_stat_not_cleaned_path                     ... ok
test behavior::test_reader                                    ... ok
test behavior::test_blocking_remove_one_file                  ... ok
test behavior::test_blocking_read_not_exist                   ... ok
test behavior::test_list_dir_with_metakey                     ... ok
test behavior::test_blocking_stat_dir                         ... ok
test behavior::test_list_dir                                  ... ok
test behavior::test_blocking_stat_not_exist                   ... ok
test behavior::test_writer_write_with_overwrite               ... ok
test behavior::test_delete_file                               ... ok
test behavior::test_blocking_read_full                        ... ok
test behavior::test_read_full                                 ... ok
test behavior::test_blocking_write_with_dir_path              ... ok
test behavior::test_blocking_delete_file                      ... ok
test behavior::test_blocking_write_file                       ... ok
test behavior::test_blocking_read_range                       ... ok
test behavior::test_blocking_remove_all_basic                 ... ok
test behavior::test_read_range                                ... ok
test behavior::test_remove_all_basic                          ... ok
test behavior::test_list_prefix                               ... ok
test behavior::test_stat_with_special_chars                   ... ok
test behavior::test_delete_with_special_chars                 ... ok
test behavior::test_list_dir_with_file_path                   ... ok
test behavior::test_blocking_stat_file                        ... ok
test behavior::test_blocking_stat_with_special_chars          ... ok
test behavior::test_list_dir_with_metakey_complete            ... ok
test behavior::test_remove_one_file                           ... ok
test behavior::test_delete_stream                             ... ok
test behavior::test_read_with_special_chars                   ... ok
test behavior::test_blocking_write_with_special_chars         ... ok

test result: ok. 81 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.76s
Xuanwo commented 1 week ago

Hi, @GG2002. It might take us a while to complete the communication. Would you like to split this PR into parts so we can merge some of them first? I believe we can add the builder, config first without introduce the client impl in.

GG2002 commented 1 week ago

Hi, @GG2002. It might take us a while to complete the communication. Would you like to split this PR into parts so we can merge some of them first? I believe we can add the builder, config first without introduce the client impl in.

It's ok.