Open QingZ11 opened 7 months ago
hi, @QingZ11 ,thinks for your good guide. I try to follow your guide. when i finish it, get compiler linking error.
I build it in the nebula-dev:ubuntu2004. It seems this library link the wangle library fail, i don't hava any experiences abount cmake. Can you help to solve it?
hi, @QingZ11 ,thinks for your good guide. I try to follow your guide. when i finish it, get compiler linking error.
I build it in the nebula-dev:ubuntu2004. It seems this library link the wangle library fail, i don't hava any experiences abount cmake. Can you help to solve it?
Hi, @fansehep thank you for your practical contribution. I am pleasure to assist you. @Salieri-004 is nebula bravo core developer, and he will help you with this issue.
@fansehep Hello, have you ever tried modifying cmake as described in the Implementation Approach?
@fansehep Hello, have you ever tried modifying cmake as described in the Implementation Approach?
yep, i had modify it. but this change just about test binary. I think the error is due to a problem with the library not linking correctly to wangle. https://github.com/vesoft-inc/nebula/blob/594ed69e37b740c57aa0f93c986fc01491428db5/src/common/function/CMakeLists.txt#L5-L18 I think the error is about this?
@fansehep Hello, have you ever tried modifying cmake as described in the Implementation Approach?
yep, i had modify it. but this change just about test binary. I think the error is due to a problem with the library not linking correctly to wangle.
I think the error is about this?
From the error messages in your image, we can see that some test compilations are failing because they can't find the implementation of md5Encode
. I believe adding PROXYGEN_LIBRARIES to these tests may be enough to solve your problem.
For example
nebula_add_executable(
NAME arena_bm
SOURCES ArenaBenchmark.cpp
OBJECTS
$<TARGET_OBJECTS:base_obj>
$<TARGET_OBJECTS:datatypes_obj>
$<TARGET_OBJECTS:expression_obj>
$<TARGET_OBJECTS:function_manager_obj>
$<TARGET_OBJECTS:agg_function_manager_obj>
$<TARGET_OBJECTS:memory_obj>
$<TARGET_OBJECTS:time_obj>
$<TARGET_OBJECTS:time_utils_obj>
$<TARGET_OBJECTS:fs_obj>
$<TARGET_OBJECTS:ast_match_path_obj>
$<TARGET_OBJECTS:wkt_wkb_io_obj>
$<TARGET_OBJECTS:datetime_parser_obj>
LIBRARIES
follybenchmark
${THRIFT_LIBRARIES}
${PROXYGEN_LIBRARIES}
)
The idea was provided by @AntiTopQuark , thank you.
Requirement Description
MD5 (Message Digest Algorithm 5) is a widely used cryptographic hash function that produces a 128-bit hash value, commonly used to ensure the integrity and consistency of information transmission. In graph databases, MD5 can be used for rapidly comparing node or edge attribute values, generating unique identifiers, etc. Therefore, implementing the MD5 function will enhance NebulaGraph's data processing capabilities and provide more flexibility in data operations.
NebulaGraph already supports a variety of functions and expressions, such as
split
andconcat
. The implementations of these functions provide us with a reference framework for writing custom functions, which includes but is not limited to:Function registration: Understanding how to register new functions into NebulaGraph's function library.
Argument handling: Becoming familiar with how to process input arguments, including type checking and conversion.
Result returning: Mastering how to compute results and return them to the caller in the appropriate data type.
Implementation Approach
By consulting the official documentation, we can find that NebulaGraph has implemented a variety of functions and expressions, such as string functions. These existing implementations provide us with valuable references, allowing us to quickly locate the corresponding implementations and commit records in the codebase by searching for specific function names, like
json_extract
. For example, we can find commit records and merge request information related tojson_extract
.By analyzing these historical commits, we learn that implementing a new function involves the following steps:
Add the registration and computational logic of the function in the
src/common/function/FunctionManager.cpp
file.Add corresponding unit tests in the
src/common/function/test/FunctionManagerTest.cpp
file.For new features, add TCK integration tests.
(Optional) Update the documentation in the nebula-docs repository, providing explanations and examples for the new feature to help users better understand and use the new functionality.
Implementing the MD5 function may not require us to handle the details of the function from scratch; we can start by conducting a global search in the codebase to see if there are any existing implementations for reference. A simple search can easily locate some available function instances.
Drawing from the implementation of the
json_extract
function, we can write the relevant code for computing MD5 and compile and test it to verify its correctness.After starting the server, we need to verify that the implemented feature meets the expectations.
Referring to the test cases of the
json_extract
function, we added some unit tests for the MD5 function to ensure that it can correctly handle NULL values and regular values.In implementing the MD5 feature, we used the third-party
proxygen::md5Encode
function. Directly compiling the unit tests might encounter some linkage errors, such as undefined references. Therefore, we need to add some dependencies to the Makefile of the unit tests.Execute the unit tests to see if the results meet the expectations:
Great, the results are as expected.
Referring to the article on how to add a test case to NebulaGraph, we added TCK integration tests for the MD5 function and verified its correctness.
Execute the integration tests and verify the results to ensure everything works as expected.
Writing clear documentation is very valuable to help users better understand and use the newly added MD5 function. We encourage each contributor to write related documentation when implementing new features, which is beneficial not only to users but also enhances the contributor's sense of achievement.
中文版描述
为 Nebula Graph 新增一个 MD5() 函数 ## 需求描述 MD5(Message Digest Algorithm 5)是一种广泛使用的哈希算法,生成 128 位的哈希值,通常用于确保信息传输的完整性和一致性。在图数据库中,MD5 可以用于快速比较节点或边的属性值、生成唯一标识符等。因此,实现 MD5 函数将增强 NebulaGraph 处理数据的能力,提供更多的数据操作灵活性。 NebulaGraph 现已支持多种函数和表达式,如`split`、`concat`等。这些函数的实现为我们提供了编写自定义函数的参考框架,包括但不限于: - **函数注册**:了解如何将新函数注册到 NebulaGraph 的函数库中。 - **参数处理**:熟悉如何处理输入参数,包括参数类型检查和转换。 - **结果返回**:掌握如何计算结果并以适当的数据类型返回给调用者。 ## 实现思路 1. 参考已实现的函数 通过查阅官方文档,我们可以发现 NebulaGraph 已经实现了多种函数和表达式,例如:[字符串函数](https://docs.nebula-graph.com.cn/3.6.0/3.ngql-guide/6.functions-and-expressions/2.string/)。这些现有的实现为我们提供了宝贵的参考,使我们能够在代码库中通过搜索特定的函数名称,如 `json_extract`,快速定位到相应的实现和提交记录。例如,我们可以查找到 `json_extract` 相关的[提交记录和合并请求信息](https://github.com/vesoft-inc/nebula/commit/47d6921739e1f06c77992a8c93958c1d14c87e08)。 通过分析这些历史提交,我们得知实现一个新函数需要执行以下步骤: 1. 在 `src/common/function/FunctionManager.cpp` 文件中添加函数的注册和计算逻辑。 2. 在 `src/common/function/test/FunctionManagerTest.cpp` 文件中添加相应的单元测试。 3. 对于新增加的功能,还需要添加TCK集成测试。 4. (可选)更新 [nebula-docs 仓库](https://github.com/vesoft-inc/nebula-docs)中的文档,为新增功能提供说明和示例,以帮助用户更好地理解和使用新功能。 ![image](https://github.com/vesoft-inc/nebula/assets/38887077/519fb998-0591-4db5-9613-92320f2df797) 2. MD5 函数实现 实现 MD5 函数可能不需要我们从头开始处理函数的细节,可以先在代码库中全局搜索看看是否有现成的实现可供参考。通过简单的搜索,我们可以轻松地找到一些可用的函数实例。 ![image](https://github.com/vesoft-inc/nebula/assets/38887077/4aca0292-755e-4c27-bffa-881b2d45cc07) 3. 相关代码实现 借鉴 `json_extract` 函数的实现,我们可以编写计算 MD5 的相关代码,并进行编译和测试以验证其正确性。 ``` # git diff src/common/function/FunctionManager.cpp diff --git a/src/common/function/FunctionManager.cpp b/src/common/function/FunctionManager.cpp index cc2393a77..6ae2b445e 100644 --- a/src/common/function/FunctionManager.cpp +++ b/src/common/function/FunctionManager.cpp @@ -11,6 +11,7 @@ #include