[BUG] User defined graph analysis algorithm error

[ilovecomet]

Describe the bug Using the official custom algorithm, an error occurred during operation

To Reproduce Steps to reproduce the behavior: code:

from graphscope.analytical.udf.decorators import pregel
from graphscope.framework.app import AppAssets

# 装饰器, 定义顶点数据和消息数据的类型
@pregel(vd_type="double", md_type="double")
class SSSP_Pregel(AppAssets):
    @staticmethod
    def Init(v, context):
        v.set_value(1000000000.0)

    @staticmethod
    def Compute(messages, v, context):
        src_id = context.get_config(b"src")
        cur_dist = v.value()
        new_dist = 1000000000.0
        if v.id() == src_id:
            new_dist = 0
        for message in messages:
            new_dist = min(message, new_dist)
        if new_dist < cur_dist:
            v.set_value(new_dist)
            for e_label_id in range(context.edge_label_num()):
                edges = v.outgoing_edges(e_label_id)
                for e in edges:
                    v.send(e.vertex(), new_dist + e.get_int(2))
        v.vote_to_halt()

    @staticmethod
    def Combine(messages):
        ret = 1000000000.0
        for m in messages:
            ret = min(ret, m)
        return ret

error:

2023-04-17 17:36:24,719 [ERROR][utils:276]: Failed to run command: cmake . -DNETWORKX=ON -DCMAKE_PREFIX_PATH='/usr/local/lib/python3.8/dist-packages/graphscope.runtime;/usr/local/lib/python3.8/dist-packages/graphscope.runtime/openmpi' -DCYTHON_PREGEL_APP=ON -DENABLE_PREGEL_COMBINE=ON, error message is: CMake Error at /usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Python (missing: Python_INCLUDE_DIRS Python_LIBRARIES
  Development Development.Module Development.Embed) (found version "2.7.17")
Call Stack (most recent call first):
  /usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/share/cmake-3.23/Modules/FindPython.cmake:561 (find_package_handle_standard_args)
  CMakeLists.txt:156 (find_package)

2023-04-17 17:36:24,751 [ERROR][rpc:188]: Runstep failed with code: ANALYTICAL_ENGINE_INTERNAL_ERROR, message: Error occurred during RunStep, The traceback is: Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 361, in _compile_on_local
    logger.debug(run_command(command, cwd=workdir))
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 281, in run_command
    raise RuntimeError(f"Failed to run command: {args}, err: {err}")
RuntimeError: Failed to run command: cmake . -DNETWORKX=ON -DCMAKE_PREFIX_PATH='/usr/local/lib/python3.8/dist-packages/graphscope.runtime;/usr/local/lib/python3.8/dist-packages/graphscope.runtime/openmpi' -DCYTHON_PREGEL_APP=ON -DENABLE_PREGEL_COMBINE=ON, err: CMake Error at /usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Python (missing: Python_INCLUDE_DIRS Python_LIBRARIES
  Development Development.Module Development.Embed) (found version "2.7.17")
Call Stack (most recent call first):
  /usr/local/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/share/cmake-3.23/Modules/FindPython.cmake:561 (find_package_handle_standard_args)
  CMakeLists.txt:156 (find_package)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/coordinator.py", line 309, in _RunStep
    head, bodies = self._operation_executor.run_on_analytical_engine(
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/monitor.py", line 193, in runOnAnalyticalEngineWarp
    res = func(instance, dag_def, dag_bodies, loader_op_bodies)
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 167, in run_on_analytical_engine
    dag_def, dag_bodies = self.pre_process(dag_def, dag_bodies, loader_op_bodies)
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 144, in pre_process
    op, _, _ = self._maybe_compile_app(op)
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 257, in _maybe_compile_app
    compiled_path = self._compile_lib_and_distribute(
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/op_executor.py", line 395, in _compile_lib_and_distribute
    lib_path, java_jar_path, java_ffi_path, app_type = compile_func(
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 540, in compile_app
    lib_path = compile_library(commands, library_dir, library_name, launcher)
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 311, in compile_library
    return _compile_on_local(commands, workdir, output_name)
  File "/usr/local/lib/python3.8/dist-packages/gscoordinator/utils.py", line 368, in _compile_on_local
    raise CompilationError(
graphscope.framework.errors.CompilationError: Failed to compile f50bd1a6e93c0147cc21f356cd7595dbe1530149785c69a5b2b72efc7146b635 on platform system: Linux
machine: x86_64
platform: Linux-4.15.0-200-generic-x86_64-with-glibc2.27
uname: uname_result(system='Linux', node='c9c8c977cf55', release='4.15.0-200-generic', version='#211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022', machine='x86_64', processor='x86_64')
kernel_ver: #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022
mac_ver: ('', ('', '', ''), '')
gcc_ver: b'gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0\nCopyright (C) 2017 Free Software Foundation, Inc.\nThis is free software; see the source for copying conditions.  There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n'
python_ver: 3.8.14

Traceback (most recent call last):
  File "/root/demo/code/graph_analysis.py", line 29, in <module>
    sssp_context = my_alg_pie(simple_graph, src=6)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/analytical/udf/wrapper.py", line 63, in call
    return app_assets(graph, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/framework/app.py", line 321, in __call__
    app_ = graph.session._wrapper(AppDAGNode(graph, self))
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/session.py", line 941, in _wrapper
    return self.run(dag_node)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/session.py", line 977, in run
    return self.run_fetches(fetches)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/session.py", line 987, in run_fetches
    response = self._grpc_client.run(fetch_handler.targets)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/rpc.py", line 98, in run
    return self._run_step_impl(dag_def)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/utils.py", line 156, in with_grpc_catch
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/graphscope/client/rpc.py", line 198, in _run_step_impl
    raise exc
graphscope.framework.errors.CompilationError: Failed to compile f50bd1a6e93c0147cc21f356cd7595dbe1530149785c69a5b2b72efc7146b635 on platform system: Linux
machine: x86_64
platform: Linux-4.15.0-200-generic-x86_64-with-glibc2.27
uname: uname_result(system='Linux', node='c9c8c977cf55', release='4.15.0-200-generic', version='#211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022', machine='x86_64', processor='x86_64')
kernel_ver: #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022
mac_ver: ('', ('', '', ''), '')
gcc_ver: b'gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0\nCopyright (C) 2017 Free Software Foundation, Inc.\nThis is free software; see the source for copying conditions.  There is NO\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\n'
python_ver: 3.8.14

Environment (please complete the following information):

• GraphScope version: [0.19.0]
• OS: [ Linux]

[wenyuanyu]

@lidongze0629 Do you have time to look at this?

[lidongze0629]

I will take a look at together with this discussion.

[lidongze0629]

@ilovecomet Hi, from your error message, I think the wrong Python version(2.7.17) was found during the compilation. Could you please try the latest Graphscope, and copy the running code if this problem still exists, thanks very much.

pip3 install graphscope -U