在kuscia-master容器执行kubectl apply -f job-best-effort-linear.yaml命令，无法看到kuscia job执行状态

Yanziwanglu commented 6 months ago

配置好了KusciaJob，在kuscia-master容器执行kubectl apply -f job-best-effort-linear.yaml命令，提示kusciajob.kuscia.secretflow/job-best-effort-linear created。

1、执行命令：kubectl get kj -n cross-domain

2、执行命令：kubectl get kj

3、kubectl apply -f job-best-effort-linear.yaml配置如下：

4、日志

error.log

问题：为什么kubectl get kj -n cross-domain看不到任务名称，kubectl get kj看不到任务状态？

zimu-yuxi commented 6 months ago

很抱歉，这是此处文档问题metadata缺少namespace配置，这个我们会进行优化。可以参考下此处的文档执行。（需要将之前的任务删除掉）

wenkesong-li commented 6 months ago

你好~ 应该是缺少了namespace的配置上面我同事已经回复您了

kubectl get kj -n {namespace的名称}

Yanziwanglu commented 6 months ago

1、增加namespace配置namespace: cross-domain，在执行的时候提示失败。

2、使用kubectl get kj job-best-effort-linear -n cross-domain -o yaml命令输出如下：

apiVersion: kuscia.secretflow/v1alpha1
kind: KusciaJob
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"kuscia.secretflow/v1alpha1","kind":"KusciaJob","metadata":{"annotations":{},"name":"job-best-effort-linear","namespace":"cross-domain"},"spec":{"initiator":"alice","maxParallelism":2,"scheduleMode":"BestEffort","tasks":[{"alias":"job-psi","appImage":"secretflow-image","parties":[{"domainID":"alice"},{"domainID":"bob"}],"priority":100,"taskID":"job-psi","taskInputConfig":"{\"sf_datasource_config\":{\"alice\":{\"id\":\"default-data-source\"},\"bob\":{\"id\":\"default-data-source\"}},\"sf_cluster_desc\":{\"parties\":[\"alice\",\"bob\"],\"devices\":[{\"name\":\"spu\",\"type\":\"spu\",\"parties\":[\"alice\",\"bob\"],\"config\":\"{\\\"runtime_config\\\":{\\\"protocol\\\":\\\"REF2K\\\",\\\"field\\\":\\\"FM64\\\"},\\\"link_desc\\\":{\\\"connect_retry_times\\\":60,\\\"connect_retry_interval_ms\\\":1000,\\\"brpc_channel_protocol\\\":\\\"http\\\",\\\"brpc_channel_connection_type\\\":\\\"pooled\\\",\\\"recv_timeout_ms\\\":1200000,\\\"http_timeout_ms\\\":1200000}}\"},{\"name\":\"heu\",\"type\":\"heu\",\"parties\":[\"alice\",\"bob\"],\"config\":\"{\\\"mode\\\": \\\"PHEU\\\", \\\"schema\\\": \\\"paillier\\\", \\\"key_size\\\": 2048}\"}],\"ray_fed_config\":{\"cross_silo_comm_backend\":\"brpc_link\"}},\"sf_node_eval_param\":{\"domain\":\"data_prep\",\"name\":\"psi\",\"version\":\"0.0.1\",\"attr_paths\":[\"input/receiver_input/key\",\"input/sender_input/key\",\"protocol\",\"precheck_input\",\"bucket_size\",\"curve_type\"],\"attrs\":[{\"ss\":[\"id1\"]},{\"ss\":[\"id2\"]},{\"s\":\"ECDH_PSI_2PC\"},{\"b\":true},{\"i64\":\"1048576\"},{\"s\":\"CURVE_FOURQ\"}]},\"sf_input_ids\":[\"alice-table\",\"bob-table\"],\"sf_output_ids\":[\"psi-output\"],\"sf_output_uris\":[\"psi-output.csv\"]}"},{"alias":"job-split","appImage":"secretflow-image","dependencies":["job-psi"],"parties":[{"domainID":"alice"},{"domainID":"bob"}],"priority":100,"taskID":"job-split","taskInputConfig":"{\"sf_datasource_config\":{\"alice\":{\"id\":\"default-data-source\"},\"bob\":{\"id\":\"default-data-source\"}},\"sf_cluster_desc\":{\"parties\":[\"alice\",\"bob\"],\"devices\":[{\"name\":\"spu\",\"type\":\"spu\",\"parties\":[\"alice\",\"bob\"],\"config\":\"{\\\"runtime_config\\\":{\\\"protocol\\\":\\\"REF2K\\\",\\\"field\\\":\\\"FM64\\\"},\\\"link_desc\\\":{\\\"connect_retry_times\\\":60,\\\"connect_retry_interval_ms\\\":1000,\\\"brpc_channel_protocol\\\":\\\"http\\\",\\\"brpc_channel_connection_type\\\":\\\"pooled\\\",\\\"recv_timeout_ms\\\":1200000,\\\"http_timeout_ms\\\":1200000}}\"},{\"name\":\"heu\",\"type\":\"heu\",\"parties\":[\"alice\",\"bob\"],\"config\":\"{\\\"mode\\\": \\\"PHEU\\\", \\\"schema\\\": \\\"paillier\\\", \\\"key_size\\\": 2048}\"}],\"ray_fed_config\":{\"cross_silo_comm_backend\":\"brpc_link\"}},\"sf_node_eval_param\":{\"domain\":\"data_prep\",\"name\":\"train_test_split\",\"version\":\"0.0.1\",\"attr_paths\":[\"train_size\",\"test_size\",\"random_state\",\"shuffle\"],\"attrs\":[{\"f\":0.75},{\"f\":0.25},{\"i64\":1234},{\"b\":true}]},\"sf_output_uris\":[\"train-dataset.csv\",\"test-dataset.csv\"],\"sf_output_ids\":[\"train-dataset\",\"test-dataset\"],\"sf_input_ids\":[\"psi-output\"]}"}]}}
    kuscia.secretflow/initiator: alice
    kuscia.secretflow/interconn-self-parties: alice_bob
    kuscia.secretflow/self-cluster-as-initiator: "true"
  creationTimestamp: "2024-05-08T06:10:05Z"
  generation: 1
  name: job-best-effort-linear
  namespace: cross-domain
  resourceVersion: "2301542"
  uid: 537749ff-5649-4088-a0e5-bbdb52ffef81
spec:
  initiator: alice
  maxParallelism: 2
  scheduleMode: BestEffort
  tasks:
  - alias: job-psi
    appImage: secretflow-image
    parties:
    - domainID: alice
    - domainID: bob
    priority: 100
    taskID: job-psi
    taskInputConfig: '{"sf_datasource_config":{"alice":{"id":"default-data-source"},"bob":{"id":"default-data-source"}},"sf_cluster_desc":{"parties":["alice","bob"],"devices":[{"name":"spu","type":"spu","parties":["alice","bob"],"config":"{\"runtime_config\":{\"protocol\":\"REF2K\",\"field\":\"FM64\"},\"link_desc\":{\"connect_retry_times\":60,\"connect_retry_interval_ms\":1000,\"brpc_channel_protocol\":\"http\",\"brpc_channel_connection_type\":\"pooled\",\"recv_timeout_ms\":1200000,\"http_timeout_ms\":1200000}}"},{"name":"heu","type":"heu","parties":["alice","bob"],"config":"{\"mode\":
      \"PHEU\", \"schema\": \"paillier\", \"key_size\": 2048}"}],"ray_fed_config":{"cross_silo_comm_backend":"brpc_link"}},"sf_node_eval_param":{"domain":"data_prep","name":"psi","version":"0.0.1","attr_paths":["input/receiver_input/key","input/sender_input/key","protocol","precheck_input","bucket_size","curve_type"],"attrs":[{"ss":["id1"]},{"ss":["id2"]},{"s":"ECDH_PSI_2PC"},{"b":true},{"i64":"1048576"},{"s":"CURVE_FOURQ"}]},"sf_input_ids":["alice-table","bob-table"],"sf_output_ids":["psi-output"],"sf_output_uris":["psi-output.csv"]}'
    tolerable: false
  - alias: job-split
    appImage: secretflow-image
    dependencies:
    - job-psi
    parties:
    - domainID: alice
    - domainID: bob
    priority: 100
    taskID: job-split
    taskInputConfig: '{"sf_datasource_config":{"alice":{"id":"default-data-source"},"bob":{"id":"default-data-source"}},"sf_cluster_desc":{"parties":["alice","bob"],"devices":[{"name":"spu","type":"spu","parties":["alice","bob"],"config":"{\"runtime_config\":{\"protocol\":\"REF2K\",\"field\":\"FM64\"},\"link_desc\":{\"connect_retry_times\":60,\"connect_retry_interval_ms\":1000,\"brpc_channel_protocol\":\"http\",\"brpc_channel_connection_type\":\"pooled\",\"recv_timeout_ms\":1200000,\"http_timeout_ms\":1200000}}"},{"name":"heu","type":"heu","parties":["alice","bob"],"config":"{\"mode\":
      \"PHEU\", \"schema\": \"paillier\", \"key_size\": 2048}"}],"ray_fed_config":{"cross_silo_comm_backend":"brpc_link"}},"sf_node_eval_param":{"domain":"data_prep","name":"train_test_split","version":"0.0.1","attr_paths":["train_size","test_size","random_state","shuffle"],"attrs":[{"f":0.75},{"f":0.25},{"i64":1234},{"b":true}]},"sf_output_uris":["train-dataset.csv","test-dataset.csv"],"sf_output_ids":["train-dataset","test-dataset"],"sf_input_ids":["psi-output"]}'
    tolerable: false
status:
  approveStatus:
    alice: JobAccepted
    bob: JobAccepted
  completionTime: "2024-05-08T06:15:18Z"
  conditions:
  - lastTransitionTime: "2024-05-08T06:10:05Z"
    status: "True"
    type: JobValidated
  lastReconcileTime: "2024-05-08T06:15:18Z"
  phase: Failed
  stageStatus:
    alice: JobCreateStageSucceeded
    bob: JobCreateStageSucceeded
  startTime: "2024-05-08T06:10:05Z"
  taskStatus:
    job-psi: Failed

3、报错日志如下： error.log

此时是否可以通过secretpad前端界面进行数据的授权?

zimu-yuxi commented 6 months ago

1.确认下数据授权是否做好 2.任务日志路径：/home/kuscia/var/stdout/pods/

Yanziwanglu commented 6 months ago

您好，在此处文档中部署中心化组网方式的，通过kubectl get kj -n cross-domain命令查看任务是失败的状态：在/home/kuscia/var/stdout/pods/下没有找到生成的日志文件，通过查看job状态，可以看到：详细错误信息： checkjob.json

yushiqie commented 6 months ago

请问你是通过什么样的方式安装的，使用的 kuscia 镜像是什么，方便看下 docker images 吗

Yanziwanglu commented 6 months ago

通过教程中的Kuscia安装脚本，执行的中心化部署部署方式。

yushiqie commented 6 months ago

是用 start_standalone.sh/deploy.sh 脚本部署的吗
可以进节点容器看下kuscia版本吗 docker exec -it wanglu-kuscia-master kuscia --version

Yanziwanglu commented 6 months ago

您好，使用的是./start_standalone.sh center 部署的，使用docker exec -it wanglu-kuscia-master kuscia --version命令查看的版本是kuscia version 578bc84。

yushiqie commented 6 months ago

start_standalone.sh 脚本部署确实存在这样的问题呢，这个问题我们将快速修复。您现在可以用我们稳定版本 0.8.0b0 ，使用 kuscia.sh 进行部署呢，这将是我们长期维护部署脚本。

Yanziwanglu commented 6 months ago

使用稳定版本 0.8.0b0进行部署，提示缺少secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/secretflow-lite-anolis8:1.6.0b0。

wenkesong-li commented 6 months ago

目前稳定版本是0.7.0b0。版本参考https://github.com/secretflow/secretpad

Yanziwanglu commented 6 months ago

目前稳定版本是0.7.0b0。版本参考https://github.com/secretflow/secretpad

之前部署的0.7.0b0版本的，在执行的过程中出现了上述任务执行失败问题。

yushiqie commented 6 months ago

secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/secretflow-lite-anolis8:1.6.0b0

从上面显示的 kuscia version，使用的不是 kuscia 0.7.0b0，而是latest 版本。可是通过指定kuscia 版本为 v0.7.0b0 镜像部署 export KUSCIA_IMAGE=secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/kuscia:0.7.0b0

Yanziwanglu commented 6 months ago

您好，目前的kuscia版本为kuscia:0.7.0b0：数据授权是成功状态的：其中job-best-effort-linear.yaml配置为：

apiVersion: kuscia.secretflow/v1alpha1
kind: KusciaJob
metadata:
  name: job-best-effort-linear
  namespace: cross-domain
spec:
  initiator: alice
  scheduleMode: BestEffort
  maxParallelism: 2
  tasks:
    - taskID: job-psi
      alias: job-psi
      priority: 100
      taskInputConfig: '{"sf_datasource_config":{"alice":{"id":"default-data-source"},"bob":{"id":"default-data-source"}},"sf_cluster_desc":{"parties":["alice","bob"],"devices":[{"name":"spu","type":"spu","parties":["alice","bob"],"config":"{\"runtime_config\":{\"protocol\":\"REF2K\",\"field\":\"FM64\"},\"link_desc\":{\"connect_retry_times\":60,\"connect_retry_interval_ms\":1000,\"brpc_channel_protocol\":\"http\",\"brpc_channel_connection_type\":\"pooled\",\"recv_timeout_ms\":1200000,\"http_timeout_ms\":1200000}}"},{"name":"heu","type":"heu","parties":["alice","bob"],"config":"{\"mode\": \"PHEU\", \"schema\": \"paillier\", \"key_size\": 2048}"}],"ray_fed_config":{"cross_silo_comm_backend":"brpc_link"}},"sf_node_eval_param":{"domain":"data_prep","name":"psi","version":"0.0.1","attr_paths":["input/receiver_input/key","input/sender_input/key","protocol","precheck_input","bucket_size","curve_type"],"attrs":[{"ss":["id1"]},{"ss":["id2"]},{"s":"ECDH_PSI_2PC"},{"b":true},{"i64":"1048576"},{"s":"CURVE_FOURQ"}]},"sf_input_ids":["alice-001","bob-001"],"sf_output_ids":["psi-output"],"sf_output_uris":["psi-output.csv"]}'
      appImage: secretflow-image
      parties:
        - domainID: alice
        - domainID: bob
    - taskID: job-split
      alias: job-split
      priority: 100
      dependencies: ['job-psi']
      taskInputConfig: '{"sf_datasource_config":{"alice":{"id":"default-data-source"},"bob":{"id":"default-data-source"}},"sf_cluster_desc":{"parties":["alice","bob"],"devices":[{"name":"spu","type":"spu","parties":["alice","bob"],"config":"{\"runtime_config\":{\"protocol\":\"REF2K\",\"field\":\"FM64\"},\"link_desc\":{\"connect_retry_times\":60,\"connect_retry_interval_ms\":1000,\"brpc_channel_protocol\":\"http\",\"brpc_channel_connection_type\":\"pooled\",\"recv_timeout_ms\":1200000,\"http_timeout_ms\":1200000}}"},{"name":"heu","type":"heu","parties":["alice","bob"],"config":"{\"mode\": \"PHEU\", \"schema\": \"paillier\", \"key_size\": 2048}"}],"ray_fed_config":{"cross_silo_comm_backend":"brpc_link"}},"sf_node_eval_param":{"domain":"data_prep","name":"train_test_split","version":"0.0.1","attr_paths":["train_size","test_size","random_state","shuffle"],"attrs":[{"f":0.75},{"f":0.25},{"i64":1234},{"b":true}]},"sf_output_uris":["train-dataset.csv","test-dataset.csv"],"sf_output_ids":["train-dataset","test-dataset"],"sf_input_ids":["psi-out001}'
      appImage: secretflow-image
      parties:
        - domainID: alice
        - domainID: bob
~

在 kuscia-master 容器中执行 kubectl apply -f job-best-effort-linear.yaml命令，查看作业状态是失败状态，报错日志如下：

{
    "status": {
        "code": 0,
        "message": "success",
        "details": []
    },
    "data": {
        "job_id": "job-best-effort-linear",
        "initiator": "alice",
        "max_parallelism": 2,
        "tasks": [
            {
                "app_image": "secretflow-image",
                "parties": [
                    {
                        "domain_id": "alice",
                        "role": ""
                    },
                    {
                        "domain_id": "bob",
                        "role": ""
                    }
                ],
                "alias": "job-psi",
                "task_id": "job-psi",
                "dependencies": [],
                "task_input_config": "{\"sf_datasource_config\":{\"alice\":{\"id\":\"default-data-source\"},\"bob\":{\"id\":\"default-data-source\"}},\"sf_cluster_desc\":{\"parties\":[\"alice\",\"bob\"],\"devices\":[{\"name\":\"spu\",\"type\":\"spu\",\"parties\":[\"alice\",\"bob\"],\"config\":\"{\\\"runtime_config\\\":{\\\"protocol\\\":\\\"REF2K\\\",\\\"field\\\":\\\"FM64\\\"},\\\"link_desc\\\":{\\\"connect_retry_times\\\":60,\\\"connect_retry_interval_ms\\\":1000,\\\"brpc_channel_protocol\\\":\\\"http\\\",\\\"brpc_channel_connection_type\\\":\\\"pooled\\\",\\\"recv_timeout_ms\\\":1200000,\\\"http_timeout_ms\\\":1200000}}\"},{\"name\":\"heu\",\"type\":\"heu\",\"parties\":[\"alice\",\"bob\"],\"config\":\"{\\\"mode\\\": \\\"PHEU\\\", \\\"schema\\\": \\\"paillier\\\", \\\"key_size\\\": 2048}\"}],\"ray_fed_config\":{\"cross_silo_comm_backend\":\"brpc_link\"}},\"sf_node_eval_param\":{\"domain\":\"data_prep\",\"name\":\"psi\",\"version\":\"0.0.1\",\"attr_paths\":[\"input/receiver_input/key\",\"input/sender_input/key\",\"protocol\",\"precheck_input\",\"bucket_size\",\"curve_type\"],\"attrs\":[{\"ss\":[\"id1\"]},{\"ss\":[\"id2\"]},{\"s\":\"ECDH_PSI_2PC\"},{\"b\":true},{\"i64\":\"1048576\"},{\"s\":\"CURVE_FOURQ\"}]},\"sf_input_ids\":[\"alice-001\",\"bob-001\"],\"sf_output_ids\":[\"psi-output\"],\"sf_output_uris\":[\"psi-output.csv\"]}",
                "priority": 100
            },
            {
                "app_image": "secretflow-image",
                "parties": [
                    {
                        "domain_id": "alice",
                        "role": ""
                    },
                    {
                        "domain_id": "bob",
                        "role": ""
                    }
                ],
                "alias": "job-split",
                "task_id": "job-split",
                "dependencies": [
                    "job-psi"
                ],
                "task_input_config": "{\"sf_datasource_config\":{\"alice\":{\"id\":\"default-data-source\"},\"bob\":{\"id\":\"default-data-source\"}},\"sf_cluster_desc\":{\"parties\":[\"alice\",\"bob\"],\"devices\":[{\"name\":\"spu\",\"type\":\"spu\",\"parties\":[\"alice\",\"bob\"],\"config\":\"{\\\"runtime_config\\\":{\\\"protocol\\\":\\\"REF2K\\\",\\\"field\\\":\\\"FM64\\\"},\\\"link_desc\\\":{\\\"connect_retry_times\\\":60,\\\"connect_retry_interval_ms\\\":1000,\\\"brpc_channel_protocol\\\":\\\"http\\\",\\\"brpc_channel_connection_type\\\":\\\"pooled\\\",\\\"recv_timeout_ms\\\":1200000,\\\"http_timeout_ms\\\":1200000}}\"},{\"name\":\"heu\",\"type\":\"heu\",\"parties\":[\"alice\",\"bob\"],\"config\":\"{\\\"mode\\\": \\\"PHEU\\\", \\\"schema\\\": \\\"paillier\\\", \\\"key_size\\\": 2048}\"}],\"ray_fed_config\":{\"cross_silo_comm_backend\":\"brpc_link\"}},\"sf_node_eval_param\":{\"domain\":\"data_prep\",\"name\":\"train_test_split\",\"version\":\"0.0.1\",\"attr_paths\":[\"train_size\",\"test_size\",\"random_state\",\"shuffle\"],\"attrs\":[{\"f\":0.75},{\"f\":0.25},{\"i64\":1234},{\"b\":true}]},\"sf_output_uris\":[\"train-dataset.csv\",\"test-dataset.csv\"],\"sf_output_ids\":[\"train-dataset\",\"test-dataset\"],\"sf_input_ids\":[\"psi-out001}",
                "priority": 100
            }
        ],
        "status": {
            "state": "Failed",
            "err_msg": "",
            "create_time": "2024-05-20T09:05:56Z",
            "start_time": "2024-05-20T09:05:56Z",
            "end_time": "2024-05-20T09:06:13Z",
            "tasks": [
                {
                    "task_id": "job-psi",
                    "state": "Failed",
                    "err_msg": "The remaining no-failed party task counts 1 are less than the threshold 2 that meets the conditions for task success. pending party[], running party[alice], successful party[], failed party[bob]",
                    "create_time": "2024-05-20T09:05:57Z",
                    "start_time": "2024-05-20T09:05:57Z",
                    "end_time": "2024-05-20T09:06:13Z",
                    "parties": [
                        {
                            "domain_id": "alice",
                            "state": "Failed",
                            "err_msg": "",
                            "endpoints": [
                                {
                                    "port_name": "fed",
                                    "scope": "Cluster",
                                    "endpoint": "job-psi-0-fed.alice.svc"
                                },
                                {
                                    "port_name": "global",
                                    "scope": "Domain",
                                    "endpoint": "job-psi-0-global.alice.svc:23999"
                                },
                                {
                                    "port_name": "spu",
                                    "scope": "Cluster",
                                    "endpoint": "job-psi-0-spu.alice.svc"
                                }
                            ]
                        },
                        {
                            "domain_id": "bob",
                            "state": "Failed",
                            "err_msg": "container[secretflow] terminated state reason \"Error\", message: \"/logs --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --session-name=session_2024-05-20_09-06-03_969108_163 --gcs-address=job-psi-0-global.bob.svc:30797 --minimal\\\" --node-name=job-psi-0-global.bob.svc` (via SIGTERM)\\n2024-05-20 09:06:06,618\\tVINFO scripts.py:1023 -- Send termination request to `/usr/local/lib/python3.8/site-packages/ray/core/src/ray/raylet/raylet --raylet_socket_name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/raylet --store_socket_name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/plasma_store --object_manager_port=30793 --min_worker_port=10002 --max_worker_port=19999 --node_manager_port=30798 --node_ip_address=job-psi-0-global.bob.svc --maximum_startup_concurrency=32 --static_resource_list=node:job-psi-0-global.bob.svc,1.0,CPU,32,memory,520120907776,object_store_memory,10000000000 \\\"--python_worker_command=/usr/local/bin/python /usr/local/lib/python3.8/site-packages/ray/_private/workers/setup_worker.py /usr/local/lib/python3.8/site-packages/ray/_private/workers/default_worker.py --node-ip-address=job-psi-0-global.bob.svc --node-manager-port=RAY_NODE_MANAGER_PORT_PLACEHOLDER --object-store-name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/plasma_store --raylet-name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/raylet --redis-address=None --storage=None --temp-dir=/tmp/ray --metrics-agent-port=54526 --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --gcs-address=job-psi-0-global.bob.svc:30797 RAY_WORKER_DYNAMIC_OPTION_PLACEHOLDER --redis-password=5241590000000000\\\" --java_worker_command= --cpp_worker_command= --native_library_path=/usr/local/lib/python3.8/site-packages/ray/cpp/lib --temp_dir=/tmp/ray --session_dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163 --log_dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/logs --resource_dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/runtime_resources --metrics-agent-port=54526 --metrics_export_port=44609 --object_store_memory=10000000000 --plasma_directory=/tmp --ray-debugger-external=0 --gcs-address=job-psi-0-global.bob.svc:30797 --session-name=session_2024-05-20_09-06-03_969108_163 \\\"--agent_command=/usr/local/bin/python -u /usr/local/lib/python3.8/site-packages/ray/dashboard/agent.py --node-ip-address=job-psi-0-global.bob.svc --metrics-export-port=44609 --dashboard-agent-port=54526 --listen-port=52365 --node-manager-port=RAY_NODE_MANAGER_PORT_PLACEHOLDER --object-store-name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/plasma_store --raylet-name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/raylet --temp-dir=/tmp/ray --session-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163 --runtime-env-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/runtime_resources --log-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/logs --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --session-name=session_2024-05-20_09-06-03_969108_163 --gcs-address=job-psi-0-global.bob.svc:30797 --minimal\\\" --node-name=job-psi-0-global.bob.svc` (via SIGTERM)\\n2024-05-20 09:06:06,621\\tVINFO scripts.py:1023 -- Send termination request to `/usr/local/bin/python -u /usr/local/lib/python3.8/site-packages/ray/_private/log_monitor.py --logs-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/logs --gcs-address=job-psi-0-global.bob.svc:30797 --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5` (via SIGTERM)\\n2024-05-20 09:06:06,623\\tVINFO scripts.py:1023 -- Send termination request to `/usr/local/lib/python3.8/site-packages/ray/core/src/ray/raylet/raylet --raylet_socket_name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/raylet --store_socket_name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/plasma_store --object_manager_port=30793 --min_worker_port=10002 --max_worker_port=19999 --node_manager_port=30798 --node_ip_address=job-psi-0-global.bob.svc --maximum_startup_concurrency=32 --static_resource_list=node:job-psi-0-global.bob.svc,1.0,CPU,32,memory,520120907776,object_store_memory,10000000000 \\\"--python_worker_command=/usr/local/bin/python /usr/local/lib/python3.8/site-packages/ray/_private/workers/setup_worker.py /usr/local/lib/python3.8/site-packages/ray/_private/workers/default_worker.py --node-ip-address=job-psi-0-global.bob.svc --node-manager-port=RAY_NODE_MANAGER_PORT_PLACEHOLDER --object-store-name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/plasma_store --raylet-name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/raylet --redis-address=None --storage=None --temp-dir=/tmp/ray --metrics-agent-port=54526 --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --gcs-address=job-psi-0-global.bob.svc:30797 RAY_WORKER_DYNAMIC_OPTION_PLACEHOLDER --redis-password=5241590000000000\\\" --java_worker_command= --cpp_worker_command= --native_library_path=/usr/local/lib/python3.8/site-packages/ray/cpp/lib --temp_dir=/tmp/ray --session_dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163 --log_dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/logs --resource_dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/runtime_resources --metrics-agent-port=54526 --metrics_export_port=44609 --object_store_memory=10000000000 --plasma_directory=/tmp --ray-debugger-external=0 --gcs-address=job-psi-0-global.bob.svc:30797 --session-name=session_2024-05-20_09-06-03_969108_163 \\\"--agent_command=/usr/local/bin/python -u /usr/local/lib/python3.8/site-packages/ray/dashboard/agent.py --node-ip-address=job-psi-0-global.bob.svc --metrics-export-port=44609 --dashboard-agent-port=54526 --listen-port=52365 --node-manager-port=RAY_NODE_MANAGER_PORT_PLACEHOLDER --object-store-name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/plasma_store --raylet-name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/raylet --temp-dir=/tmp/ray --session-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163 --runtime-env-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/runtime_resources --log-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/logs --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --session-name=session_2024-05-20_09-06-03_969108_163 --gcs-address=job-psi-0-global.bob.svc:30797 --minimal\\\" --node-name=job-psi-0-global.bob.svc` (via SIGTERM)\\n2024-05-20 09:06:06,623\\tVINFO scripts.py:1023 -- Send termination request to `/usr/local/bin/python -u /usr/local/lib/python3.8/site-packages/ray/dashboard/agent.py --node-ip-address=job-psi-0-global.bob.svc --metrics-export-port=44609 --dashboard-agent-port=54526 --listen-port=52365 --node-manager-port=30798 --object-store-name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/plasma_store --raylet-name=/tmp/ray/session_2024-05-20_09-06-03_969108_163/sockets/raylet --temp-dir=/tmp/ray --session-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163 --runtime-env-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/runtime_resources --log-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/logs --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --session-name=session_2024-05-20_09-06-03_969108_163 --gcs-address=job-psi-0-global.bob.svc:30797 --minimal --agent-id 1059961393` (via SIGTERM)\\n2024-05-20 09:06:06,624\\tVINFO scripts.py:1023 -- Send termination request to `/usr/local/bin/python /usr/local/lib/python3.8/site-packages/ray/dashboard/dashboard.py --host=localhost --port=8265 --port-retries=0 --temp-dir=/tmp/ray --log-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163/logs --session-dir=/tmp/ray/session_2024-05-20_09-06-03_969108_163 --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --gcs-address=job-psi-0-global.bob.svc:30797 --minimal --modules-to-load=UsageStatsHead` (via SIGTERM)\\n2024-05-20 09:06:06,647\\tINFO scripts.py:1051 -- 1/7 stopped.\\r2024-05-20 09:06:06,819\\tINFO scripts.py:1051 -- 2/7 stopped.\\r2024-05-20 09:06:06,819\\tINFO scripts.py:1051 -- 3/7 stopped.\\r2024-05-20 09:06:06,819\\tINFO scripts.py:1051 -- 4/7 stopped.\\r2024-05-20 09:06:06,819\\tINFO scripts.py:1051 -- 5/7 stopped.\\r2024-05-20 09:06:06,991\\tINFO scripts.py:1051 -- 6/7 stopped.\\r2024-05-20 09:06:12,272\\tINFO scripts.py:1051 -- 7/7 stopped.\\r2024-05-20 09:06:12,273\\tSUCC scripts.py:1063 -- Stopped all 7 Ray processes.\\n\"",
                            "endpoints": [
                                {
                                    "port_name": "spu",
                                    "scope": "Cluster",
                                    "endpoint": "job-psi-0-spu.bob.svc"
                                },
                                {
                                    "port_name": "fed",
                                    "scope": "Cluster",
                                    "endpoint": "job-psi-0-fed.bob.svc"
                                },
                                {
                                    "port_name": "global",
                                    "scope": "Domain",
                                    "endpoint": "job-psi-0-global.bob.svc:30797"
                                }
                            ]
                        }
                    ]
                },
                {
                    "task_id": "job-split",
                    "state": "Pending",
                    "err_msg": "",
                    "create_time": "",
                    "start_time": "",
                    "end_time": "",
                    "parties": []
                }
            ],
            "stage_status_list": [
                {
                    "domain_id": "alice",
                    "state": "JobCreateStageSucceeded"
                },
                {
                    "domain_id": "bob",
                    "state": "JobCreateStageSucceeded"
                }
            ],
            "approve_status_list": [
                {
                    "domain_id": "alice",
                    "state": "JobAccepted"
                },
                {
                    "domain_id": "bob",
                    "state": "JobAccepted"
                }
            ]
        },
        "custom_fields": {}
    }
}

wenkesong-li commented 6 months ago

你好，请检查一下组件版本~ 我看一下你给出的psi版本是 0.0.1 kuscia对应的其他模块的版本号参考：

https://github.com/secretflow/secretpad

Yanziwanglu commented 6 months ago

修改psi的版本为0.0.4，执行状态仍然是失败的状态。

apiVersion: kuscia.secretflow/v1alpha1
kind: KusciaJob
metadata:
  name: job-best-effort-linear
  namespace: cross-domain
spec:
  initiator: alice
  scheduleMode: BestEffort
  maxParallelism: 2
  tasks:
    - taskID: job-psi
      alias: job-psi
      priority: 100
      taskInputConfig: '{"sf_datasource_config":{"alice":{"id":"default-data-source"},"bob":{"id":"default-data-source"}},"sf_cluster_desc":{"parties":["alice","bob"],"devices":[{"name":"spu","type":"spu","parties":["alice","bob"],"config":"{\"runtime_config\":{\"protocol\":\"REF2K\",\"field\":\"FM64\"},\"link_desc\":{\"connect_retry_times\":60,\"connect_retry_interval_ms\":1000,\"brpc_channel_protocol\":\"http\",\"brpc_channel_connection_type\":\"pooled\",\"recv_timeout_ms\":1200000,\"http_timeout_ms\":1200000}}"},{"name":"heu","type":"heu","parties":["alice","bob"],"config":"{\"mode\": \"PHEU\", \"schema\": \"paillier\", \"key_size\": 2048}"}],"ray_fed_config":{"cross_silo_comm_backend":"brpc_link"}},"sf_node_eval_param":{"domain":"data_prep","name":"psi","version":"0.0.4","attr_paths":["input/receiver_input/key","input/sender_input/key","protocol","precheck_input","bucket_size","curve_type"],"attrs":[{"ss":["id1"]},{"ss":["id2"]},{"s":"ECDH_PSI_2PC"},{"b":true},{"i64":"1048576"},{"s":"CURVE_FOURQ"}]},"sf_input_ids":["alice-001","bob-001"],"sf_output_ids":["psi-output"],"sf_output_uris":["psi-output.csv"]}'
      appImage: secretflow-image
      parties:
        - domainID: alice
        - domainID: bob
    - taskID: job-split
      alias: job-split
      priority: 100
      dependencies: ['job-psi']
      taskInputConfig: '{"sf_datasource_config":{"alice":{"id":"default-data-source"},"bob":{"id":"default-data-source"}},"sf_cluster_desc":{"parties":["alice","bob"],"devices":[{"name":"spu","type":"spu","parties":["alice","bob"],"config":"{\"runtime_config\":{\"protocol\":\"REF2K\",\"field\":\"FM64\"},\"link_desc\":{\"connect_retry_times\":60,\"connect_retry_interval_ms\":1000,\"brpc_channel_protocol\":\"http\",\"brpc_channel_connection_type\":\"pooled\",\"recv_timeout_ms\":1200000,\"http_timeout_ms\":1200000}}"},{"name":"heu","type":"heu","parties":["alice","bob"],"config":"{\"mode\": \"PHEU\", \"schema\": \"paillier\", \"key_size\": 2048}"}],"ray_fed_config":{"cross_silo_comm_backend":"brpc_link"}},"sf_node_eval_param":{"domain":"data_prep","name":"train_test_split","version":"0.0.1","attr_paths":["train_size","test_size","random_state","shuffle"],"attrs":[{"f":0.75},{"f":0.25},{"i64":1234},{"b":true}]},"sf_output_uris":["train-dataset.csv","test-dataset.csv"],"sf_output_ids":["train-dataset","test-dataset"],"sf_input_ids":["psi-out001}'
      appImage: secretflow-image
      parties:
        - domainID: alice
        - domainID: bob

wenkesong-li commented 6 months ago

你好，麻烦检查一下，其他模块是否是最新的~ 因为我看你之前的包好像都不是新的，建议重新拉取最新的镜像和包再试试。

Yanziwanglu commented 6 months ago

您好，我拉取的是kuscia:0.7.0b0版本的，不是很清楚您说的之前的包是指哪个。现在是拉取kuscia:0.8.0b0版本的吗？

wenkesong-li commented 6 months ago

比如sf和其他依赖，如果是mvp安装的，可以重新下载一下mvp重新试一下。

Yanziwanglu commented 6 months ago

您好，修改start_standalone.sh里SF_IMAGE_TAG参数为"1.5.0b0"，重新部署了一下，还是failed状态。请问是还需要修改哪些参数吗？官方是否方便提供一键部署的脚本呢

wenkesong-li commented 6 months ago

猜测是您之前下载的版本太旧，导致组件版本不兼容，建议删除后，重新下载新版本使用~

https://www.secretflow.org.cn/zh-CN/docs/secretpad-all-in-one/latest/mvp-platform

Yanziwanglu commented 6 months ago

猜测是您之前下载的版本太旧，导致组件版本不兼容，建议删除后，重新下载新版本使用~

https://www.secretflow.org.cn/zh-CN/docs/secretpad-all-in-one/latest/mvp-platform

您好，我按照这个链接下载了当前版本进行了部署，执行结果依旧失败。

zimu-yuxi commented 6 months ago

找到当前任务名称的文件夹，贴一下里边的日志路径参考：cd home/kuscia/var/stdout/pods/

Yanziwanglu commented 6 months ago

找到当前任务名称的文件夹，贴一下里边的日志路径参考：cd home/kuscia/var/stdout/pods/

您好，日志如下：

[root@root-kuscia-lite-alice kuscia]# cat var/stdout/pods/alice_job-psi-0_185f945a-51e8-44ef-9e63-cfb341a82dc3/secretflow/0.log 
2024-05-21T13:45:50.742165214+08:00 stderr F WARNING:root:Since the GPL-licensed package `unidecode` is not installed, using Python's `unicodedata` package which yields worse results.
2024-05-21T13:45:54.17022884+08:00 stdout F 2024-05-21 05:45:54,169|alice|INFO|secretflow|entry.py:start_ray:59| ray_conf: RayConfig(ray_node_ip_address='job-psi-0-global.alice.svc', ray_node_manager_port=24034, ray_object_manager_port=24035, ray_client_server_port=24036, ray_worker_ports=[], ray_gcs_port=24039)
2024-05-21T13:45:54.170376368+08:00 stdout F 2024-05-21 05:45:54,170|alice|INFO|secretflow|entry.py:start_ray:63| Trying to start ray head node at job-psi-0-global.alice.svc, start command: RAY_BACKEND_LOG_LEVEL=debug RAY_grpc_enable_http_proxy=true OMP_NUM_THREADS=112 ray start --head --include-dashboard=false --disable-usage-stats --num-cpus=32 --node-ip-address=job-psi-0-global.alice.svc --port=24039 --node-manager-port=24034 --object-manager-port=24035 --ray-client-server-port=24036
2024-05-21T13:45:57.773848705+08:00 stdout F 2024-05-21 05:45:57,773|alice|INFO|secretflow|entry.py:start_ray:80| 2024-05-21 05:45:55,718   INFO usage_lib.py:423 -- Usage stats collection is disabled.
2024-05-21T13:45:57.773866604+08:00 stdout F 2024-05-21 05:45:55,718    INFO scripts.py:744 -- Local node IP: job-psi-0-global.alice.svc
2024-05-21T13:45:57.773870097+08:00 stdout F 2024-05-21 05:45:57,632    SUCC scripts.py:781 -- --------------------
2024-05-21T13:45:57.773872959+08:00 stdout F 2024-05-21 05:45:57,632    SUCC scripts.py:782 -- Ray runtime started.
2024-05-21T13:45:57.773876139+08:00 stdout F 2024-05-21 05:45:57,632    SUCC scripts.py:783 -- --------------------
2024-05-21T13:45:57.773879125+08:00 stdout F 2024-05-21 05:45:57,632    INFO scripts.py:785 -- Next steps
2024-05-21T13:45:57.773882058+08:00 stdout F 2024-05-21 05:45:57,633    INFO scripts.py:788 -- To add another node to this Ray cluster, run
2024-05-21T13:45:57.773885267+08:00 stdout F 2024-05-21 05:45:57,633    INFO scripts.py:791 --   ray start --address='job-psi-0-global.alice.svc:24039'
2024-05-21T13:45:57.773888379+08:00 stdout F 2024-05-21 05:45:57,633    INFO scripts.py:800 -- To connect to this Ray cluster:
2024-05-21T13:45:57.77389325+08:00 stdout F 2024-05-21 05:45:57,633 INFO scripts.py:802 -- import ray
2024-05-21T13:45:57.773897746+08:00 stdout F 2024-05-21 05:45:57,633    INFO scripts.py:803 -- ray.init(_node_ip_address='job-psi-0-global.alice.svc')
2024-05-21T13:45:57.773902043+08:00 stdout F 2024-05-21 05:45:57,633    INFO scripts.py:834 -- To terminate the Ray runtime, run
2024-05-21T13:45:57.77390498+08:00 stdout F 2024-05-21 05:45:57,633 INFO scripts.py:835 --   ray stop
2024-05-21T13:45:57.773907662+08:00 stdout F 2024-05-21 05:45:57,633    INFO scripts.py:838 -- To view the status of the cluster, use
2024-05-21T13:45:57.773910409+08:00 stdout F 2024-05-21 05:45:57,633    INFO scripts.py:839 --   ray status
2024-05-21T13:45:57.773912981+08:00 stdout F 
2024-05-21T13:45:57.77392544+08:00 stdout F 2024-05-21 05:45:57,773|alice|INFO|secretflow|entry.py:start_ray:81| Succeeded to start ray head node at job-psi-0-global.alice.svc.
2024-05-21T13:45:57.77468953+08:00 stdout F 2024-05-21 05:45:57,774|alice|INFO|secretflow|entry.py:main:510| datasource.access_directly True
2024-05-21T13:45:57.774709331+08:00 stdout F sf_node_eval_param  {
2024-05-21T13:45:57.774716604+08:00 stdout F   "domain": "data_prep",
2024-05-21T13:45:57.77472177+08:00 stdout F   "name": "psi",
2024-05-21T13:45:57.774726882+08:00 stdout F   "version": "0.0.4",
2024-05-21T13:45:57.774731654+08:00 stdout F   "attrPaths": [
2024-05-21T13:45:57.774736916+08:00 stdout F     "input/receiver_input/key",
2024-05-21T13:45:57.774741612+08:00 stdout F     "input/sender_input/key",
2024-05-21T13:45:57.774766455+08:00 stdout F     "protocol",
2024-05-21T13:45:57.774782082+08:00 stdout F     "precheck_input",
2024-05-21T13:45:57.774787094+08:00 stdout F     "bucket_size",
2024-05-21T13:45:57.774791836+08:00 stdout F     "curve_type"
2024-05-21T13:45:57.774796922+08:00 stdout F   ],
2024-05-21T13:45:57.774801919+08:00 stdout F   "attrs": [
2024-05-21T13:45:57.774806681+08:00 stdout F     {
2024-05-21T13:45:57.774826298+08:00 stdout F       "ss": [
2024-05-21T13:45:57.77483179+08:00 stdout F         "id1"
2024-05-21T13:45:57.774836807+08:00 stdout F       ]
2024-05-21T13:45:57.774841639+08:00 stdout F     },
2024-05-21T13:45:57.77484645+08:00 stdout F     {
2024-05-21T13:45:57.774851312+08:00 stdout F       "ss": [
2024-05-21T13:45:57.774855982+08:00 stdout F         "id2"
2024-05-21T13:45:57.774860583+08:00 stdout F       ]
2024-05-21T13:45:57.774865309+08:00 stdout F     },
2024-05-21T13:45:57.77486993+08:00 stdout F     {
2024-05-21T13:45:57.774874806+08:00 stdout F       "s": "ECDH_PSI_2PC"
2024-05-21T13:45:57.774880164+08:00 stdout F     },
2024-05-21T13:45:57.774884775+08:00 stdout F     {
2024-05-21T13:45:57.774889546+08:00 stdout F       "b": true
2024-05-21T13:45:57.774894297+08:00 stdout F     },
2024-05-21T13:45:57.774898978+08:00 stdout F     {
2024-05-21T13:45:57.7749036+08:00 stdout F       "i64": "1048576"
2024-05-21T13:45:57.774908445+08:00 stdout F     },
2024-05-21T13:45:57.774913227+08:00 stdout F     {
2024-05-21T13:45:57.774917904+08:00 stdout F       "s": "CURVE_FOURQ"
2024-05-21T13:45:57.7749228+08:00 stdout F     }
2024-05-21T13:45:57.774927762+08:00 stdout F   ]
2024-05-21T13:45:57.774932507+08:00 stdout F } 
2024-05-21T13:45:57.783686505+08:00 stderr F Traceback (most recent call last):
2024-05-21T13:45:57.78369926+08:00 stderr F   File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
2024-05-21T13:45:57.804290756+08:00 stderr F     return _run_code(code, main_globals, None,
2024-05-21T13:45:57.804294985+08:00 stderr F   File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
2024-05-21T13:45:57.804358869+08:00 stderr F     exec(code, run_globals)
2024-05-21T13:45:57.804361382+08:00 stderr F   File "/usr/local/lib/python3.10/site-packages/secretflow/kuscia/entry.py", line 547, in <module>
2024-05-21T13:45:57.804509334+08:00 stderr F     main()
2024-05-21T13:45:57.80451223+08:00 stderr F   File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
2024-05-21T13:45:57.845371791+08:00 stderr F     return self.main(*args, **kwargs)
2024-05-21T13:45:57.845376758+08:00 stderr F   File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1078, in main
2024-05-21T13:45:57.845603739+08:00 stderr F     rv = self.invoke(ctx)
2024-05-21T13:45:57.845605845+08:00 stderr F   File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
2024-05-21T13:45:57.845870104+08:00 stderr F     return ctx.invoke(self.callback, **ctx.params)
2024-05-21T13:45:57.845872481+08:00 stderr F   File "/usr/local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
2024-05-21T13:45:57.846022286+08:00 stderr F     return __callback(*args, **kwargs)
2024-05-21T13:45:57.846024186+08:00 stderr F   File "/usr/local/lib/python3.10/site-packages/secretflow/kuscia/entry.py", line 514, in main
2024-05-21T13:45:57.84614718+08:00 stderr F     sf_node_eval_param = preprocess_sf_node_eval_param(
2024-05-21T13:45:57.84614967+08:00 stderr F   File "/usr/local/lib/python3.10/site-packages/secretflow/kuscia/entry.py", line 290, in preprocess_sf_node_eval_param
2024-05-21T13:45:57.846251638+08:00 stderr F     domaindata_id_to_dist_data(
2024-05-21T13:45:57.846254227+08:00 stderr F   File "/usr/local/lib/python3.10/site-packages/secretflow/kuscia/entry.py", line 149, in domaindata_id_to_dist_data
2024-05-21T13:45:57.846259651+08:00 stderr F     raise RuntimeError(
2024-05-21T13:45:57.846305771+08:00 stderr F RuntimeError: datasource_id of domain_data [alice-001] is test-alice-datasource-id, which doesn't match global datasource_id default-data-source

gshilei commented 6 months ago

Hi @Yanziwanglu, 你可以参考v0.8.0b0分支文档进行相关操作.

以下是文档中快速体验的教程:

1. 指定Kuscia_Image
export KUSCIA_IMAGE=secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/kuscia:0.8.0b0
2. 准备部署脚本kuscia.sh
docker pull $KUSCIA_IMAGE && docker run --rm $KUSCIA_IMAGE cat /home/kuscia/scripts/deploy/kuscia.sh > kuscia.sh && chmod u+x kuscia.sh
3. 安装kuscia
./kuscia.sh center
4. 创建并启动作业（两方 PSI 任务）。
docker exec -it ${USER}-kuscia-master scripts/user/create_example_job.sh
5. 查看作业状态。
docker exec -it ${USER}-kuscia-master kubectl get kj -n cross-domain

使用该分支文档中的教程时，请保证kuscia镜像的版本为secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/kuscia:0.8.0b0

Yanziwanglu commented 6 months ago

Hi @Yanziwanglu, 你可以参考v0.8.0b0分支文档进行相关操作.

以下是文档中快速体验的教程:

1. 指定Kuscia_Image
export KUSCIA_IMAGE=secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/kuscia:0.8.0b0
2. 准备部署脚本kuscia.sh
docker pull $KUSCIA_IMAGE && docker run --rm $KUSCIA_IMAGE cat /home/kuscia/scripts/deploy/kuscia.sh > kuscia.sh && chmod u+x kuscia.sh
3. 安装kuscia
./kuscia.sh center
4. 创建并启动作业（两方 PSI 任务）。
docker exec -it ${USER}-kuscia-master scripts/user/create_example_job.sh
5. 查看作业状态。
docker exec -it ${USER}-kuscia-master kubectl get kj -n cross-domain

使用该分支文档中的教程时，请保证kuscia镜像的版本为secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/kuscia:0.8.0b0

您好，这个教程已经跑过了，作业状态时成功的。当前出现的问题目前推测是taskInputConfig里面算子参数设置的问题，在sf_node_eval_param里面需要修改相关参数。

gshilei commented 6 months ago

@Yanziwanglu 可能是的。因为secretflow还处于不断迭代的过程，所以组件的参数和版本在不同的secretflow版本之间存在差异。因此，针对这种情况，我们从 v0.8.0b0 版本开始，会逐渐对 Kuscia 的版本和 secretflow 版本进行管理。直接体现是正式分支的 v0.8.0b0 文档中，我们使用的是具体的 Kuscia镜像版本，而不是latest版本。

secretflow / kuscia

在kuscia-master容器执行kubectl apply -f job-best-effort-linear.yaml命令，无法看到kuscia job执行状态 #297