search

secretflow / kuscia

Kuscia(Kubernetes-based Secure Collaborative InfrA) is a K8s-based privacy-preserving computing task orchestration framework.
https://www.secretflow.org.cn/docs/kuscia/latest/zh-Hans
Apache License 2.0
73 stars 56 forks source link

导出模型异常 #397

Closed 14ctt closed 3 months ago

14ctt commented 3 months ago

Issue Type

Others

Search for existing issues similar to yours

Yes

Kuscia Version

kuscia 0.10.0b0

Link to Relevant Documentation

https://github.com/secretflow/secretflow/issues/1420

Question Details

遇到的问题:
使用model_export导出模型遇到以下异常。提供的异常信息和提供的配置参数可能不匹配，但是异常时一样。
预期行为:预期导出模型tar压缩包。
实际行为：报异常，使用通义分析结果：从提供的错误信息来看，问题出现在 get_init_pyus 函数中。该函数期望只有一个垂直表（v_table）作为输入，但是似乎没有找到这样的输入或者找到了多个这样的输入。这违反了组件的预期行为，即只支持一个垂直表作为输入。
尝试过的解决方案：
1.修改ml-train-ss-xgb-train组件的特征列参数，只选择一个特征列。因为怀疑有多少特征列就产生多少个文件。
2.ml-train-ss-xgb-train的sf_output_uri参数加上 “.tar”文件后缀，怀疑没有配置文件后缀导致产生多个文件。
异常信息：
container[secretflow] terminated state reason "Error", message: "n},\\n{\\n\\\"type\\\": \\\"pyu\\\",\\n\\\"dataRefIdxs\\\": [\\n29\\n]\\n}\\n],\\n\\\"publicInfo\\\": \\\"{\\\\\\\"major_version\\\\\\\": 0, \\\\\\\"minor_version\\\\\\\": 1, \\\\\\\"public_info\\\\\\\": \\\\\\\"{\\\\\\\\\\\\\\\"objective\\\\\\\\\\\\\\\": \\\\\\\\\\\\\\\"logistic\\\\\\\\\\\\\\\", \\\\\\\\\\\\\\\"base\\\\\\\\\\\\\\\": 0.0, \\\\\\\\\\\\\\\"tree_num\\\\\\\\\\\\\\\": 10, \\\\\\\\\\\\\\\"feature_names\\\\\\\\\\\\\\\": [\\\\\\\\\\\\\\\"X0\\\\\\\\\\\\\\\"], \\\\\\\\\\\\\\\"label_col\\\\\\\\\\\\\\\": [\\\\\\\\\\\\\\\"y\\\\\\\\\\\\\\\"], \\\\\\\\\\\\\\\"party_features_length\\\\\\\\\\\\\\\": {\\\\\\\\\\\\\\\"alice\\\\\\\\\\\\\\\": 1}}\\\\\\\"}\\\"\\n},\\n\\\"dataRefs\\\": [\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/0\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/0\\\",\\n\\\"party\\\": \\\"bob\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/1\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/1\\\",\\n\\\"party\\\": \\\"bob\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/2\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/2\\\",\\n\\\"party\\\": \\\"bob\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/3\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/3\\\",\\n\\\"party\\\": \\\"bob\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/4\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/4\\\",\\n\\\"party\\\": \\\"bob\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/5\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/5\\\",\\n\\\"party\\\": \\\"bob\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/6\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/6\\\",\\n\\\"party\\\": \\\"bob\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/7\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/7\\\",\\n\\\"party\\\": \\\"bob\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/8\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/8\\\",\\n\\\"party\\\": \\\"bob\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/9\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/9\\\",\\n\\\"party\\\": \\\"bob\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/10\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/11\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/12\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/13\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/14\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/15\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/16\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/17\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/18\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n},\\n{\\n\\\"uri\\\": \\\"ml-train-ss-xgb-train-0-0-1-0-model-1-1723539426957/19\\\",\\n\\\"party\\\": \\\"alice\\\",\\n\\\"format\\\": \\\"pickle\\\"\\n}\\n]\\n}\"\n}\nattrs {\n  ss: \"eyJkb21haW4iOiJkYXRhX3ByZXAiLCJuYW1lIjoicHNpIiwiYXR0clBhdGhzIjpbInByb3RvY29sIiwiYWxsb3dfZHVwbGljYXRlX2tleXMiLCJhbGxvd19kdXBsaWNhdGVfa2V5cy9uby9jaGVja19oYXNoX2RpZ2VzdCIsImVjZGhfY3VydmUiLCJpbnB1dC9yZWNlaXZlcl9pbnB1dC9rZXkiLCJzb3J0X3Jlc3VsdCIsImlucHV0L3NlbmRlcl9pbnB1dC9rZXkiLCJmaWxsX3ZhbHVlX2ludCIsImFsbG93X2R1cGxpY2F0ZV9rZXlzL25vL3NraXBfZHVwbGljYXRlc19jaGVjayJdLCJ2ZXJzaW9uIjoiMC4wLjUiLCJhdHRycyI6W3sicyI6IlBST1RPQ09MX1JSMjIifSx7InMiOiJubyJ9LHsiYiI6ZmFsc2V9LHsicyI6IkNVUlZFX0ZPVVJRIn0seyJzcyI6WyJJRCJdfSx7ImIiOnRydWV9LHsic3MiOlsiaWQiXX0seyJpNjQiOjB9LHsiYiI6ZmFsc2V9XX0=\"\n  ss: \"eyJkb21haW4iOiJkYXRhX3ByZXAiLCJuYW1lIjoidHJhaW5fdGVzdF9zcGxpdCIsImF0dHJQYXRocyI6WyJyYW5kb21fc3RhdGUiLCJzaHVmZmxlIiwidGVzdF9zaXplIiwidHJhaW5fc2l6ZSJdLCJ2ZXJzaW9uIjoiMC4wLjEiLCJhdHRycyI6W3siaTY0IjoxMDI0fSx7ImIiOmZhbHNlfSx7ImYiOjAuMjV9LHsiZiI6MC43NX1dfQ==\"\n  ss: \"eyJkb21haW4iOiJtbC50cmFpbiIsIm5hbWUiOiJzc194Z2JfdHJhaW4iLCJhdHRyUGF0aHMiOlsicmVnX2xhbWJkYSIsImlucHV0L3RyYWluX2RhdGFzZXQvZmVhdHVyZV9zZWxlY3RzIiwic2tldGNoX2VwcyIsImNvbHNhbXBsZV9ieV90cmVlIiwiYmFzZV9zY29yZSIsInNlZWQiLCJudW1fYm9vc3Rfcm91bmQiLCJtYXhfZGVwdGgiLCJzdWJzYW1wbGUiLCJsZWFybmluZ19yYXRlIiwib2JqZWN0aXZlIiwiaW5wdXQvdHJhaW5fZGF0YXNldC9sYWJlbCJdLCJ2ZXJzaW9uIjoiMC4wLjEiLCJhdHRycyI6W3siZiI6MC4xfSx7InNzIjpbIlgwIl19LHsiZiI6MC4xfSx7ImYiOjAuMX0seyJmIjowfSx7Imk2NCI6NDJ9LHsiaTY0IjoxMH0seyJpNjQiOjV9LHsiZiI6MC4xfSx7ImYiOjAuMX0seyJzIjoibG9naXN0aWMifSx7InNzIjpbInkiXX1dfQ==\"\n}\noutput_uris: \"model-export-package-output-1723539426957.tar\"\noutput_uris: \"model-export-report-1723539426957.csv\"\n failed, error <only support one v table input for now>\n2024-08-13 09:00:17.582 INFO api.py:342 [bob] -- [Anonymous_job] Shutdowning rayfed intendedly...\n2024-08-13 09:00:17.582 INFO api.py:356 [bob] -- [Anonymous_job] No wait for data sending.\n2024-08-13 09:00:17.583 INFO message_queue.py:72 [bob] -- [Anonymous_job] Notify message polling thread[DataSendingQueueThread] to exit.\n2024-08-13 09:00:17.584 INFO message_queue.py:72 [bob] -- [Anonymous_job] Notify message polling thread[ErrorSendingQueueThread] to exit.\n2024-08-13 09:00:17.584 INFO api.py:384 [bob] -- [Anonymous_job] Shutdowned rayfed.\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.10/runpy.py\", line 196, in _run_module_as_main\n    return _run_code(code, main_globals, None,\n  File \"/usr/local/lib/python3.10/runpy.py\", line 86, in _run_code\n    exec(code, run_globals)\n  File \"/usr/local/lib/python3.10/site-packages/secretflow/kuscia/entry.py\", line 551, in <module>\n    main()\n  File \"/usr/local/lib/python3.10/site-packages/click/core.py\", line 1157, in __call__\n    return self.main(*args, **kwargs)\n  File \"/usr/local/lib/python3.10/site-packages/click/core.py\", line 1078, in main\n    rv = self.invoke(ctx)\n  File \"/usr/local/lib/python3.10/site-packages/click/core.py\", line 1434, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File \"/usr/local/lib/python3.10/site-packages/click/core.py\", line 783, in invoke\n    return __callback(*args, **kwargs)\n  File \"/usr/local/lib/python3.10/site-packages/secretflow/kuscia/entry.py\", line 531, in main\n    res = comp_eval(sf_node_eval_param, storage_config, sf_cluster_config)\n  File \"/usr/local/lib/python3.10/site-packages/secretflow/component/entry.py\", line 176, in comp_eval\n    res = comp.eval(\n  File \"/usr/local/lib/python3.10/site-packages/secretflow/component/component.py\", line 1195, in eval\n    raise e from None\n  File \"/usr/local/lib/python3.10/site-packages/secretflow/component/component.py\", line 1190, in eval\n    ret = self.__eval_callback(ctx=ctx, **kwargs)\n  File \"/usr/local/lib/python3.10/site-packages/secretflow/component/model_export/model_export.py\", line 316, in model_export_comp_fn\n    pyus, complete_schemas = get_init_pyus(input_datasets, component_eval_params)\n  File \"/usr/local/lib/python3.10/site-packages/secretflow/component/model_export/model_export.py\", line 126, in get_init_pyus\n    assert len(v_tables) == 1, \"only support one v table input for now\"\nAssertionError: only support one v table input for now\n"
zimu-yuxi commented 3 months ago

links