search

secretflow / interconnection-impl

Apache License 2.0
3 stars 3 forks source link

编译impl出现问题：bzl依赖库 #21

Open reacher1130 opened 4 days ago

reacher1130 commented 4 days ago

f如果使用官方interconnection-impl的源码，不修改repositories.bzl，其中 def ic_impl_deps(): _com_github_nlohmann_json() maybe( git_repository, name = "spulib", commit = SPU_COMMIT_ID, remote = "{}/{}.git".format(SECRETFLOW_GIT, SPU_REPOSITORY),

remote = "https://github.com/reacher1130/spu.git",

)

这是可以bazel编译通过的； 如果将其中的spu的git依赖库改成自己fork过的spu的main分支，并且spu中的repositories.bzl的yacl的git依赖库也改成自己fork过的yacl的main分支，就会出现如下编译问题： ERROR: /root/develop/interconnection-impl/ic_impl/algo/lr/BUILD.bazel:19:11: error loading package '@spulib//libspu/mpc': at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/spulib/bazel/spu.bzl:20:6: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/yacl/bazel/yacl.bzl:20:6: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/rules_foreign_cc/foreign_cc/defs.bzl:8:6: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/rules_foreign_cc/foreign_cc/ninja.bzl:8:5: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/rules_foreign_cc/foreign_cc/private/framework.bzl:5:6: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/bazel_features/features.bzl:3:6: Unable to find package for @bazel_features_globals//:globals.bzl: The repository '@bazel_features_globals' could not be resolved: Repository '@bazel_features_globals' is not defined. and referenced by '//ic_impl/algo/lr:lr_handler'

需求：对yacl其中相关互联互通部分的代码进行修改和日志输出，因此需要在bzl中修改为自己的yacl库，非常感谢！

reacher1130 commented 4 days ago

或者出现如下问题： ERROR: /root/develop/interconnection-impl/ic_impl/algo/psi/v2/BUILD.bazel:29:11: error loading package '@psi//psi/legacy': at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/psi/bazel/psi.bzl:20:6: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/yacl/bazel/yacl.bzl:20:6: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/rules_foreign_cc/foreign_cc/defs.bzl:3:6: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/rules_foreign_cc/foreign_cc/boost_build.bzl:5:5: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/rules_foreign_cc/foreign_cc/private/framework.bzl:5:6: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/bazel_features/features.bzl:3:6: Unable to find package for @bazel_features_globals//:globals.bzl: The repository '@bazel_features_globals' could not be resolved: Repository '@bazel_features_globals' is not defined. and referenced by '//ic_impl/algo/psi/v2:psi_context_v2'

或者随机出现： ERROR: /root/develop/interconnection-impl/ic_impl/BUILD.bazel:74:11: error loading package '@yacl//yacl/base': at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/yacl/bazel/yacl.bzl:20:6: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/rules_foreign_cc/foreign_cc/defs.bzl:4:6: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/rules_foreign_cc/foreign_cc/cmake.bzl:140:5: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/rules_foreign_cc/foreign_cc/private/framework.bzl:5:6: at /root/.cache/bazel/_bazel_root/b12e4e3f9a14f11e902c5f6f1fe2e080/external/bazel_features/features.bzl:3:6: Unable to find package for @bazel_features_globals//:globals.bzl: The repository '@bazel_features_globals' could not be resolved: Repository '@bazel_features_globals' is not defined. and referenced by '//ic_impl:util'

shaojian-ant commented 4 days ago

yacl 最新版本更新了 rules_foreign_cc 包。需要在 interconnection-impl 的 WORKSPACE 文件中增加下面两行代码：

load("@bazel_features//:deps.bzl", "bazel_features_deps")
bazel_features_deps()

参看 SPU 的配置文件：https://github.com/secretflow/spu/blob/main/WORKSPACE#L56

reacher1130 commented 4 days ago

但yacl最新版本的workspace里存在这两句load和依赖，因此在interconnection-impl的workspace里增加这两行代码时会提示重复： ERROR: Failed to load Starlark extension '@bazel_features//:deps.bzl'. Cycle in the workspace file detected. This indicates that a repository is used prior to being defined. The following chain of repository dependencies lead to the missing definition.

shaojian-ant commented 4 days ago

interconnection-impl 不会用到 yacl WORKSPACE 里面的代码，只用到 yacl repositories.bazel 里面的代码。你的 WORKSPACE 文件是怎么修改的呢

shaojian-ant commented 4 days ago

只需要把 下面代码放在 interconnection-impl WORKSPACE 文件的最后应该就可以

load("@bazel_features//:deps.bzl", "bazel_features_deps")
bazel_features_deps()