search

secretflow / scql

SCQL (Secure Collaborative Query Language) is a system that allows multiple distrusting parties to run joint analysis without revealing their private data.
https://www.secretflow.org.cn/docs/scql/en/
Apache License 2.0
122 stars 46 forks source link

SCQL中心化节点部署流程中问题 #325

Open daoshouchen opened 1 month ago

daoshouchen commented 1 month ago

Issue Type

Others

Have you searched for existing issues?

Yes

Link to Relevant Documentation

https://www.secretflow.org.cn/zh-CN/docs/scql/0.8.1b1/topics/deployment/how-to-deploy-centralized-cluster

Question Details

1.在部署scql过程中发现节点中连不上go官网。
构建scdbclient go build -o scdbclient cmd/scdbclient/main.go 会报错。
443: connect: connection timed out
pkg/util/message/message_io_util.go:21:2: google.golang.org/protobuf@v1.33.0: Get "https://proxy.golang.org/google.golang.org/protobuf/@v/v1.33.0.zip": dial tcp 142.251.215.241:443: connect: connection timed out
pkg/util/message/message_io_util.go:22:2: google.golang.org/protobuf@v1.33.0: Get "https://proxy.golang.org/google.golang.org/protobuf/@v/v1.33.0.zip": dial tcp 142.251.215.241:443: connect: connection timed out
cmd/scdbclient/main.go:30:2: github.com/spf13/cobra@v1.6.1: Get "https://proxy.golang.org/github.com/spf13/cobra/@v/v1.6.1.zip": dial tcp 142.251.215.241:443: connect: connection timed out
383004576 commented 1 month ago

您好，通过报错看起来是网络问题，通过scdbclient go build -o scdbclient cmd/scdbclient/main.go部署scql需要保证外网通讯畅通，建议先检查下网络情况。

daoshouchen commented 1 month ago

确实是网络问题，目前存在网络没办法连接上下载所需依赖，能否提供一个不需要实时网络下载就能构建完成的方案。 因为无论在公司服务器节点或者生产服务节点，几乎都是无法满足网络去下载所需依赖的。

daoshouchen commented 1 month ago

有没有可能将所需依赖打包成一份lib，之后进行构建使用？

383004576 commented 1 month ago

打包构建目前不支持，建议先使用外网环境验证，后续可以参考文档和代码通过将镜像和依赖包手动上传来配置。

daoshouchen commented 1 month ago

我尝试用本机进行操作，依然还是无法下载依赖。完全没法进展下去了，有没有可能提供点有效解决方案或者处理思路？

383004576 commented 1 month ago

麻烦提供下失败信息或截图

jingshi-ant commented 1 month ago

可以尝试配置下go的代理，然后重试： go env -w GOPROXY=https://goproxy.cn,direct

daoshouchen commented 1 month ago

可以尝试配置下go的代理，然后重试： go env -w GOPROXY=https://goproxy.cn,direct

有效；出现新问题： go build -o scdbclient cmd/scdbclient/main.go go: downloading github.com/influxdata/go-prompt v0.2.8 go: downloading github.com/spf13/cobra v1.6.1 go: downloading github.com/olekukonko/tablewriter v0.0.5 go: downloading github.com/golang/mock v1.6.0 go: downloading google.golang.org/genproto/googleapis/api v0.0.0-20240227224415-6ceb2ff114de go: downloading google.golang.org/grpc v1.63.2 go: downloading google.golang.org/protobuf v1.33.0 go: downloading google.golang.org/genproto v0.0.0-20240227224415-6ceb2ff114de go: downloading github.com/mattn/go-runewidth v0.0.13 go: downloading golang.org/x/sys v0.19.0 go: downloading github.com/spf13/pflag v1.0.5 go: downloading golang.org/x/net v0.24.0 go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20240415180920-8c6c420018be go: downloading github.com/pkg/term v1.2.0-beta.2 go: downloading github.com/rivo/uniseg v0.2.0 go: downloading golang.org/x/text v0.14.0

runtime/internal/sys

/usr/local/go/src/runtime/internal/sys/consts.go:18:7: DefaultPhysPageSize redeclared in this block /usr/local/go/src/runtime/internal/sys/arch_amd64.go:10:2: other declaration of DefaultPhysPageSize /usr/local/go/src/runtime/internal/sys/consts.go:22:7: PCQuantum redeclared in this block /usr/local/go/src/runtime/internal/sys/arch_amd64.go:11:2: other declaration of PCQuantum /usr/local/go/src/runtime/internal/sys/consts.go:25:7: Int64Align redeclared in this block /usr/local/go/src/runtime/internal/sys/arch_amd64.go:12:2: other declaration of Int64Align /usr/local/go/src/runtime/internal/sys/consts.go:32:7: MinFrameSize redeclared in this block /usr/local/go/src/runtime/internal/sys/arch_amd64.go:14:2: other declaration of MinFrameSize /usr/local/go/src/runtime/internal/sys/stubs.go:16:7: StackGuardMultiplier redeclared in this block /usr/local/go/src/runtime/internal/sys/consts.go:15:7: other declaration of StackGuardMultiplier /usr/local/go/src/runtime/internal/sys/stubs.go:16:30: undefined: StackGuardMultiplierDefault

math

/usr/local/go/src/math/acosh.go:43:6: Acosh defined in both Go and assembly /usr/local/go/src/math/asin.go:20:6: Asin defined in both Go and assembly /usr/local/go/src/math/asin.go:58:6: Acos defined in both Go and assembly /usr/local/go/src/math/asinh.go:40:6: Asinh defined in both Go and assembly /usr/local/go/src/math/atan.go:96:6: Atan defined in both Go and assembly /usr/local/go/src/math/atan2.go:30:6: Atan2 defined in both Go and assembly /usr/local/go/src/math/atanh.go:48:6: Atanh defined in both Go and assembly /usr/local/go/src/math/cbrt.go:26:6: Cbrt defined in both Go and assembly /usr/local/go/src/math/erf.go:189:6: Erf defined in both Go and assembly /usr/local/go/src/math/erf.go:274:6: Erfc defined in both Go and assembly /usr/local/go/src/math/erf.go:274:6: too many errors

jingshi-ant commented 1 month ago

check下go的版本（go version），如果版本老旧，需要按照官网升级go。

daoshouchen commented 1 month ago

go version go version go1.23.0 linux/amd64

jingshi-ant commented 1 month ago

可以参考下，可能是go安装的问题：https://stackoverflow.com/questions/62330084/fail-to-build-with-code-go-redeclared-in-this-block

jingshi-ant commented 1 month ago

看报错是端口服务无法访问，而且scdb log并没有接收到请求，maybe是跨网络连接的问题？可以检查下远程节点的8080端口是否开放访问权限。

daoshouchen commented 1 month ago

create database demo [submit and get]err: Post "http://10.199.0.20:18080/public/submit_and_get": dial tcp 10.199.0.20:18080: connect: connection refused 以下是scdb配置，劳烦看一下哪里有问题？ 1.scdb config.yml 配置 scdb_host: http://10.199.0.20:18080 port: 18080 2.docker-compose.yaml
version: "3.8" services: scdb: image: secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/scql:latest environment:

daoshouchen commented 1 month ago

问题已解决，是docker的宿主机端口未配置

daoshouchen commented 1 month ago

总结一下在部署过程中遇到的一些常见问题： 1、在配置镜像时可能会连不上，需要配置改成阿里云镜像仓库:secretflow-registry.cn-hangzhou.cr.aliyuncs.com/secretflow/scql:latest 2、部署过程中依赖的openssl需要较高版本，我升级到openssl-3.2.2.tar.gz，生成密钥才正常。 3、在部署过程中依赖的go环境版本需要1.22及以上，通常需要升级，到官网下载最新go版本升级，参考https://stackoverflow.com/questions/62330084/fail-to-build-with-code-go-redeclared-in-this-block 4、在构建scdbclient 客户端时需要依赖go网络下载依赖，有可能网络连不上，采用go代理：go env -w GOPROXY=https://goproxy.cn,direct/ 5、在节点以及scdb启动时docker的启动端口，宿主机端口，监听端口有时会弄混。建议采用scdb 18080,engine 28080,38080这种区分。并将宿主机端口映射设定好，以免随机生成； 6、原生有mysql的，在docker拉起时可以忽略mysql配置，提前创建好数据库及执行完成sql文件。

Chrisdehe commented 1 month ago

@daoshouchen 已收到，非常感谢！我们将重视您的意见反馈，将帮助我们改进SCQL，使其更加完善和易用。

daoshouchen commented 1 month ago

请教一个高可用部署的问题，如何在生产中保证各节点的高可用方案？是可以和传统应用一样alice多节点使用负载，bob多节点使用负载，各方通过负载地址进行通信吗？

jingshi-ant commented 1 month ago

scql本身的p2p模式支持多实例部署，可以部署多个broker/engine，但流量均衡、节点健康探查之类的机制缺失（这类功能放在平台、网关实现maybe更合适）。实践上业务可配合secretflow/kuscia平台实现高可用部署。 中心化模式模式中scdb模块是单节点，engine可以部署多个实例，但也需要配合网关解决路由问题。

daoshouchen commented 1 month ago

在中心化模式中，在部署多个engine实例时的密钥文件是一定要同一个对吗？

jingshi-ant commented 1 month ago

如果启用了authorized_profile_path，则会校验公钥，因此需要是同一个

jingshi-ant commented 1 month ago

补充一下：authorized_profile_path里的公钥是创建user时对应的秘钥，和engine里用来加密https的是两回事，一个用于辨别身份，一个是https加密，如果两个秘钥不复用，则engine多实例的秘钥可以不同。

github-actions[bot] commented 6 days ago

Stale issue message. Please comment to remove stale tag. Otherwise this issue will be closed soon.