search

oap-project / raydp

RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.
Apache License 2.0
308 stars 68 forks source link

[raydp-332] Fix security issue of protobuf < 3.19.5 #333

Closed jiafuzha closed 1 year ago

jiafuzha commented 1 year ago

There is a security issue report, https://github.com/oap-project/raydp/security/dependabot/6.

Package protobuf

Affected versions >= 3.19.0, < 3.19.5

Patched version 3.19.5

 protobuf-cpp and protobuf-python have potential Denial of Service issue

To fix it, we limit protobuf > 3.19.5 and <= 3.20.3

jiafuzha commented 1 year ago

@carsonwang as tested with ray 2.1 - 2.3, it worked though there is a warning like below,

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
grpcio-tools 1.51.3 requires protobuf<5.0dev,>=4.21.6, but you have protobuf 3.20.3 which is incompatible.
kira-lin commented 1 year ago

Is grpcio-tools used in our MPI test? Maybe we should downgrade its version as well?

jiafuzha commented 1 year ago

Is grpcio-tools used in our MPI test? Maybe we should downgrade its version as well? It's dep of ray Requirement already satisfied: grpcio>=1.32.0 in /home/jiafu/anaconda3/envs/ray2.1/lib/python3.9/site-packages (from ray>=2.1.0->raydp==1.6.0.dev0) (1.51.3)

jiafuzha commented 1 year ago

@kira-lin @carsonwang do you have more comments?

kira-lin commented 1 year ago

LGTM

kira-lin commented 1 year ago

Ray's requirement for protobuf is "protobuf >= 3.15.3, != 3.19.5", why are we limiting it <=3.20.3?

jiafuzha commented 1 year ago

grpcio-tools 1.51.3 requires protobuf<5.0dev,>=4.21.6,

It's due to grpcio-tools install protobuf-4.22.* which doesn't work well with raydp.

kira-lin commented 1 year ago

Thanks, merged