OPM / ResInsight

3D viewer and post processing of reservoir models
http://resinsight.org/
GNU General Public License v3.0
173 stars 91 forks source link

Remove pinning of Python modules #9304

Closed magnesj closed 1 year ago

magnesj commented 1 year ago

Pinning of some Python modules was introduced due to issues in protobuf. These issues are fixed, and pinning can be removed.

Original issue description

From the log, we can see that the github action variable pythonLocationis changing from 3.8 to 3.10 after running action https://github.com/jurplel/install-qt-action/tree/v2

https://github.com/OPM/ResInsight/actions/runs/3080820654/jobs/4978625692

Related issue related to protobuf https://github.com/protocolbuffers/protobuf/issues/10075

magnesj commented 1 year ago

Issues have been seen in protobuf>3.20 with the current version of grpc used in ResInsight.

berland commented 1 year ago

Can this be retested, and checked whether that will allow unpinning of protobuf in requirements.txt?

oysteoh commented 1 year ago

Bumping the question from Håvard here , possible to have a look at the pinning @magnesj ?

magnesj commented 1 year ago

I can look into this next week. What is your preferred solution, no pinning at all?

oysteoh commented 1 year ago

If possible the best thing is to remove pinning :+1:

magnesj commented 1 year ago

The pinning of modules was introduced in https://github.com/OPM/ResInsight/commit/587f700ae4db4dfcca273449423e55aa273d44e6. At this point, the latest version of grpc and protobuf caused issues.

The protobuf issue was fixed in 4.21.9, see https://github.com/protocolbuffers/protobuf/issues/10075#issuecomment-1309567465

These issues are now resolved and the pinning can be removed.

magnesj commented 1 year ago

@oysteoh, pinning is now removed. Please let us know if this solves your issues.

oysteoh commented 1 year ago

Are you planning on making a new tag on this release on pypi? We are getting the rips package from there and cannot test the change until that is done 👍

magnesj commented 1 year ago

c++ files related to grpc are generated using the protoc tool. This tool is installed using vcpkg and, the current version of this tool is 3.15.8. The protoc tool is launched from cmake in https://github.com/OPM/ResInsight/blob/dev/GrpcInterface/CMakeLists.txt

Python code is generated based on the same proto files, and they are generated using the Python package grpcio-tools in CMakeLists.txt

The current version of grpcio-tools is 1.33. This version causes issues on readthedocs https://readthedocs.org/projects/python-rips-doc/builds/21129013/

Investigate if it is possible to use the latest version of grpcio-tools with the current grpc used in c++.

Latest release of vcpkg contains protoc version 3.21.12 https://github.com/microsoft/vcpkg/commits/master/ports/protobuf

magnesj commented 1 year ago

Updated python packages are verified working as expected on Equinor infrastructure.