linkedin / pygradle

Using Gradle to build Python projects
https://github.com/linkedin/pygradle
Apache License 2.0
588 stars 147 forks source link

Make wheel cache storing much more robust. #297

Closed zvezdan closed 5 years ago

zvezdan commented 5 years ago

Besides gracefully accepting already present wheel, we need to accept absence of the wheel or cache directory. Nothing bad will happen if we do not store the wheel from one layer into another. On the other hand, we found out that rogue Gradle tasks can "pull the rug under us" by triggering cleanup tasks and removing project layer cache directory. This change provides for graceful handling of these exceptions and recovery from them.

Similarly, we now do not fail when wheel building does not succeed. Again, in testing we found the libraries that do not build the wheel for the current project, but don't need it because they publish only an sdist package. So, we now gracefully accept fail wheel build, raise the flag that some wheels are not completed and let pip install from sdist/editable. The flag allows build wheels task to try rebuilding missing wheels one more time as it used to do which happens only for container builds.

One small change was also added to the numpy environment to account for LAPACK setting as custom.