search

NREL / OpenOA

This library provides a framework for assessing wind plant performance using operational assessment (OA) methodologies that consume time series data from wind plants. The goal of the project is to provide an open source implementation of common data structures, analysis methods, and utility functions relevant to wind plant OA.
https://openoa.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
194 stars 63 forks source link

Fix the recurring environment issues in test and docs pipeline #177

Closed jordanperr closed 2 years ago

jordanperr commented 3 years ago

We've been having recurring issues with the CI pipeline breaking for reasons unrelated to the pull request at hand. The main reason for this issue is that packages are updated in public registries, which our CI pipeline then pulls from.

One solution is to implement a containerized CI/CD pipeline. That is, build a Docker container - and host this on DockerHub - which contains a static version of the environment. Typically, the creation of this container is also an automated process.

jordanperr commented 2 years ago

It appears the github action setup-python (which we are already using in our pipeline) has a built in dependency caching feature. If my understanding of this is correct, it would use cached versions of the dependencies as long as the requirements.txt file remains unchanged. Downloading new versions of the dependencies when the requirements file changes.

Link to documentation for setup-python caching.

I implemented the change in this branch and will test the functionality before submitting a pull request: https://github.com/jordanperr/OpenOA/tree/maintenence/12-02-2021-enable-dependency-caching

SauravMaheshkar commented 2 years ago

Hey @jordanperr can I work on this ? I'd love to contribute to the repository and this seems like a "good first issue"

jordanperr commented 2 years ago

Hi @SauravMaheshkar , thank you so much for your interest. I apologize for not getting back to you sooner. We have decided to move away from caching and instead add a scheduled build to the workflow to catch these issues earlier: https://github.com/NREL/OpenOA/blob/c1c163007d16986d04bc34deefbf1c1e1c754aa8/.github/workflows/ci-tests.yml#L8

I would be very interested to hear any other ideas on how to improve this CI pipeline. Closing the issue for now.