search

chevah / pythia

Chevah's Python 3 distribution
MIT License
4 stars 2 forks source link

Initial continuous integration. #2

Closed dumol closed 3 years ago

dumol commented 3 years ago

Scope

Try to see if it's possible to only rely on GitHub actions and Travis CI. As per https://github.com/chevah/pythia/issues/1.

Changes

Imported relevant files from chevah/python-package.

Expanded the setup to cover as many OS'es as possible without needlessly overlapping.

Note that SHA512 checks are disabled on Windows under GitHub/Travis CI. More at https://github.com/chevah/pythia/issues/3.

Drive-by fixes:

Testing

Automated for all OS'es supported on GitHub and Travis CI.

Exceptions:

dumol commented 3 years ago

This implements testing on GitHub/Travis CI. From what I've gathered, it's not possible to cover all needed platforms this way.

GitHub's "self runners" would help expand to RHEL 7 and 8, possibly even Amazon 2. We also need at least:

Then there's also the security aspect of running self-hosted runners with public repos, more at https://docs.github.com/en/free-pro-team@latest/actions/hosting-your-own-runners/about-self-hosted-runners#self-hosted-runner-security-with-public-repositories.

Building on our Buildbot and uploading new packages to be implemented in a separate PR, after assessing the above.

adiroiban commented 3 years ago

Thanks Mișu. Impressive work :)

For Alpine 3.12 and maybe older Linux version we can run the tests inside a docker container

I have not tested, but I read that GitHub actions has some helpers for running tests inside a container: https://stackoverflow.com/a/58521709/539264

It would be super cool if we could have a way to start a VM from an base image let GitHub Runner run a job and then kill the VM and start a new one from a fresh image. This is what MS are doing with their infrastructure.

For a start, I would go with pythia as a private repo for the code. I am not sure we will receive contributions from outside of our team. We can make the resulting binaries public.

As the same time, for python 3 the focus will be on latest Ubuntu and Windows so I think that GitHub VMs have a good coverage.

But, I would still like to run our own GitHub Runners, as in this way, on strange errors we can ssh to the VM and debug.

So... I see python with python2 as a prototype for the future python3 support.

We should start with the focus on just latest Windows, latest Ubuntu and latest macOS

We should be able to run various Linux version inside Docker.

Make the project private so that we can use our own private runners.

dumol commented 3 years ago

Thanks! Agreed on almost all points, but I really hope we can avoid going private with the repo. Same issues would apply for the other public repos with CI: compat and brink.

GitHub and Travis CI envs are good enough for a basic setup. For other OS'es, my hope is we can get away with GitHub's CI support for Docker. Firing up VMs on our infrastructure as needed would require much more work.

In this particular repo, we need to build on older OS versions too, as long as they are specifically supported (e.g. Ubuntu 18.04). Also for Windows 2016, which we use for the x86 package. Not sure about macOS, there we might get away with only building on GitHub's latest macOS runner, but for now I want to have Travis build on the side, as a backup.

In regards to debugging, I'm fine with the tmate actions on Linux and macOS. Will look for something similar for Windows, starting with your suggestion: https://github.com/nelsonjchen/reverse-rdp-windows-github-actions. Also, the Try-Patch action from brink would be useful for debugging and having a cleaner list of commits, forgot to try it here.