patch-graph / pass-the-fork

[WIP] inject forked dependencies into CI builds
MIT License
1 stars 1 forks source link

Python support #11

Open jayvdb opened 5 years ago

jayvdb commented 5 years ago

Arriving from https://travis-ci.community/t/api-triggered-build-with-config-overriddes-shows-unmodified-config-in-ui/2330

It would be lovely to have support for Python for @coala . The main need there would be coala/coala PRs triggering a build of coala/coala-bears (currently broken, but should be fixed shortly https://github.com/coala/coala-bears/pull/2910 ). I have an issue about it somewhere which I'll fetch later.

Another crazier, but more important need, is there are a few orgs that are heavily templated from a configuration management repository using a tool https://github.com/moremoban/moban . coala is one of them, but our CM repo is on gitlab, so I guess that is out of scope at least for the moment. An easier one is @pyexcel , which uses two CM repos, https://github.com/pyexcel/pyexcel-mobans layered on top of (overriding) https://github.com/moremoban/pypi-mobans . Even easier is the moban tool's repo itself (https://github.com/moremoban/), which only templates from https://github.com/moremoban/pypi-mobans .

When there is a proposed change to https://github.com/moremoban/pypi-mobans (usually a .jj2 template modification) , we really want to see how that change would impact https://github.com/moremoban/moban , https://github.com/pyexcel/pyexcel (and maybe a few others) , https://github.com/coala/coala , etc. Trying to build comprehensive tests suites for that is crazy work, as there are extremely large numbers of combinations of all of the configuration variables, and many are outdated and being phased out - it is simpler to do builds of real configurations - a finite sample set. https://github.com/moremoban/pypi-mobans doesnt produce a Python package , so the dependency could only use the branch on the fork. However, the moban tool can be pointed at different branches (requires editing the control file but it is a simple enhancement for an envvar to override that). c.f. https://github.com/moremoban/moban/issues/286 for the way I was planning on solving this - the template editor manually creating PRs in each dependent repo and editing the control file to point at the upstream PR.

Do you see those needs as something you're keen to add soon? I can do some of the work, but I will need guidance.

cben commented 4 years ago

Hi :wave:. Apologies for the radio silence — I had a frantic time at work and life (moved apartments) and this sunk to the bottom of the stack :-( It's also silly that I intended this to be a "by the devs, for the devs" project and give all contributors commit rights in the spirit of https://openopensource.org/, but that doesn't work if I ignore contributors :) I'll invite you to the org.

Sure, let Python be the next language :+1: I got out of touch with Python packaging nowdays... Which dependency tool/format(s) is worth supporting? pip/requirements.txt? pipenv/Pipfile?

Looking at https://github.com/moremoban/moban/issues/286 it sounds like you're not so much talking about overriding pip install but about overriding git cloning done by moban (according to mobanfile.yaml)?


I have to point this project's goal is to enable "unplanned" dependency replacement, with no cooperation from the involved repos. Imagine another scenario: a contributor wants a change in moban dependent on a change in say crayons, and that's not a combo these projects planned for; and likely they've never heard of pass-the-fork. But the contributor knows about it and can use it to run moban's tests with one and link to them :sparkles:.

However in your case you have a recurring need to test some projects together, and are willing to prepare them. Perhaps you can build something simpler (?). Feel free to use parts from here of course.


Gitlab is out of scope for run.js which uses Travis to run tests (hmm I should rename it to say run-travis.js), and Travis only supports Github (https://github.com/travis-ci/travis-ci/issues/5931#issuecomment-213519395). I hope to eventually create something for Gitlab-CI too but it's low priority for myself. But the injectors/ and shims/ parts should be reusable.


I'll try to answer your other issue tomorrow.

cben commented 4 years ago

See #10 for some details on the `git shim.