Automate conda-forge PRs with rendered recipes

[jlstevens]

One of the goals of pyct is to avoid lists of dependencies spread across different locations. This has been minimized for our own conda packaging with meta.yaml getting the dependencies for setup.py. However, this does not help us for conda-forge which has its own recipe meta.yaml specification.

Conda forge has an regro-cf-autotick-bot which updates the version and source URL (typically PyPI) but it does not try to update the project dependencies. This bot is very convenient but it is easy for dependency/metadata information to become stale.

Having access to the correctly rendered recipe would be helpful but there are many more steps that need to be handled before we have a convenient conda-forge workflow:

1. The source field would need to be set to be the correct PyPI URL in the rendered recipe.
2. It would be much easier if pyct could also handle: a. Creating/updating the fork for the conda-forge feedstock (conda forge want people to use forks) b. Making a commit with the freshly re-rendered recipe. c. Making a PR back to the feedstock on conda-forge.

Having step 1 handled would help but always updating the recipe with the rendered recipe is a pain. It is much easier to hit merge on the regro-cf-autotick-bot to build a new package even if that risks having the dependency specification go stale!

Note that if using the GitHub APIs to automate all this is too much of a pain, it would be helpful to put the rendered recipe online at an easy-to-visit URL so that we could copy/paste it to create a PR directly on GitHub. This would be easier than putting the rendered recipe into the package archive which would need to be downloaded/extracted/inspected.

[ceball]

I agree with the spirit of what you want. I.e. I am all for things happening without manual intervention where possible. If I could click my fingers and have the PR, I would merge it.

As a possible alternative, do you think it would be worth discussing with conda-forge whether they'd ever accept 'metadata' (dependencies, description, supported python versions, etc) coming from an external source?

It is much easier to hit merge on the regro-cf-autotick-bot to build a new package even if that risks having the dependency specification go stale!

I view dependencies not as something that 'might go stale' - where on an automatically created PR you should 'Please check that the dependencies have not changed' - but as something as important as the source code itself.

conda forge want people to use forks

Incidentally, I believe that's the most common workflow on github by far (though I don't have data to back that up).

[jlstevens]

As a possible alternative, do you think it would be worth discussing with conda-forge whether they'd ever accept 'metadata' (dependencies, description, supported python versions, etc) coming from an external source?

That would be nice but might be hard to organize.

Maybe as a first step, there could be a doit command to render the template at any time, showing the output on standard output. Then it might be possible to copy/paste that in a workflow that is purely on GitHub by making the commit through the website.

Note: I do know there is some standard way to render the template with conda, but if there is a doit command, then I wouldn't have to look it up each time.

[jbednar]

It is much easier to hit merge on the regro-cf-autotick-bot to build a new package even if that risks having the dependency specification go stale!

I view dependencies not as something that 'might go stale'

I agree; having an easy way to build a broken package isn't a good thing, it's a bad thing!

[jlstevens]

I agree; having an easy way to build a broken package isn't a good thing, it's a bad thing!

I also agree but there is this nice conda forge bot that helpfully lets me build potentially broken packages without thinking too much about it. :-)

I want the same thing for properly rendered recipes.