search

DiamondLightSource / python3-pip-skeleton

Archived in favour of https://github.com/DiamondLightSource/python-copier-template
Apache License 2.0
4 stars 4 forks source link

Add Conda Upload to release CI #94

Closed garryod closed 1 year ago

garryod commented 1 year ago

Resolves #87 by adding conda upload to the release CI

codecov[bot] commented 1 year ago

Codecov Report

Merging #94 (4981421) into main (4981421) will not change coverage. The diff coverage is n/a.

:exclamation: Current head 4981421 differs from pull request most recent head 74b599b. Consider uploading reports for the commit 74b599b to get more accurate results

@@            Coverage Diff            @@
##              main       #94   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            2         2           
  Lines           13        13           
=========================================
  Hits            13        13

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

coretl commented 1 year ago

What steps do you need to take on the conda forge side to make this work? An org level token? Or do you need to manually make a project there and give a token from that?

garryod commented 1 year ago

What steps do you need to take on the conda forge side to make this work? An org level token? Or do you need to manually make a project there and give a token from that?

An org level token is sufficient. I believe @PeterC-DLS currently owns the DiamondLightSource org, and have been in touch with him to look at that side of things

garryod commented 1 year ago

See the following CI workflows for proper testing of the publish action:

Package is here: https://anaconda.org/garryod/python3-pip-skeleton

gilesknap commented 1 year ago

Looks good. I have a good candidate project to try out the 'token gated' Pypi release on which I'll try to test on Friday.

garryod commented 1 year ago

To preempt the question of "why do we now have this awful build stage?"... Uploading the dist to anaconda did indeed give us a package hosted on anaconda.org, this package was however a pip package installable via pip install -i https://anaconda.org/garryod/ python3-pip-skeleton and carried with it the same compatibility issues experience when installing pip packages from pypi into a conda environment

See this CI job for a run with publish to anaconda.org

MJGaughran commented 1 year ago

At the moment, we do not automatically push to global PyPI just by adopting the pip skeleton.

According to Giles, a global conda token has already been provided. Can we please make the skeleton behave identically for both and not automatically release to conda?

PeterC-DLS commented 1 year ago

There is no Diamond-wide org on PyPI (I have a DLS-Scisoft one). PyPI's organization support is rather lacking but a project exists to improve things.

garryod commented 1 year ago

At the moment, we do not automatically push to global PyPI just by adopting the pip skeleton.

According to Giles, a global conda token has already been provided. Can we please make the skeleton behave identically for both and not automatically release to conda?

Yes, a organisation level anaconda token is provided for all public github repos, meaning that the default behaviour will be pushing to anaconda on release, my understanding is that it is our intention to have the same setup for PyPI once support for organisations is implemented

Is there a particular use case in which we wouldn't want to push to our anaconda channel by default?

MJGaughran commented 1 year ago

At the moment, we do not automatically push to global PyPI just by adopting the pip skeleton. According to Giles, a global conda token has already been provided. Can we please make the skeleton behave identically for both and not automatically release to conda?

Yes, a organisation level anaconda token is provided for all public github repos, meaning that the default behaviour will be pushing to anaconda on release, my understanding is that it is our intention to have the same setup for PyPI once support for organisations is implemented

Is there a particular use case in which we wouldn't want to push to our anaconda channel by default?

As of this moment, I just want PyPI and Conda to have similar behaviours. Personally, I prefer to 'enable' things like pushing to PyPI or Conda, although I understand you do need to push a tag.

I believe private services, test repos etc make this opt-in behaviour useful, but I understand that some people would disagree.

garryod commented 1 year ago

At the moment, we do not automatically push to global PyPI just by adopting the pip skeleton. According to Giles, a global conda token has already been provided. Can we please make the skeleton behave identically for both and not automatically release to conda?

Yes, a organisation level anaconda token is provided for all public github repos, meaning that the default behaviour will be pushing to anaconda on release, my understanding is that it is our intention to have the same setup for PyPI once support for organisations is implemented Is there a particular use case in which we wouldn't want to push to our anaconda channel by default?

As of this moment, I just want PyPI and Conda to have similar behaviours. Personally, I prefer to 'enable' things like pushing to PyPI or Conda, although I understand you do need to push a tag.

I believe private services, test repos etc make this opt-in behaviour useful, but I understand that some people would disagree.

Yeah, I would have to agree that similar behaviour is desirable, though there will always be some intrinsic difference as PyPI adopts a flat structure whilst Anaconda has "channels".

In my head the visibility of the repository makes for a good indicator of whether it should be visible on package repositories, but perhaps this is not the case for some of your projects?

coretl commented 1 year ago

If we want to do a switch on public/private we probably need to use the API to get that info: https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#get-a-repository

PeterC-DLS commented 1 year ago

You can also switch public/private for the conda channel.

garryod commented 1 year ago

If we want to do a switch on public/private we probably need to use the API to get that info: https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#get-a-repository

The org level anaconda key is only for public repos, so this is built in as is

PeterC-DLS commented 1 year ago

Can be done afterwards manually - it's a one-off for each package (not each version).

garryod commented 1 year ago

I think we are talking about two different things here:

coretl commented 1 year ago

I'm happy with this if @MJGaughran is satisfied with the "global token" behaviour

MJGaughran commented 1 year ago

This seems acceptable until we can get the global PyPI key set up. We should just make it clear in the docs that this will happen automatically.

I would be wary about adding such things to the skeleton in the future, though. You wouldn't expect such functionality to be added as part of an update merge.