regro / cf-scripts

Flagship repo for cf-regro-autotick-bot
Other
47 stars 72 forks source link

Auto update R packages #745

Open CJ-Wright opened 4 years ago

CJ-Wright commented 4 years ago

In discussion with @ocefpaf and @regro/auto-tick

It seems that the R skeleton system may be advanced enough to auto update the R recipes. We could have the bot perform this as part of its Version bump activities.

ocefpaf commented 4 years ago

Pinging @jdblischak and @bgruening who may be interested in this and have more info on the R skeleton status.

jdblischak commented 4 years ago

It seems that the R skeleton system may be advanced enough to auto update the R recipes.

@CJ-Wright Could you please expand on this? The R skeleton command creates a new R recipe like the other recipe skeletons. Is there a specific feature you had in mind that would help facilitate this?

The R maintainers have discussed this before here. My biggest concern is not losing any system dependencies that were manually added. In other words, the current dependencies starting with r- can be replaced by whatever the skeleton has added for the latest version of the recipe, but all other dependencies should be maintained.

@conda-forge/r doesn't seem to work here, so also tagging @cbrueffer @dpryan79 @dbast

CJ-Wright commented 4 years ago

(@conda-forge/r doesn't work because formally this repo isn't part of the CF org)

ocefpaf commented 4 years ago

@CJ-Wright Could you please expand on this?

The packages in defaults are updated via the skeleton and, we are close to do the same in conda-forge.

The R skeleton command creates a new R recipe like the other recipe skeletons. Is there a specific feature you had in mind that would help facilitate this?

New or modified is not the issue. The issue is if we loose a change that should stay. It is my impression that this could be "fixed" with the post script that @conda-forge/r uses, no? If not, this issue is to assess what needs to be added to ensure we can just "regenerate" the recipe in the same fashion defaults does, with minimal users tweaks.

The R maintainers have discussed this before here. My biggest concern is not losing any system dependencies that were manually added. In other words, the current dependencies starting with r- can be replaced by whatever the skeleton has added for the latest version of the recipe, but all other dependencies should be maintained.

That is something we should improve in the skeleton. Pinging @marcelotrevisani here.

PS: for python we are working with PyPA to add a new metadata info for "external dependencies." Not sure if R has something similar.

jdblischak commented 4 years ago

Not sure if R has something similar.

The DESCRIPTION file has an optional field SystemRequirements. It is free-form text and not standardized at all (especially when the package author provides minimum version information). The most comprehensive effort I know of to standardize this information for CRAN packages is sysreqsdb.

marcelotrevisani commented 4 years ago

Hi,

Thanks for the info!

I have been working to refactor the entire skeleton and I extracted the code which I was developing to a separated repository to be easier to maintain and to generate a package for it (in a second step we can integrate it to the conda-build) https://github.com/marcelotrevisani/grayskull

Anything which you find that would be useful/nice to have in skeleton, please feel free to open an issue in that repository. That would be good to track the features which we are missing or we want to develop.

A priori, the code I am developing we will be able to auto-update the recipes and to be able to also update just some parts of the recipe, for example, maybe you just want to update the build section or just the requirements that will be possible.

I still have some stuff on my local computer, some things I might do some refactoring in that repository. But, for now, it is a Tracer Bullet software, which we can evolve over time.

This week I am a bit busy with some personal things, but I hope to generate the first release next week. This release we will be able to generate recipes for python, right after that I hope to tackle the R recipes and the help from you guys would be good (at least regarding the R dependencies and build, the code onBioconda will be helpful as well) :)