Open yuvipanda opened 5 years ago
Is there a list somewhere of existing buildpacks and their implementation? I had a quick look at https://buildpacks.io/docs/ but couldn't find that list.
From buildpacks.slack.com I found https://github.com/cloudfoundry/conda-cnb
Wow this was exciting stuff @yuvipanda ! :heart: :tada:
https://github.com/buildpack/pack is probably the best way to get started playing around locally. (Disclosure I'm the "anchor" (tech lead) on the pivotal buildpacks team). CNBs are, for now, generally consumed through a builder image (what that is is not especially important for the purpose of this comment). You can see the "blessed" builder images by running
# pack suggest-builders
Suggested builders:
Cloud Foundry: cloudfoundry/cnb:bionic Ubuntu bionic base image with buildpacks for Java, NodeJS and Golang
Cloud Foundry: cloudfoundry/cnb:cflinuxfs3 cflinuxfs3 base image with buildpacks for Java, NodeJS, Python, Golang, PHP, HTTPD and NGINX
Heroku: heroku/buildpacks:18 heroku-18 base image with buildpacks for Ruby, Java, Node.js, Python, Golang, & PHP
If you inspect our cflinuxfs3 builder image, for example, you can get an idea of the CNBs that we've published that are "ready", although we have a bunch more in flight:
# pack inspect-builder cloudfoundry/cnb:cflinuxfs3
Inspecting builder: cloudfoundry/cnb:cflinuxfs3
Remote
------
Description: cflinuxfs3 base image with buildpacks for Java, NodeJS, Python, Golang, PHP, HTTPD and NGINX
Stack: org.cloudfoundry.stacks.cflinuxfs3
Lifecycle Version: 0.4.0
Run Images:
cloudfoundry/run:full-cnb
Buildpacks:
ID VERSION LATEST
org.cloudfoundry.node-engine 0.0.46 true
org.cloudfoundry.npm 0.0.29 true
org.cloudfoundry.yarn 0.0.24 true
org.cloudfoundry.python 0.0.20 true
org.cloudfoundry.pip 0.0.20 true
org.cloudfoundry.pipenv 0.0.14 true
org.cloudfoundry.conda 0.0.13 true
org.cloudfoundry.go-compiler 0.0.20 true
org.cloudfoundry.go-mod 0.0.18 true
org.cloudfoundry.dep 0.0.17 true
org.cloudfoundry.php-dist 0.0.28 true
org.cloudfoundry.php-composer 0.0.16 true
org.cloudfoundry.httpd 0.0.18 true
org.cloudfoundry.nginx 0.0.20 true
org.cloudfoundry.php-web 0.0.19 true
org.cloudfoundry.openjdk 1.0.0-RC03 true
org.cloudfoundry.buildsystem 1.0.0-RC03 true
org.cloudfoundry.jvmapplication 1.0.0-RC03 true
org.cloudfoundry.azureapplicationinsights 1.0.0-RC03 true
org.cloudfoundry.debug 1.0.0-RC03 true
org.cloudfoundry.googlestackdriver 1.0.0-RC03 true
org.cloudfoundry.jmx 1.0.0-RC03 true
org.cloudfoundry.procfile 1.0.0-RC03 true
org.cloudfoundry.dotnet-core-conf 0.0.20 true
org.cloudfoundry.archiveexpanding 1.0.0-RC03 true
org.cloudfoundry.tomcat 1.0.0-RC03 true
org.cloudfoundry.jdbc 1.0.0-RC03 true
org.cloudfoundry.springautoreconfiguration 1.0.0-RC03 true
org.cloudfoundry.springboot 1.0.0-RC03 true
org.cloudfoundry.distzip 1.0.0-RC03 true
I should further add that our slack https://slack.buildpacks.io/ is a great place to chat about all things buildpacks and https://hub.docker.com/r/cloudfoundry/cnb is a decent overview with links to a bunch of the CNBs.
I'll dig in a bit more because I think this is a really cool project:
A buildpack for environment.yml
https://github.com/cloudfoundry/conda-cnb
A buildpack for install.R
We have not written an R cnb but its somewhere in our backlog. a julia cnb is slightly higher in priority.
A buildpack for postBuild
We don't have this written, but I don't think it would be massively hard to implement
A buildpack for apt.txt
This will possibly be handled by https://github.com/buildpack/rfcs/pull/23
In case it's useful, here is a link to a recent post on the topic by Jose Diaz-Gonzalez, the lead developer of dokku, including some notes on how CNB tech differs between cloudfoundry, heroku and herokuish https://dokku.github.io/technology/comparing-buildpack-v3-to-herokuish
Is this something that could be used for new repo2docker buildpacks, e.g. Java? https://github.com/jupyter/repo2docker/issues/780
Hello :wave: I am a maintainer on the Cloud Native Buildpacks project. Happy to answer questions and provide support if y'all decide to go forward with CNB support :)
Just some links that might be useful in this context. You can also write CNBs in Python (if that reduces the integration barrier here). Here are some example buildpacks written in python -
https://github.com/samj1912/proc-descriptor-buildpack/blob/main/main.py https://github.com/samj1912/runtime-env-descriptor-buildpack/blob/main/main.py https://github.com/samj1912/label-descriptor-buildpack/blob/main/main.py
You can take these small pieces and compose them into a "meta-buildpack" if you want to which allows you to alias the combination of the above buildpacks in a simple-to-use wrapper - https://github.com/samj1912/project-descriptor-buildpack (this is mostly just a shell with an order file https://github.com/samj1912/project-descriptor-buildpack/blob/0978e93ab8e417a63baae9f9092c646b71685518/buildpack.toml#L14 (Note that all buildpacks here are optional so you could have 2**3 -1 valid combinations here which are automatically handled)
Link to CNB Python bindings library that was used to create these - https://github.com/samj1912/python-libcnb
Link to the official golang CNB bindings - https://github.com/buildpacks/libcnb
Our katacoda tutorials to help you get off the ground quickly (fully set up with CNB tools and an interactive walkthrough on creating a simple buildpack in bash) -
https://katacoda.com/buildpacks
EDIT - Here is a simple postBuild
buildpack - https://github.com/samj1912/postbuild-buildpack/blob/main/main.py
This issue has been mentioned on Jupyter Community Forum. There might be relevant details there:
https://discourse.jupyter.org/t/building-a-the-littlest-binderhub/9824/8
From @jchesterpivotal in https://github.com/jupyter/repo2docker/issues/707#issuecomment-505904267
I've bolded the bits that I think are most relevant to us. It would be great if someone could take a look at https://buildpacks.io to see if we can base repo2docker off v3 of buildpacks. http://words.yuvi.in/post/why-not-s2i/ contains reasons why we moved off s2i (which is similar to v2 of buildpacks).
A useful test case would be to try to make:
And then see how easy / hard it is to have a repo with any combination of these 4 files produce one single image. My rudimentary math skills tell me that there's
4!
possible combinations here (24), and we shouldn't have to write more than 4 buildpacks...