CI: Deploy docker container

[om26er]

Currently on each commit, autobahn-python publishes a new docker container. We need to do a similar treatment for crossbar as that would make using the bleeding edge on different targets very easy.

Is there a reason we don't do that today ?

[oberstet]

well, we do that .. even though it is currently broken again, because of changes:

[oberstet]

ah, right: this is a private repo.

[om26er]

So what would the fix look like ? As a starting point would it make sense to enable docker container builds (not publish) after a PR lands ?

[om26er]

Also, is it correct to assume that the current docker tooling for crossbar is not open-source ?

[oberstet]

ok, actually I guess we should discuss this from a DAO angle ... there are also reasons why the binaries producing stuff is split out ...

[oberstet]

so one idea/proposal is the following:

• from above repo, split out the parts that produce CB binaries, eg Docker, snaps and EXEs
• this artifacts would be specified in a WAMP Enhancement Proposal (WEP) as outlined (well, mentioned at least;) in https://github.com/wamp-proto/wamp-proto/pull/384
• the binaries are published by the DAO and have a separate License/EULA
• the binaries will have the code for page 2 https://github.com/wamp-proto/wamp-proto/blob/457b92a47e2615d4cdf67dee07d82eceb054bb94/wep/wep002/wamp_ap_r2r.pdf - that is, enable cross-domain rlinks via on-chain trust anchors
• the existing design IP and code is donated (by CB company) as part of a WEP (after the DAO has been established) and is supposed to attract/gather financing

what do you think?

[om26er]

@oberstet This sounds good to me and is something that is clearly needed as a first step for moving things forward (after the establishment of the DAO of course).

Needless to say, I am currently working to make sure that HA + point-to-point integration are really rock solid in crossbar i.e. performant + reliable. And having a release out as quickly as possible will allow me to get more testing of the changes that I have been working on.

As a stop gap, does the license allow to publish a crossbar docker container under my name either on dockerhub or github registry ?

[oberstet]

cool, this sounds great! of course contributing and helping with "rlinks / performant + reliable" is highly appreciated! I think it'll be widely used, and is indeed probably the single most important feature people have been asking for.

after the establishment of the DAO

yes, this should now happen quit soon (DAO & Swiss Assoc.). I've indeed been contacted by "others" that also want to sponsor the project and/or features and are basically waiting for a legal entity to exist. so yeah;)

As a stop gap, does the license allow to publish a crossbar docker container under my name either on dockerhub or github registry ?

not sure I get what you mean exactly ... however, the IP to DAO transfer should be done within the next weeks (legally). we then have to split out the Docker/snap/EXE into a separate public repo and add license/eula.

[om26er]

not sure I get what you mean exactly ... however, the IP to DAO transfer should be done within the next weeks (legally). we then have to split out the Docker/snap/EXE into a separate public repo and add license/eula.

what I meant was that: would it be "reasonable" if I create a docker container for Crossbar using the master branch, so that we can use it (before the DAO and IP transfer happens).

[oberstet]

ah, sure, that's fine of course! it's OSS, though duplicate work (to create docker scripts). just don't redistribute the docker images to the outside world or as part of a distributed product please. this is mainly to preserve the DAO built and "official" binaries option. however, strictly speaking, this is really more sort of "please", less formal legal necessity (the code is OSS with defined license, and we retain the trademarks) ...

on another note, once we have Docker, snap and EXE transferred to the DAO, and have the DAO build and publish official binaries, a natural question would be: what about the CB Yocto layer?

[oberstet]

though duplicate work (to create docker scripts)

this really sucks, and hence I've split out the relevant parts to a private (for now) repo

https://github.com/crossbario/crossbar-binaries

and invited you with write access. you can use this along the lines described above ^ ...

the GH build scripts (the dirs therein) / matterhorn buildslave association will need adjustment .. but the build scripts / tooling is all there. will double check ..

[etoccalino]

Hi all. Any news about updating the build+push process of docker images for crossbar? The current latest is from Jan 2021. If fixing/changing the CI system is out of scope, then any (up to date) instructions on how to build it that can be shared, that would be very helpful too.

[oberstet]

latest docker is 21.11.1: https://hub.docker.com/repository/docker/crossbario/crossbar/tags?page=1&ordering=last_updated&name=21.11.1

does that work for you?

[etoccalino]

crossbario/crossbar:cpy-slim-amd64-21.11.1 is a much newer version than latest, thanks for the link! We'd like to stay current with both autobahn and crossbar. If we wanted newer version (e.g. 22.1.1), should we install via pip (i.e. build our own docker image)? is there an alternative I'm missing?

[oberstet]

ideally, you should use a binary release, means docker, snap or single-file exe.

we also have builds for this, but it needs to be triggered for releases. working on that;) also, this stuff is not easy to keep working all the time. anyways, if you really want to keep up with latest master:

• I use a plain python venv and run crossbar from a "pip install -e" dev install .. I can run uncommitted dev mods. probably too hard core for users;)
• the github CI in this repo, once a change has been merged to master, will also auto build and deploy a python wheel:

https://crossbarbuilder.s3.eu-central-1.amazonaws.com/wheels/crossbar-latest-py2.py3-none-any.whl https://github.com/crossbario/crossbar/runs/5204047332

this is quite good and easy to use! as long as you have all dependencies;)

[etoccalino]

I see. We'll stay tuned for the docker releases, then. In the meantime we'll probably build our own images (our crossbar is running on k8s). Thank you very much for the wheel link and pointers on how to build with pip, that's going to help us out a long way.

[oberstet]

as described, this will be addressed via the DAO / https://github.com/crossbario/crossbar-binaries - however, since we need the Docker images ourself, currently we're refreshing the images on Dockerhub manually quite regularily