OSX workers

[supermarin]

Currently, Strider seems to be coupled with Docker for adding separate worker machines. The use case we have is - we need to be able to run iOS / Xcode jobs on separate workers, decoupled from the webapp machine.

Use case

Running iOS jobs on workers

Docker

While being a great solution for containing apps, Docker doesn't run natively on OSX, because containers.

Ideally

Worker machines should be as dumb and lean as possible. Thus, ideally all the complex software should be managed by the master webapp. Maybe using SSH vs webapp/sockets would do the job. We don't have problem deploying the webapp on a Linux box using Docker, as long as it's able to talk with non-docker workers.

At the end of the day, it should be as easy as possible to add and maintain the workers (no manual maintenance if possible)

[niallo]

This would be a nice feature. There is nothing coupling Strider to Docker, it just happens that the remote worker plugin which we have is for Docker.

We could just as easily have workers which speak HTTPS or whatever. Indeed, checkout https://github.com/Strider-CD/strider-ssh-deploy

[kfatehi]

Oh yeah, I forgot about strider-ssh-deploy, I wonder if I just tweaked it for it to run for the test phase if it would solve the problem.

[kfatehi]

@supermarin check this line out https://github.com/Strider-CD/strider-ssh-deploy/blob/master/worker.js#L3

if you change "deploy" to "test" there, you should be able to use ssh to conduct your tests :)

as far as achieving parallelism, there is a magic variable here that must be adjusted https://github.com/Strider-CD/strider-simple-runner/blob/master/lib/index.js#L63

those two tweaks should be all that is needed to hackily achieve what you are looking for. think you could try it in your setup ?

[kfatehi]

@niallo would the "right way" to do this be to create a runner plugin ? like strider-ssh-runner ? also i think that magic variable needs to get exposed on our global app config page we have been talking about in various issues @knownasilya

[niallo]

@keyvanfatehi yeah, a custom runner would definitely be the Right Way to go with this. Like you say, your existing strider-ssh-deploy plugin is basically already there! It wouldn't take much.

[supermarin]

@keyvanfatehi thanks for the pointers, will try and see what happens :)

Terminology-wise, I believe worker is more correct than runner; I'd expect that runner is the webapp,. think of it as master - slave, we're building a slave.

[kfatehi]

@supermarin well not quite, right? in the design we're thinking here, the runner is part of the web app and knows how to use ssh and sets all of that up. the worker is the remote ssh process that the runner is orchestrating or running for us. check out the code and see how things work, it will make sense. i hope you try hacking it into submission because it's very close. then i'll go make an official strider runner plugin, unless of course you beat me to it or something :)

[mbroadst]

+1

[hudongcheng]

@keyvanfatehi what about Windows? I want to use strider in windows native app. I need a runner (strider-windows-runner), which can build & test windows native app. Something like buildbot slave do.

[kfatehi]

@zemug im not familiar with buildbot, but it sounds similar to strider in many ways (very modular, user-tailored cicd solution) but written in python. what makes you want to use strider rather than buildbot?

for the record I think that a solution such as this should not really be based on ssh, but rather needs a remote-slave concept to work best.

[hudongcheng]

@keyvanfatehi I can't install buildbot, due to sqlalchemy-migrate break with sqlalchemy which buildbot depend on.

I have to due with svn & git, and build in windows, I think I should try install buildbot.

[kfatehi]

@zemug hmm, sounds like something they would be able to address via their issue tracker more quickly than we would be able to address by adding the desired functionality to strider. That said, I think we'll have this feature eventually, someone just has to want it badly enough to make a runner plugin. @jaredly may have some suggestions (and code) in case you want to take this on. Currently the closest example I know of is https://github.com/Strider-CD/strider-docker-runner -- it would be cool to have a cross-platform "docker server" which would implement just the server portion yet be able to spawn and control processes via JSON ... essentially this is the basis of the build slave: supporting the subset of features the docker runner expects of the docker server, but without actually using docker

[supermarin]

@keyvanfatehi docker isn't supported by OSX, so the closest you can get is a "vagrant server". I think this shouldn't be coupled with anything more than a SSH connection