Separation of API server and Dyno server

[tombh]

I've pushed some branches that successfully separate the API server and the Dynohost server. See;

• vagrant-openruko
• api-server
• dynohost Diff is mostly whitespace changes, only notable things are that the dyno's /etc/hosts file inherits from the server's, and a tweak to the tests that casts the dyno_id to a string.
• httprouting
• Integration-tests

So now there are 2 Vagrantfiles in vagrant-openruko, the one in /standalone is as before, it allows you to build Openruko on just 1 server as usual (and all the tests work as normal). Then there is another Vagrantfile in /cluster that boots up 2 VMS, :api and :dynohost, if you then run the tests in the :api VM all dynos are created on the other server! It's like magic! Clearly this is hugely significant for scalability.

I haven't pushed these to any master branches yet because I'd like your opinions on how this is implemented and it'd be good to have another pair of eyes sanity check everything.

tom :)

[Filirom1]

Wooo great ! I will look at it right now.

[Filirom1]

Good :+1:

[Filirom1]

It will fix this issue:

Add dyno distribution (HTTPRouting and commands such as "run bash" should work with disitributed dynos but are still implemented in a way that requests that dynohost and the apiserver be in the same machines);

[tombh]

Thanks Romain! Yes, it does fix that issue that the Azuki folk highlighted - all the tests are passing :)

Thank you so much for your comments. I will tidy up the code (especially the slobox refs, I keep doing that, sorry) and then merge in the next few days.

[nuxlli]

@tombh our ideas for improvements are very similar to what you have already stared implementing.

My main concern right now regards our development cycle, including this recent pull request: https://github.com/openruko/vagrant-openruko/pull/11

Besides that, there are many feature branches and issues in different projects, which makes my head hurt at times :)

I still don't have a clear picture of how we could improve this scenario. However, as for now I thought of changing the Vagrantfile:

...
config.vm.customize [
  "setextradata",
  :id,
  "VBoxInternal2/SharedFoldersEnableSymlinksCreate/v-root",
  "1"
]
...
chef.json["openruko"]["home"] = "/vagrant/openruko"
...

I'm still not sure about the resulting integration tests or about its execution over Travis CI. Any suggestions on this are very welcome.

[nuxlli]

Did you manage to use the Vagrantfile.local file? I can't seem to customize the chef configurations in my own tests. According to what I've read, the proxy elements on cluster/Vagrantfile may not work either.

Finally, according to the link below, config.vm.provision can only be called once by each VM.

http://stackoverflow.com/questions/13065576/override-vagrant-configuration-settings-locally-per-dev

[tombh]

Okay, so I've gone and merged everything now, for all the repos, that should make things easier.

Yes, I know what you mean about all the branches and repos, it's hard to keep up. Also, I k now what it's like having to then maintain all those repos in a different project, as in Slotbox for me and Azukiapp for you!

One thing I've been thinking about is merging Codonhooks and Rukorun into the Dynohost repo, that would get rid of 2 repos.

I don't quite understand that change tot he Vagrantfile you're suggesting.

I only use Vagrantfile.local to mount my own custom folder at the moment. Though I assumed I could also use it to scale the number of Dyno server instances. But from what you're saying is that possible? Or is it that you want Vagrantfile.local to override the provisioner to use AWS?

The state of the Vagrant files at the moment is not so important, the main thing is that the API and dynos have been separated :) So pleas have a go at experimenting with the Vagrantfile layouts.

I have disabled the Travis CI integration as it no longer supports LXC, very sad indeed :'( So I've been trying to get integration tests working on drone.io, but without much luck yet.

[rogerleite]

@tombh great job! Just to register, to work here i have to edit these files:

cookbooks/openruko/attributes/default.rb
-default['openruko']['repositories']['httprouting'] = "https://github.com/Filirom1/httprouting.git
+default['openruko']['repositories']['httprouting'] = "https://github.com/slotbox/httprouting"

solo.json
     "versions" : {
-      "apiserver": "v0.1.2",
+      "apiserver": "8784e4434c17d6adad490d87c182f269b93290d2",
       "client": "v0.1.1",
       "codonhooks": "v0.1.1",
-      "dynohost": "v0.1.1",
-      "httprouting": "16e5c4011938f3964ed3bcaadca30fb58851f3a1",
-      "integration-tests": "v0.1.1",
+      "dynohost": "2f4240b35aa63c6cc86b41fe127619d80e59bc05",
+      "httprouting": "e3d437d9abba70f5de451156bc0ca58e48950b2e",
+      "integration-tests": "cb09e20f3543e66af9b9b1bb2f8d674face470a2",
       "logplex": "v0.1.0",
-      "gitmouth": "v0.1.1",
+      "gitmouth": "v0.1.2",
       "rukorun": "v0.1.1"
     },

:shipit:

[tombh]

That should all be fixed in the latest master. Httprouting is part of Openruko org now and is referenced from the default attributes, so no need to use Slotbox's version. I've also updated all the tags and linked to them from solo.json.

Basically you shouldn't need to change anything from what's in master now :)

[rogerleite]

@nuxlli saw my comment and told me that you update everything on master ... @tombh thanks for the answer.