esl / release_poller

3 stars 2 forks source link

[feature] publish new repo tags to rabbitmq #2

Closed sescobb27 closed 6 years ago

sescobb27 commented 6 years ago

Feature in this PR

Publish new repo tags via RabbitMQ to run CI Jobs later in those new tags, e.g run docker builds

General Overview

High Level Architecture

https://github.com/sescobb27/release_poller/pull/2#issuecomment-413748024

sescobb27 commented 6 years ago

this is the architecture so far for pooling RabbitMQ connections and channels

screen shot 2018-08-16 at 5 24 06 pm

sescobb27 commented 6 years ago

i changed it a little bit, now when starting a connection worker we are going to start within it a pool of multiplexed channels to RabbitMQ and store them in its state (we can move this later to ets). Then, inside the connection worker we are going to trap exits and link each channel to it. this way if a channel crashes, the connection worker is going to be able to start another channel and if a connection to RabbitMQ crashes we are going to be able to restart that connection, remove all crashed channels and then restart them with a new connection; also we are going to be able to easily monitor client accessing channels, queue an dequeue channels from the pool in order to make them accessible by 1 client at a time making them race condition free.

screen shot 2018-08-16 at 8 40 23 pm

or more correct image (connections spawn channels and return their pid and we link the connection worker to those pids) screen shot 2018-08-17 at 7 51 36 am

sescobb27 commented 6 years ago

API so far for getting a channel from the pool is

BugsBunny.with_channel(pool_id, fn channel ->
  encoded_tags = Poison.encode!(new_tags)
  BugsBunny.RabbitMQ.publish(channel, "exchange", "new_releases.queue", encoded_tags)
end)