search

orbitalci / orbital

Orbital is a self-hosted CI system for solo/small consulting dev teams. Written in Rust.
GNU General Public License v3.0
29 stars 2 forks source link

Poller docs - Basic deployment + configuration steps, NSQ interaction with topics #201

Closed tjtelan closed 4 years ago

tjtelan commented 5 years ago

More documentation needed around poller deployment and configuration! I'm writing this issue w/o looking at the poller code, only what is in the wiki. Poller isn't included in the Vagrant setup either for this reason (#177).

Background:

From our existing production deployment, our NSQ deployment had poll_please and no_poll_please topics. When I had to redeploy NSQ, these topics were deleted. (Wah wah)

I had to restart Poller because it was throwing errors after losing these topics. The first thing Poller does is call github.com/shankj3/ocelot/build_signaler/poll.WriteCronFile and re-configure the cron timers (which is inline with my expectations!). Then rolls into this double sleep loop looking for topics in NSQ.

{"function":"main.consume","level":"info","msg":"about to sleep for 10s because could not find topic no_poll_please","time":"2019-02-13T03:24:12Z"}
{"function":"main.consume","level":"info","msg":"about to sleep for 10s because could not find topic poll_please","time":"2019-02-13T03:24:12Z"}

I'm unsure if this is expected behavior because no one has created new polling rules? (which would make sense if admin initiates creating the topics), but we need more docs to understand how this all strings together.

In particular, this needs to be paired with a detailed walkthrough for how to deploy in polling-mode or a webhook-mode. The audience for this doc is an administrator (as opposed to a regular user)

tjtelan commented 5 years ago

This ended up being very similar to the other components. The biggest difference is needing poller to run as a privileged user, since it directly touches files within the /etc/cron.d directory.

This runs within the Vagrant environment now.