Pull Agents into Ruby Gems

[cantino]

Let's start this architecture discussion.

I think Huginn Agents should be distributed as ruby gems, inheriting from a huginn-agent gem containing the base models and agent behaviors. Agent gems should also be allowed to distributed partials and controllers for any custom agent behavior; as the agent model grows it will make sense to distribute dashboards and custom UIs for some types of agents.

What is the best way to organize a Rails system like this? Engines?

[albertsun]

Will having agents as ruby gems make the process of writing a new agent easier or harder?

While I think pulling agents out of Huginn into separate gems would be right for the long term, right now it adds a lot of complexity that will make it too hard for people to get started using it.

I'd favor having a broad set of useful agents and sample flows built into "core" so that people can start using Huginn immediately out of the box without having to go out and find and install any additional gems. Look at the number of different triggers that ifttt has, Huginn is not even close right now.

[cantino]

That's a fair point @albertsun. I've been investigating the process involved in turning Huginn itself into a Rails::Engine so that it could be mounted inside of another Rails app, installed with Bundler, extended via gems, etc.

I agree that this is experimental for a while, and I won't ask everyone to transition their Agents yet.

One thing I'd like to do is add a concept of Huginn Dashboards/Widgets that visualize data from one or more Agents. Eventually, it will make sense for Dashboards/Widgets to also be gems themselves so that they can be required via Bundler, can declare their own Agent dependencies, etc. In the shorter term, perhaps I'll just add them to core.

[0xdevalias]

Related: #41 #132 #215 #317

[CloCkWeRX]

So I think one of the key things to do is the break the coupling to Activerecord - persisting your settings has very little to do with receiving or handling events.

Refactoring 1: You can basically grab all of the class methods from Agent and boot them somewhere else, as none of the child agents are going to redefine these methods.

class MyAgent
   include AgentBehaviours

   cannot_be_scheduled!
end

Refactoring 2: Anything labelled Implement me in your subclass of Agent. should probably be pulled out too, and turned into AgentBehaviours / Reflection probably should be used to loosely check that it fulfils the basic API.

Refactoring 3: description & event_description are views/templates. They are useful as API documentation and useful appearing in the UI at the moment, but fit a lot better under app/views/agents/whatever/_description.md and app/views/agents/whatever/_event_description.md

Refactoring 4: Introduce a component pattern to organise the views. I'd recommend something like http://shopify.github.io/dashing/ 's approach because:

• It's solved the component problem enough to be usable
• It's got a similar concept of scheduled jobs passing data to a frontend - you could completely separate the frontend (graph display) from the backend.
• There's a load of pre-made widgets (https://github.com/Shopify/dashing/wiki/Additional-Widgets)

Examples:

• https://github.com/danillotuhumury/klimato-dashing-widget
• https://github.com/mtchavez/dashing-circleci/blob/master/jobs/circleci.rb

Refactoring 5: Achieve API parity re scheduling:

SCHEDULER.every('1m', { first_in: '2s', allow_overlapping: false }) do

which I can't imagine being too hard, since both projects rely on rufus-scheduler!

[CloCkWeRX]

Oh wow; dashing simply does:

SCHEDULER = Rufus::Scheduler.new

and loads a bunch of jobs which fire off:

def send_event(id, body, target=nil)
  body[:id] = id
  body[:updatedAt] ||= Time.now.to_i
  event = format_event(body.to_json, target)
  Sinatra::Application.settings.history[id] = event unless target == 'dashboards'
  Sinatra::Application.settings.connections.each { |out| out << event }
end

and

def format_event(body, name=nil)
  str = ""
  str << "event: #{name}\n" if name
  str << "data: #{body}\n\n"
end

Adding methods to push this into the huginn backend or use HuginnScheduler would be pretty easy; just a bit of extra autoloading magic.

[cantino]

I mostly agree with these refactors. We're definitely getting to the point where some Agents should be extracted into gems, especially those that carry many dependencies.

I agree about the API extraction.

I'm a bit torn about pulling the descriptions out. I mostly agree in principle, but also appreciate seeing them when browsing the agent directory.

On Sunday, June 8, 2014, CloCkWeRX notifications@github.com wrote:

Oh wow; dashing simply does:

SCHEDULER = Rufus::Scheduler.new

and loads a bunch of jobs which fire off:

def send_event(id, body, target=nil) body[:id] = id body[:updatedAt] ||= Time.now.to_i event = format_event(body.to_json, target) Sinatra::Application.settings.history[id] = event unless target == 'dashboards' Sinatra::Application.settings.connections.each { |out| out << event } end

and

def format_event(body, name=nil) str = "" str << "event: #{name}\n" if name str << "data: #{body}\n\n" end

Adding methods to push this into the huginn backend or use HuginnScheduler would be pretty easy; just a bit of extra autoloading magic.

— Reply to this email directly or view it on GitHub https://github.com/cantino/huginn/issues/60#issuecomment-45466099.