Expanding to more chat platforms

[rikai]

I'd like to see ShowBot accessible on more chat platforms and eventually more generalized and not so JupiterBroadcasting-specific as a result, maybe moving the JupiterBroadcasting-specific plugins into their own subfolder or something.

I'll list a few i'd like to see:

Definitely

• [ ] Discord
• [ ] Twitch

Maybe

• [ ] Telegram
• [ ] Slack
• [ ] Youtube (Does this make sense?)

Maybe some others as well? Feel free to suggest more, just know that suggesting something doesn't necessarily mean it will be implemented!

I do realize all of the above with the exclusion of YouTube have IRC bridges, but i'd rather see them accessible via the native APIs where possible to allow for better integration.

Obviously not all plugins would make sense for all services, and some services probably wouldn't use most of the plugins, but might instead have new plugins of their own tailored to that audience. This would probably require the implementation of something to flag the plugins as being only for certain services.

If anyone here is interested in working on any of these, please let me know and i'll open up an issue with more details on things that make sense for the service you wish to work on.

Blockers

• 20 is going to probably be a long-term requirement for easily interfacing with these services APIs.

• 46 will also probably be a necessity as we expand the web side of things.

Considerations

• Our plugin infrastructure may need to become more abstracted, with the code-path being conditional based on what service is being accessed through what gems.

• We will probably also want to consider #40 and #63 when reworking these things.

• A much expanded #48 is probably also a consideration so communication can be a two-way street (as well as open up the possibility of third party apps and remote management)

[thefirstofthe300]

Given just how large this particular change is going to have to be. We should probably start working on a design proposal so we can come up with a sane architecture. Once we hash out the architecture, we can move onto actually building it. I just see the design proposal as a necessity; otherwise, the contributors are working towards a nebulously defined goal.

I could take a stab at a design proposal but I am a bit new to the software engineering side of things so it may take a while a rather large number of iterations before I come up with something sane.

@rikai sound good? Anybody else you want to loop in for architectural design decisions?

[eriknelson]

Saw this come across my feed, thought I'd pipe up :)

@rikai you're on the money with the abstraction; In an ideal world, JBots core logic should live in its own layer that is entirely agnostic to the various platforms, make sure platform specifics do not bleed into it. Leverage "adapters" or "plugins" to encapsulate those specifics by implementing interfaces, and make them configurable so they can be enabled and disabled easily. Makes onboarding new services entirely decoupled from JBot's core logic, since all you need to do is implement an interface. Additionally, maintenance becomes easier because you're able to update the discord adapter when they inevitably make a breaking change to their API without even opening jbot core logic. Also makes the platform adapters much more testable etc.

To be clear, I'm seeing a couple distinct abstract concepts here:

• Core plugins - These are cinch plugins; its where JBot's core business logic and features are coded.
• Platform adapters - These are the components that understand how to connect, talk to, and accept input from various platforms (IRC, discord, telegram)

Cinch complicates things. To get to ^ that, you need to decouple cinch from IRC. Cinch is great because it delivers the core plugin architecture for extending JBot behavior. If I had no constraints, I would fork cinch, rip out IRC specifics, and hack it up to be the core framework for implementing JBot behavior.

On top of this, you probably want some kind of "router" between the adapters and the core plugs that acts as the traffic conductor. Message comes in from an adapter, make sure it gets stamped with appropriate identifying information so that when the core responds, it says to the router "hey, send back this response to the place where this message came from", router looks at it, figures out which adapter it needs to ship things off to, and says to the correct adapter "hey, send this message in whatever specific way you need to (platform APIs)"

As far as actually doing this, I would do it phased:

I) Refactor the code base with this design. Clean, well defined layers, and IRC logic pushed into an adapter. Configure JBot to just use the IRC client for now, and get that into a test environment to bang on for regression testing. If it performs as it does today and has an obvious way forwards for implementing a discord interface, that's success. II) Start implementing adapters to other services and iterate on the core design.

So summary:

• JBot, platform agnostic behavior plugins (possibly cinch based?)
• Platform specific adapters for IO.
• Router sitting in between the two brokering comms

This doesn't even take into account identity authorization and authentication. For that, I would definitely delegate that to another service and hopefully use OAuth (or implement a dead simple, JBot identity system)

I realize this is almost certainly not feasible given the eyes on the project right now, but that's my brain dump in a perfect world :smile:. It's such a departure from what exists, I would almost throw things out and start from the ground up. JBot 2.0, but designed to be a generic solution.

[thefirstofthe300]

I agree with @eriknelson for the most part. In a perfect world, we would build most everything from the ground up since all of JBot's business logic exists in Cinch plugins which inherently constrains it to IRC.

One question I have, If we are doing a whole-cloth rewrite, why not flesh out a fully-designed architecture based on our needs and future plans and investigate other servers/libraries that could potentially replace Cinch. Doing so would allow us to evaluate what technical debt we currently have that we could get rid of while simultaneously allowing us to prevent taking on and maintaining codebases that aren't ours.

I see little advantage to gutting, rebuilding, and maintaining a fork of Cinch if there are other platforms that would be better suited to our needs/wants. We do have limited resources. 😃

From what I can see, what we really want is more along the lines of a server that we can interact with via HTTP for the frontend while simultaneously providing an easy way to interface with IRC, Discord, Twitch, and other chat platforms.

I propose we re-architect based around an HTTP server backend, move as much of the business logic and state into the HTTP server as possible, and build a new bot for each service we wish to support.

In my mind, if we use an HTTP API as a backend for data storage and authentication, we immediately unblock a whole host of new clients that can interface with JBot. We can spin up a web interface with minimal effort and since bot frameworks are a dime-a-dozen for each new platform and are generally capable of handling their own lifecycles, we can theoretically add a new platform to JBot in O(days) by choosing a bot framework and rolling out the code with the existing commands and HTTP calls.

As long as we design a sane API from the start, we can rollout new features to each bot by adding a new command that with a basic HTTP call and minimal processing on the bot end. Heck, we could probably even Dockerize JBot entirely for easy deployment and maintenance.

I see this whole-cloth rewrite taking on the order of six to bring it to feature parity with the current implementation if one person works on the code off and on. It could probably be done much quicker if we can get several people rolling on it.

Thoughts, suggestions, criticisms?

[eriknelson]

One question I have, If we are doing a whole-cloth rewrite, why not flesh out a fully-designed architecture based on our needs and future plans and investigate other servers/libraries that could potentially replace Cinch.

That's exactly what I'm suggesting.

I see little advantage to gutting, rebuilding, and maintaining a fork of Cinch if there are other platforms that would be better suited to our needs/wants. We do have limited resources.

+1 all around

I propose we re-architect based around an HTTP server backend, move as much of the business logic and state into the HTTP server as possible, and build a new bot for each service we wish to support.

That sounds like microservices. I strongly feel like that's a mistake.

I'm picturing a few systems, maybe you're seeing the same thing @thefirstofthe300.

• api.jbot.com - data server and all around core app sitting in front of the DB
• id.jbot.com - identity server, where accounts and auth actually take place
• www.jbot.com - the web server, serves our clients that talk to the api

Thoughts?

[thefirstofthe300]

I was picturing the same general architecture as you, Erik, although we may be disagreeing on certain details.

api.jbot.com - data server and all around core app sitting in front of the DB

Can you expound on what you mean by "core app"? Is this going to be the bots?

id.jbot.com - identity server, where accounts and auth actually take place

Probably a very good idea to separate the auth and accounts from the server for security purposes.

www.jbot.com - the web server, serves our clients that talk to the api

Isn't the API server already a web server in this design? Isn't this technically redundant?

I was picturing an architecture more along the lines of

• api.jbot.com - this contains the business logic and data storage routines
• id.jbot.com - no questions here
• www.jbot.com - the web frontend built with a web framework like Angular or React (I'd personally prefer Angular) that makes calls to api.jbot.com. This frontend could theoretically be turned into an electron app and would be a fully-featured client.
• IRC bot - existing Cinch bot refactored to make calls to api.jbot.com
• Discord bot - under-development: uses the same API calls as the Cinch bot

The biggest issues I see:

• Duplication of code in the bots;
• The introduction of more binaries and thus more moving parts/potential for breakage

Where I see the biggest advantages:

• We free ourselves of a large amount of technical debt.
• Bot and client releases can be independent of the backends.
• Bots can take advantage of existing, unmodified frameworks for ease of maintenance.
• jBot would have smaller failure domains, i.e. a single bad bot release doesn't hose the rest of the clients.

Most of this design has been gleaned through my observation of a highly successful web company's choice of design paradigms; however, I realize that this company has an immeasurably greater amount of resources to pour into their designs. If I am missing something that makes this design impractical for a smaller project, please feel free to point out the error of my ways. 😄

I am very curious to know your thoughts on why a "microservices" (wth does that even mean???) architecture is a bad idea for jBot?

[thefirstofthe300]

Thinking about my reply, I may come across as passive-aggressive. That's not my intent (I need to work on people skills). I am just curious to know everyone's criticisms of my proposed architecture so we can all come to an agreement on how to best move forward.

I'm not trying to name drop either. I just have observed this particular architecture design to be extremely effective for the company I am contracted to. They are in the same situation of trying to provide an open API while trying to integrate existing software with new software.