Support multiple orgs on the same instance

[engelgabriel]

Transparently divide a single server into several different instances shared between different groups of users.

Evaluate https://atmospherejs.com/mizzao/partitioner

[mhurwi]

Hi! We might be interested in helping on this one.

Is it to be the same 'teams' feature as on Slack? Has anyone started on it yet?

[Sing-Li]

Cool. Please join the #contributors channel on the demo chat. Can definitely use the help, especially if you guys have experience in working with mizzao:partioner already!

[mhurwi]

Not used it, but reading the docs and discussion, seems like a nice solution. Will see you in the chat room soon!

[Sing-Li]

Great, @mhurwi ! Catch you on demo soon. Can't wait to see what sort of real world performance hit partitioning brings. Day job's got me running 'round in circles on a Sunday :grinning:

[engelgabriel]

THe good news is, I just talked to 2 companies that want this feature so bad, they will allocate their own developers to contribute and build it :)

One of the companies if from around the block, so they will just come to the office for a few days and work from here.

[geekgonecrazy]

:+1: this is awesome

[Sing-Li]

Haha what a team! @mhurwi , you just got yourself some teamates from the real sunny south! :grinning:

[liuliming2008]

Partition is the same as "team" at slack? At slack, one user(email)can create and join multiple teams, can can switch team. If Partition is only a solution like load balance, rocket cloud add team feature at one instance first.

[mhurwi]

One thing about mizzao:meteor-partitioner: it seems set up to allow one user to belong to only one group.

https://github.com/mizzao/meteor-partitioner/blob/master/grouping.coffee#L19-L23

Partitioner.setUserGroup = (userId, groupId) ->
  check(userId, String)
  check(groupId, String)
  if Grouping.findOne(userId)
    throw new Meteor.Error(403, "User is already in a group")

This makes sense, compared to Slack. From my experience, Slack requires a unique new password for each team. That could mean that each team represents a new instance and behind the scenes, on Slack I have a separate account for each team I belong to.

Maybe using meteor-partitioner would require creating a new account for a user when they want to join a new group?

Not sure what you all think about this?

[Sing-Li]

Just copying a post from the other thread (use a context id):

Can we 'virtualise' the real email + an appearance id ... and then gen a unique guid email to be used by meteor?? just a thought

the appearance id might just be a 'context identifier', from which white-labeled instance (or a team name) the user is logging in from (as), for example:

(singli@hisemail.com + 'rocketchat team') -> kskfjsafosajdfasf@dsalfsakfaslf.com (singli@hisemail.com + 'sings own family chat') -> o24i3i435o35345@oi324323432.com

[engelgabriel]

I think we should look into mizzao:partioner, but not use it, just understand it and apply some of the logic into our project. The main reason being that we want this behaviour to be optional. Some organization will just want a very simple Rocket.Chat deployment, for a single team. So we need to add the hooks on the right places, and only activate them via settings.

[liuliming2008]

I suggest to add the "team" feature as optional settings. And then consider how to expand one instance to more.

[geekgonecrazy]

@liuliming2008 that's a completely separate thing and is being tracked here. RocketChat/Rocket.Chat#520

High Availability / Scaling to multiple instances.

[engelgabriel]

Lets implement that using the new abstraction layer at Rocket.Chat/packages/rocketchat-lib/server/models/_Base.coffee

[engelgabriel]

Althou we will use the word "TEAMS" to denominate the enviroment variable, the property name on the schema should be configurable, so we can keep compatibility with https://atmospherejs.com/mizzao/partitioner as they use _groupId

[joaocarloscabral]

Hello guys. I'll work on this issue on the next days! As @engelgabriel said before, we have a product that needs this feature and will be a pleasure for me to help.

Yesterday I was talking with @engelgabriel and @rodrigok about the first steps to develop and the suggestion was to start changing this file: https://github.com/joaocarloscabral/Rocket.Chat/blob/master/packages/rocketchat-lib/server/models/_Base.coffee

The idea is intercept all calls to DB, add a property named as _teamId and check if the property was not changed from the client.

Right? :)

[engelgabriel]

@joaocarloscabral Take a look on the link below for how to grab the connection object for that fiber. https://github.com/RocketChat/Rocket.Chat/compare/EnvironmentConnection#diff-e214084ef39ef04ea8cfce4bd72a420bR26

[geekgonecrazy]

@engelgabriel when I think teams i'm thinking of an organizational unit. A way to say this user is part of this team, this channel is part of this team. A way of visually organizing and displaying things. But essentially its still all in the same organization, and same instance and they could directly interact with other teams.

Like Company X has:

• Sales Team
• Marketing Team
• Engineering Team

Etc, but they can still talk in general shared channels.

When I read through this thread, this looks like you guys are talking multi-organizational support? Or partitioning of a single server to be able to take care of multiple organizations. Completely separate but on the same instance.

I'm I correct in thinking this is actually multi-organization support not really teams?

[mhurwi]

We are in favor of 'multi-organization'. We use slack this way-- each team is a different customer organization that we manage. These different organizations do not talk to each other.

'sales team' and 'marketing team' seem like people that belong to the same organization. They can use private channels to stay separate, but they can still direct message whoever they want and also join in the public channels.

[geekgonecrazy]

@mhurwi right that makes sense that you would have the option to separate out to a multi-org if you want isolation between teams.

@engelgabriel I'm going to go ahead and change this title so we its a little more clear.

[liuliming2008]

hi @joaocarloscabral and @engelgabriel: If do ti at Rocket.Chat/packages/rocketchat-lib/server/models/_Base.coffee, can not support one user create or join mutiple organizations like slack, right?

[engelgabriel]

@liuliming2008 We will do it in a way to allow a user to join multiple organizations... or I didn't understand you question...

[liuliming2008]

@engelgabriel I see you said "Lets implement that using the new abstraction layer at Rocket.Chat/packages/rocketchat-lib/server/models/_Base.coffee", I think this cloud be not the way to allow a user to join multiple organizations. You now do it in another way, and I have no question. thanks

[artisin]

I have not had the time to sit down and really look at RocketChat's code base but would a implementation that leverage minimongo for latency compensation be a better solution? Correct me if I'm wrong but if we implement a solution of that sort it would grant us a bit more flexibility in terms of privileges and roles without making a potentially expensive server call.

[Sing-Li]

Please elaborate. Interesting idea. Are you suggesting 'client side' only multi-teams?

[engelgabriel]

@artisin Meteor apps use minimongo for latency compensation as default. Can you elaborate?

[artisin]

Sorry, but after briefly looking at how you handle things I'm going to say disregard my previous comment. Although, to make sure I'm on the same page here the plan is to create a new LiveResultSet for each team?

[rodrigok]

@artisin, users will access one team per connection, so if you access the chat application in 2 tabs you can access 2 different teams in the same time using the same account.

[artisin]

Thanks @rodrigok. Elegant solution, compared to a alternative like publication overloading.

[engelgabriel]

Hi @joaocarloscabral

I am back from holiday! :) Can we create a PR for the changes done to date?

[neurobe]

There will be "Teams" delimited by Role (ie delimited by the impending Moderator/Supervisor role who might have Admin permissions over a set of Rooms) and "Teams" delimited by Partitioning. Given that, I think "organisation" is a better term than "teams" for the latter. In my understanding, for Partitioning, each instance will have its own badge, which might say "Team No1" or it might say "General Electric Boston". Happy to have my understanding corrected.

The former seems straight forward to implement. The latter seems a brave new world. Love to have both of course.

[engelgabriel]

We agree that "organisation" is a better term than "teams". We may have both in the future, but this issue is related to "organisation". The main difference being that users/rooms/messages from "organisation" would be completely invisible/isolated to users/rooms/messages from another "organisation" on the same server. What we are trying to achieve here is a multi tenant database architecture with shared database and shared schema.

[bnusunny]

We are also working on the multi-tenant support and hit a road block. We need your advise.

We started with mizzao:partitioner package and partitioned all the models in _Base.coffee.

packages/rocketchat-lib/server/models/_Base.coffee:5

    _initModel: (name) ->
        check name, String

        @model = new Meteor.Collection @_baseName() + name

        Partitioner.partitionCollection @model

However, Rocket.chat can't run with this change.

W20151129-14:13:33.309(8)? (STDERR) 
W20151129-14:13:33.309(8)? (STDERR) C:\Users\Harold\AppData\Local\.meteor\packages\meteor-tool\1.1.10\mt-os.windows.x86_32\dev_bundle\server-lib\node_modules\fibers\future.js:245
W20151129-14:13:33.309(8)? (STDERR)                                             throw(ex);
W20151129-14:13:33.309(8)? (STDERR)                                                   ^
W20151129-14:13:33.309(8)? (STDERR) Error: Must be logged in to operate on partitioned collection [403]
W20151129-14:13:33.309(8)? (STDERR)     at Object.findHook (packages/mizzao_partitioner/grouping.coffee:153:15)
W20151129-14:13:33.309(8)? (STDERR)     at packages/matb33_collection-hooks/find.js:12:1
W20151129-14:13:33.309(8)? (STDERR)     at Array.forEach (native)
W20151129-14:13:33.309(8)? (STDERR)     at Function._.each._.forEach (packages/underscore/underscore.js:105:1)
W20151129-14:13:33.309(8)? (STDERR)     at Object.CollectionHooks.defineAdvice.self (packages/matb33_collection-hooks/find.js:11:1)
W20151129-14:13:33.309(8)? (STDERR)     at Object.collection.(anonymous function) [as find] (packages/matb33_collection-hooks/collection-hooks.js:117:1)
W20151129-14:13:33.309(8)? (STDERR)     at [object Object]._.extend.find (packages/mongo/collection.js:283:1)
W20151129-14:13:33.309(8)? (STDERR)     at _Class.RocketChat.models._Base._Class.find (packages/rocketchat_lib/server/models/_Base.coffee:14:22)
W20151129-14:13:33.309(8)? (STDERR)     at __coffeescriptShare (packages/rocketchat_lib/server/startup/oAuthServicesUpdate.coffee:51:28)
W20151129-14:13:33.309(8)? (STDERR)     at packages/rocketchat_lib/server/startup/oAuthServicesUpdate.coffee:1:1

Checking the code, we found that models methods were used during Meteor startup. At that point of time, no user was logged in and can't determine the tenant.

packages/rocketchat-lib/server/startup/oAuthServicesUpdate.coffee:51

RocketChat.models.Settings.find().observe
    added: (record) ->
        if /^Accounts_OAuth_.+/.test record._id
            oAuthServicesUpdate()

    changed: (record) ->
        if /^Accounts_OAuth_.+/.test record._id
            oAuthServicesUpdate()

    removed: (record) ->
        if /^Accounts_OAuth_.+/.test record._id
            oAuthServicesUpdate()

In our case, different tenants should be able to maintain their own settings besides rooms/messages/users. Is it possible to move these tenant related configuration code outside of Meteor startup and call them later when tenant information is available?

[steedos]

Hello guys, we are working on a admin dashboard project, plans to support multiple teams on the same instance.

https://github.com/steedos/admin

We separate users to spaces, one user can join multiple spaces. We'll be glad to integrate this to RocketChat as an Admin UI for Space Owners.

[artisin]

@bnusunny I ran into that same error a while back while working with Partitioner did you try wrapping it in the Partitioner.directOperation method?

[IQ2022]

how about multiple sites, different groups -- transparent to site users (each site user stay within their group)

[liuliming2008]

@steedos I can say that: the space equals channel at slack/rochekt.chat, and organizations in your screenshot equals organization at slack/rochekt.chat?

[steedos]

@liuliming2008 , sorry for reply late.

Spaces equals teams in slack, which may have a domain name Space_Users contains users joined the space, one user can join multiple space. Organizations is departments inside space, it is a tree model, user can in multiple organizations

Space has a owner and admins Space admins can modify space users and organizations Space admins can add space users via email, if email not exists in users collection, system will create a new user, send invitation can add the new user to space.

some api references here. https://github.com/steedos/api/tree/master/core

[liuliming2008]

@steedos: Good project. Spaces means company or teams in slack, which may have a domain name. And Organizations means departments inside space, a tree model. If can integrated with RochetChat(admin can create channels with organization members, not one by one), it will more flexible on channel member management than Slack. I dont think rocket team will do similar work recently, "Support multiple orgs on the same instance " is hard work and they need time to improve this I think. Looking for your demo work.

[steedos]

@liuliming2008, thanks for your advice. The organization can also be used to browse users by department in tree view, for large organizations. the admin panel is ready to work now, i will try to integrate it with rocket.chat in the next month.

[srocke]

@steedos How is this coming? Wanting to do multi-tenant offering (different companies using different domains) and curious as to the progress. Thanks.

[jalagrange]

This would be amazing! Any updates?!

[ToxikCZ]

+1 Do you have some estimate for release?

[Megatronic79]

+1

[liuliming2008]

very difficulty, almost need to rearchitect, let's wait

[srocke]

How is this so difficult?

Sent from my iPhone

On Feb 3, 2016, at 1:06 AM, liuliming2008 notifications@github.com wrote:

very difficulty, almost need to rearchitect, let's wait

— Reply to this email directly or view it on GitHub.

[dalareo]

+1000 @steedos how can I integrate rocket.chat in Steedos Admin??

[IQ2022]

so we can have one admin or site owner to manage the different groups and allow who can view a shared rooms (common rooms) - in the same it can be transparent, one challenge i may think of is the URL address probably we need to come with something fancy, to change it or probably dedicate a URL just for Chat?

Also how about oauth would that still apply, users from one site can't access the chat room if they are not verified users, regardless

[cbrummitt]

I'm looking into using Rocket.Chat for a research experiment in summer 2016 that will involve about 20 teams with ~250 users each. This feature and @steedos 's dashboard seem like the ideal tool!

[beatelite]

+1. Looking forward to this.