Admin ability to view history for all channels, direct messages and private channels

[gt50]

Your Rocket.Chat version: 0.26.0

I would like to be able to view all conversations that have occurred on the RocketChat server. I would like to use it in an work environment. There is no expectation of privacy and the information would be accessed when needed for auditing purposes.

I imagine this either being added to the history area, or allowing an admin to view a room, or add themselves to a room from the rooms area in adminstration.

[edman545]

I agree, there is some expectation from upper management that all electronic communication is being logged somewhere. Even if if it just dumped every message sent to a log file somewhere, that would be enough for our purposes. It is very rare that I have to dig into such information, but sometimes there is good reason.

[tholu]

Could be implemented in the admin rooms view (where all direct messages are already listed as well), when clicking on the room to show the logs of it. Or in a separate audit view.

[kdahmani]

I also think that from a compliance point of view it would be very important to be able to access all chats on the server

[heytimc]

Definitely need logging for compliance here. Anything would be great; text, syslog, d/b, don't care how. We won't be able to go live until this is available.

[dereckhall]

same requirement for our office as well. currently utilizing openfire jabber for interoffice communication w/ their archive plugin. something similar or anything really would be great.

[thelilmatt]

+1 for this. It's the only thing holding us back from implementing RocketChat

[dupree972]

I would love this functionality too...

On Jun 10, 2016, at 3:36 PM, thelilmatt notifications@github.com wrote:

+1 for this. It's the only thing holding us back from implementing RocketChat

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/RocketChat/Rocket.Chat/issues/2869#issuecomment-225288931, or mute the thread https://github.com/notifications/unsubscribe/AOVtMEqR3suEpU_gECJHTbf_otfHbdZdks5qKcrGgaJpZM4IGrw8.

[widget7h15]

Rocketchat is a fantastic platform but I would really like to see this feature implemented so we can use this in our organization. Pretty much every other chat style program has at least one way to audit conversations for HR purposes.

[dupree972]

Is there any progress or has anyone even found a work around for this. We are a SEC and CFTC regulated entity and our compliance officer has the need to randomly review a certain small percentage of messages per month. Also we have the need to archive messages. It does not need to be fancy, even if they all dropped into a text file or something that could be ingested somewhere like a smarsh or other archive that would be terrific.

[slowbiz]

This is also functionality we will need before moving to production in our environment.

[heytimc]

Yay! Merged to develop :)

[slowbiz]

I see the Smarsh Connector in the latest build, which is great. Is there any plan to offer a built-in log/archive/history page to audit conversations?

[patit72]

Here too, we need that function before moving to production

[megabit-computing]

This functionality is vital before we can consider using rocket Chat hosted version!

[rajsardhara]

+1

[Nepoxx]

You can already do this by connecting to your Mongo database.

List the rooms with db.rocketchat_room.find({}), grab the ID of the room you want the history for.

Then you can see that room's history with db.rocketchat_message.find({"rid": "your_chatroom_id"}, {"u.username": 1, "msg": 1, "_id": 0}).sort({ts: 1}).map(function (d) {return d.u.username + ": " + d.msg})

It's not as intuitive as a web interface, but auditing your users' chat logs should seldom be done therefore this is a viable workaround in the meantime.

This topic attracts a great deal of controversy, as it should. That being said, this method is a workaround until there is a corresponding UI. This also means that you are already being recorded when chatting using RocketChat since your messages are saved in the database unencrypted (unless you use OTR). If you allow your users to edit/delete their messages, they are indeed modified/deleted from the database, so take that into account.

[dereckhall]

@Nepoxx For the majority of users here this is a business requirement.

Also, your method doesn't take into account that users can still edit their original message, which modifies the original within mongo.

[philippevk]

@dereckhall He just gave us a workaround. You can disagree about the spying part (I don't), but where I'm from when someone help you, you say "thank you"

@Nepoxx thanks!

[xandershelley]

If this feature is enabled it should be made clear the chat is being recorded. RocketChat is already being used for private chat servers as well.

[patit72]

It might be required for forensic audit, and its use should be logged too. For sure it is a feature needed in business environments

[christopherturner]

Are there still plans to integrate this into the UI?

[sebastientaggart]

+1

[deiansp]

To have history for audit is one of the major mandatory business requirements for any company. So +1 for an easy UI interface. 10x!

[Nemra1]

we need that for admins only..we dont know what bad planes happend in direct rooms

[evaldoprestes]

+1

[mauriciovirtual]

+1

[anhnt]

-1

[Tallestthomas]

Has there been anyone working on this? We love rocketchat at my company, but can't move forward with using it until theres some sort of logging in place. I've started to build an external logger using the api, but it would be great if there was an in app solution to this.

[Vacoi]

+1

[timiil]

+1!!!

[Darkestlyrics]

All Messages are logged in the logs as debug events.

You will have to change you logging level from Error only to Error, Information and Debug.

This will then allow you to get the following from the logs

I20180227-08:59:05.806(0) rocketchat_logger.js:278 Integrations ➔ Outgoing WebHook.debug Got the event arguments for the event: sendMessage { event: 'sendMessage', message: { _id: 'eXePYf2wfXgs46k2J', rid: 'bWQH3mKraFHwcyMNvrJY49bWrYTtd67m5E', msg: 'test', ts: 2018-02-27T08:59:05.780Z, u: { _id: 'bWQH3mKraFHwcyMNv', username: 'USER1', name: 'USER1' }, mentions: [], channels: [], _updatedAt: 2018-02-27T08:59:05.782Z, sandstormSessionId: null }, room: { _id: 'bWQH3mKraFHwcyMNvrJY49bWrYTtd67m5E', _updatedAt: 2018-02-27T08:57:51.741Z, msgs: 7, t: 'd', ts: 2018-02-26T10:13:17.126Z, meta: { revision: 7, created: 1519639997131, version: 0, updated: 1519721871744 }, '$loki': 15, username: 'USER1', lm: 2018-02-27T08:57:51.730Z, usernames: [ 'USER1', 'USER2' ] } }

[leonardoap]

+1 !!!

[paulomatheusgrr]

Do we have any news about this?

I can´t install rocket chat in my business while this don't works!!

[sadkins76]

Tallestthomas, Did you make any progress with the external logger? If so, would you mind sharing? I need this badly....

[vynmera]

It's probably fair to (if this does get implemented in a way where this is possible) add a warning to private channels and DMs that admins can read these chats (especially for public instances).

As it is right now, DMs could have a warning if the Read other IMs option is enabled in server settings.

I believe transparency is key with these things!

[gt50]

Last year I created a python script that I could run to get the data that I needed. I just uploaded it to a gist. Enjoy!

https://gist.github.com/gt50/82005184d9a030cbe6e2ea91a4ae39b7

[ratsclub]

After trying to get some room's histories I was thinking: is it possible to use the REST API to read the messages? I was trying to get the users message as an admin and I couldn't because I didn't have the user's credentials to login into the API.

[vynmera]

@dotsomething You can get DM messages, yes. You can use https://rocket.chat/docs/developer-guides/rest-api/im/messages-others/ together with https://rocket.chat/docs/developer-guides/rest-api/im/list-everyone/ for this. Simply use im.list.everyone and read the DMs with im.messages.others. To use this, you'll need to turn on the Enable Direct Message History Endpoint setting!

[Nepoxx]

@dotsomething Do you have access to the mongo database? If so you can query it to see all the messages you need. See my comment above which contains a sample query.

https://github.com/RocketChat/Rocket.Chat/issues/2869#issuecomment-280046547

[ratsclub]

@vynmera I have tried that method.

My scenario is:

I'm logging in into my admin account and retrieving my authToken and using the im.list.everyone. But am I supposed to do the following?

            method: 'GET',
            url: URL + "/api/v1/im.list.everyone",
            headers: {
                "X-Auth-Token": 'my token',
                "X-User-Id": 'the user's ID I wanna get the conversations from',
                "Content-Type": "application/json"
            }

Because it returns this:

{ status: "error", message: "You must be logged in to do this." }

@Nepoxx Yes, I do have access to the database. But I was going to build a static page as a web interface to audit in case something wrong happens at the company. Now I'm going to take a look at the solution @vynmera gave me and perhaps use Nodejs with mongodb.

Thank you, guys!

[vynmera]

@dotsomething You run im.list.everyone as yourself, then find the IM you want to look into. (Hint: the _id field for a DM consists of the 2 user IDs that are talking) Once you have a _id, use im.messages.others with that id, again as yourself.

[ratsclub]

@vynmera Thank you, dude! Going to work on that and if it turns out to be a good piece of software I'll provide it here as a repo. 😉

[engelgabriel]

Here are the screenshots of what we are developing.

And each time the admin looks into conversations, it will leave a log that can be searched to autiding the auditor :)

Since this a sensitive feature, we will be releasing it as one of our first Rocket.Chat Apps available via our marketplace next month.

[rajsardhara]

Sensitive Feature 🤔 🤔

[engelgabriel]

I forgot to say, if you want to be updated when it is available and are interested on being beta testers, please send an email to contact@rocket.chat

[engelgabriel]

@rajsardhara sorry for the short "Sensitive Feature", let me elaborate.

We always thought that since all the msgs were available on the DB, that this would be already enough for most use cases. But, as this thread here shows, it was clear not the case and so we proposed that we were going to build that feature in the admin panel.

Then we received a lot of feedback from the community explaining that they specifically don't want a feature like that to be available on the admin panel by default, making far to easy for admins to "spy" on users' conversations.

Thats why we considered it a "Sensitive Feature", because the more we looked into it, the more complicated it got.

We realised that to make it simpler, we had to take it out of the main project, and deal with it as an extra plugin (now called Apps) so only the admins of deployments that wanted that feature would install it.

We also realised that this is a very enterprise focused functionality and we believe that this can be one of the ways to monetise the project and we are studding a few licensing models for this app, and we would love to hear your thoughts about it.

[sadkins76]

I LOVE the way this is looking. Not sure how I feel about the monetization of an app to add functionality that should be part of the program to begin with. I really have no issue paying for the functionality, but please, for the love of all that is holy, make it a flat rate fee. Not another subscription? PLEASE?

I don't understand the issue people have with an ADMIN seeing conversations, especially in a corporate environment. There are far more valuable things that an admin has access to than a chat conversation log to worry about. Looks amazing though guys! Count me in as a tester when you make it available!

[vynmera]

Looking very good, count me in! On the topic of monetization: it's not a bad idea at all. However, we should always be wary of greed, and make sure we don't restrict access to basic features (unlike Mattermost, in which a basic permission system is already paid). Furthermore, we should inform users beforehand on the website what features are paid and what aren't!

@sadkins76 Well, I suppose the idea is that some people use Rocket.Chat for public services - having your conversations recorded and delivered to the admin on a silver platter may feel dirty to users. I'd be one of them, surely. Of course, an admin can get all this data already if they desire, but that takes extra effort and would break the trust between the user and the admin. A simple warning would allow the admin to look at the DMs, yet keep the user's trust (and also get a nicer UI to read the messages with).

[engelgabriel]

And also for clarity, ALL the messages are stored on the DB, so they can be searched and viewed by the sysadmins and DBAs.

Regarding the edited msgs, there is a setting in the admin panel that when turned on, will save all version of the messages on the DB also.

@sadkins76 in a corporate environment most people agree with you, but on other environments, people seems to disagree :)

[engelgabriel]

@vynmera indeed the topic of monetisation is a delicate one, but very important for the continuity of the project, so we will get the community input on this to make sure we are aligned.