[Feature Request] Persistent chat messages

[lichtmetzger]

This was posted a few years ago on Sourceforge and probably got lost here on Github. I'd really like to see this feature, too:

"It would be really nice to have a chat log. Like a constantly logging text file in the client directory. This would create a different text file for every channel, user private message, or global. Basically a different text file according to who the message was sent to or received from. This could be opened regardless of whether mumble is open or not. We have sound recording (which is awesome), why not text logs?"

https://sourceforge.net/p/mumble/feature-requests/961/

---

Summary:

• Activation & Privacy:

  • Activation/Implementation per Channel would be best
  • Server/Admin choice (mandatory),
  • Client/User choice (would be good; in some cases legally required (see below))
    • Two methods for denial to choose from:
      • Blocking method: If one disagrees, Logging is disabled for that channel (for the time the client is in it)
      • Exception method: If one disagrees, only their messages get excluded (remaining problem: Content can be guessed from other people's content)
    • Important (Law Stuff):
      • Some laws require approval for each server (so we should not implement a general accept setting; instead a pop-up for each server/channel (at least an option to activate the latter))
      • Also some laws require an option to disagree later and for already saved content to be deleted (so it would be good to have options for that)
  • Alternative to Client/User choice: Only inform the user about logging.

• Encryption:

  • Two topics:
    • Transmission encryption
    • Log Data encryption
  • Two methods:
    • server-side encryption: makes it harder for attackers/other server users etc. to read the logs
    • E2E (end-to-end): logs would only be readable for the users

• Logs (Technical details and options):

  • logs per channel
  • file format: textfile, database
  • log size limits
  • content: text, html, images, more?
  • storage time: days, weeks, months, years, unlimited
  • compression
  • auto-resizing of images (would be useful in general)

• Special Feature: Limit participating users? To ensure the number of people being able to read the logs is limited. For example with:

  • Implementation only for password-protected channels
  • Special channels with invite on consent-function

• Additional Features:

  • Offline-Messaging (requires logging to happen on the server)

Legal requirement: For example the GDPR (the General Data Protection Regulation) of the EU (European Union) requires at least for companies and organizations to implement multiple things (asking for approval etc.) See https://github.com/mumble-voip/mumble/issues/2560#issuecomment-620195673 for details.

Implementation

Either client-side logging or server-side logging, or a combination of both approaches, for example:

• Server serves only as a cache, so that users can read messages they haven't received yet.
• Both options are combined

[cap-42]

I agree, persistent chat would be a gamechanger and ever so convenient ;)

[Kamori]

I'd like to pose a question in regards to implementation of this.

Would it be better to log client side so if you're not on you can't see the messages. Or make it persistent to the server so when you login you can see messages from the day?

[cap-42]

I personally don't have a need for client-side. Leaving offline messages for users/chatrooms would be great to have. EDIT: that came out wrong. What I meant was, I haven't been able to find a bot that spits out the chat history either, and personally wouldn't know where to start. What class takes care of relaying text chat?

[lichtmetzger]

I'd prefer client-side logging of messages.

If you do it server-side, there is always the chance that single users don't want their messages to be stored.

[Kamori]

https://github.com/mumble-voip/mumble/blob/master/src/mumble/Log.cpp

This should be the class that controls the messages that go into the chat messages. (Hopefully i'm right)

My thoughts on how to accomplish this.

define file save location on init load the file and populate Log window with the data close file on chat/message insert open file for write and append, close file

[shumvgolove]

That's the feature I really miss in Mumble. Any updates?

[mkrautz]

We'd need to come up with a design for this first.

Since Mumble supports HTML messages, the log format would need to include images. People send Mumble images inline in messages most of the time (data: URLs). That means that log files would quickly grow big.

How would Mumble store the logs? Should they be stored per-session? Single file per server? A new log file per day? Etc. etc.

[shumvgolove]

As written before, it seems to me that it is best to store logs for each user, because there is a possibility that someone will not want his logs to be stored on the server.

From my point of view, we need to give users the choice of how long to save logs -- day, week, month, year or as much as they want themselves (including forever).

Lastly, to be convenient to understand, I think it is better to save logs of text chat separate for each voice chat.

[Kamori]

So if the idea is server side, I think that is up to the admin to decide, not the user. While yea it would be cool to offer flexibility for ignoring users from chats, I'm of the opinion per channel will would provide the easiest steps forward.

Length of time, i'm flexible on personally. If i need logs rotated out i can just setup log rotate on my own box.

[shumvgolove]

Any progress?

[davidebeatrici]

Unfortunately not, but we will start working on this after #3465 is completed and merged.

[shumvgolove]

Thank you for reply.

[davidebeatrici]

You're welcome.

Feel free to write here any suggestions you have about the feature.

[XP-Fan]

Well, I also wish that mumble had a persistent chat, just like discord. Of course, logging should be able server-side, so when let's say you're on holiday for 3 weeks and you come back, you can read what your co-workers have been discussing in the meantime. However, as for the design, what we also have to take into account is that it should then be possible for someone to write an private message to a contact who is offline, which is then waiting on the server until he logs back in. This would finally make it possible to replace discord with mumble in so many, if not all cases, and that would just be incredibly awesome, considering that mumble runs on Windows XP...

[IIIXJokerXIII]

I have an idea about persistent chat that may solve the "I don't want to be tracked users"

1. The Server has a chat log that tracks chat based on a clients history setting.
2. Clients have a history setting of how long they would liked to be tracked days: "0 to All"
3. If a client has a history setting of 0 then the server does not store that users messages in the chat log, if a users has 3 then the server would save 3 days of that users messages. When a client joins their chat log would be updated from the server's chat log based on how many days they have set. 4.Give Admins control over the size of the server's chat log file size. This gives users control over how much they want to be tracked & at the same time is limited by the server's file size set by admin

[XP-Fan]

Well good idea, but then it should be also possible to set the whole thing per channel, so for example within my company I would like everything to be persistent, just like Discord or IRC with a bouncer.

[mctoddenstein]

This is actually the major point keeping my friends and I from switching from Discord, etc.

Aside from the switch decision, it'd also be good to have some logrotate functions in there for the log itself. Iterate through like chat.log, chat.log.1, chat.log.2, etc to however many are decided by the team. This would all depend on settings that were established by the server admins themselves.

Even better would be per-channel chat log settings assigned based on channel name or channel ID. In that case, you could create _chat.log. With that the server could parse and load in the log based on the channel name or even channel ID (depending on which is more convenient).

[toby63]

As a supporter of privacy I think you should consider some things:

1. Even if users choices about logging are respected (like @IIIXJokerXIII's concept), users can still guess the content of the messages of users who don't want to be tracked, by the stored messages of others. This is a big privacy concern, especially if other people than a fixed group (examples for others include: admins & new members) can have access to the logs.
2. Some laws might require at least the approval of users. The fairest option would be to ask for permission per channel.
3. Server logging is always dubious because of the problem of Plausible deniability.
4. logs can be stolen and misused, so they should be time limited and encrypted by default.

My Recommendations: General (and for client logs):

1. the users should be informed (per server or per channel), so that they know that others are logging them
2. an option for the server conf or even per user to prevent logging
3. Very important: No auto enable, or ask in the setup wizard

For server logs:

1. per channel logging only (and only on password-protected or access-restricted channels)
2. determined groups for channels (see 3,4,5)
3. encrypted logging (no access to others, including the server admin(s)), so that only the group has access
4. no access to (older) logs for new (group) members (or 5.)
5. only access to new group members with consent of other group members
6. enable logs only if all members agree to it

Edit: Added point 3 in general

[sugoidogo]

I think the suggestions by toby are overkill. Specifically, logging should be available for any channel, not only password protected ones, and their permissions should be the same as the channel by default. The user should be made aware that the channel/server is logging before they send a message, since they won't get logged if they don't send any messages. Encryption, expiry, and size of logs should all be configurable, to allow setups as strict as tobys or as (insert opinion here) as discord.

[sugoidogo]

I should specify that I would use Toby's settings for my own server, and agree that any privacy-respecting server should, but it doesn't fit all use cases. At minimum a user should be informed when logging is active, and be able to view the logging settings (who can view logs, expiry, etc) but anything beyond that need be adjustable. I also think client side logging should only be available as a mirror of server logs. A user should not have to worry about another user joining chat, and suddenly they're being logged.

[toby63]

@josephsmendoza While I can see specific use cases for your unlimited concept (like a server just for members of an organisation), I want to add one layer to the already existing layer ("what should be done"). This layer is "the law": I guess many of you have heard about the GDPR (the General Data Protection Regulation), which is defacto a European Union Law. While I am no lawyer, i understand it like this: Note: This applies mainly to server logging. Client logging might be a different case. We can assume that:

• chat logs are personal data
• chat logs are not necessary (data processing) * mumble is a project in the EU (?; many of the dev-team seem to be in europe) (irrelevant for this specific law topic)
• many servers are in the EU and many users are in the EU

This means: Servers (or rather their owners of course):

1. must ask for users permission to save/store and provide their data. 1.a) Some interpretations go even further and say that the permission must not be a requirement to use the service (so users could disagree and still use the server).
2. must ensure data protection (also on a technological level (aka encryption etc.)) by default.
3. must limit the data processing and access to a minimum (so no unlimited option).
4. must delete the data at users request.
5. must declare exactly how, why, how long etc. the data is stored and processed.

Now there might be some things open to interpretation, like:

• Are the server owners professional data controllers or processors? If the server is just run by a private individual for "purely" private purposes the GDPR does not apply (see Article 2, 2.c), but there might be other (national) laws for these cases. Also the question is, what are "private purposes"?
  • If the server is available to the public it might not be a private purpose.
  • If the server is run for an organisation it might also not be a private purpose.

tldr; Sry for the long text, but doing unlawful things can get expensive (just search for "gdpr fines" and you will see how high the costs can get). So I wanted to clarify that.

[streaps]

@toby63 everything what you wrote applies to audio recording too, which is already possible without consent.

[toby63]

@streaps

everything what you wrote applies to audio recording too, which is already possible without consent.

No, because there is an important difference: Audio recording is done and stored by the user. I was referring to server logging (maybe should clarify that above), which is done and stored by/in the server itself. So server logging (and recording) is a clear case of the law (imo), while client/user recording (and logging) is not. Nonetheless there might be other laws to apply to that.

Also: Just because something is done, doesn't mean it is also lawful, or done in a lawful way.

[IIIXJokerXIII]

I'm going to try and keep this train from running anymore off the tracks. You should never bring up the LAW of a specific country as it will not apply. The reason for this is because in the terms of use you could easily make the user give up ANY rights by using the program. You just let the user know that if he/she installs and uses the program their information may be recorded and used. This is how all software agreements work just look at Microsoft, Google, Facebook if you want to see some real scary terms lol. Users are okay with this they will just click next and the ones that are not don't install the program anyways. Before any of you say No remember the terms now pretty much cover this already ;)

[toby63]

@IIIXJokerXIII You missunderstand. My law topic is not about the software (mumble), it is about the obligations of the server owners, as they will use the software. Now mumble can of course say "we don't care", but it would be much better to help the server owners by adding options to comply with the law.

[mctoddenstein]

@toby63 i agree with @IIIXJokerXIII that we're getting into semantics of how to apply the law instead of the feature request itself.

With regards to implementation, I can understand informing the client-side users of the update. If the server gets updated and has this built in at that point, yes, it would be important to let people know that logs are being saved for SOME period of time.

The HOW of the matter would likely require adding entries into logrotate or some other way of log rotation to ensure that data is only retained for as long as server admins want to keep it. At that point, the logs start to iterate and roll off. Making sure that these chats stay specific to individual channels is also important instead of having a global chat that spans across multiple voip channels. How the association would be handled per channel would be outside of my knowledge

[sugoidogo]

My law topic is not about the software (mumble), it is about the obligations of the server owners, as they will use the software.

We are here to talk about Mumble. Mumble is a tool, like any other software. A tool should not be opinionated, and this tool is not specialized to GPDR countries. The implementation should allow the end user/admin to customize such settings to their requirements, so let's leave the privacy and law concerns to them.

I believe this feature makes the most sense serverside, as a way for offline users to see old messages. If I'm not mistaken, murmur is cross-platform, so the initial implementation could be plain text files per channel, and then logrotate, database, etc, can be added after the most basic implementation is done.

[sugoidogo]

It would be nice for murmur to default to the most reasonable option per-platform once they are available.

[streaps]

@josephsmendoza so you think it's not relevant that people who run a public murmur server or the mumble project itself could get legal troubles?

It's also not only about the European laws (there are or will be similar laws in other countries), it's also about the user. I do want to know if my conversations are logged and for how long.

[streaps]

The reason for this is because in the terms of use you could easily make the user give up ANY rights by using the program.

Sorry, but this is bullshit. If you offer a public service to users in the EU there is no way TOS around the GDPR. If it were that easy the GDPR would be useless.

[sugoidogo]

@streaps I'll assume you weren't reading my posts, or are having a hard time understanding them, so I'll simplify my statements for you. Murmur should let admins do what they want with logs. Users should be notified of logging settings. Admins are responsible for keeping their logging legal. Additionally, logging should be disabled by default. Does that address all of your concerns?

[streaps]

No

[sugoidogo]

@streaps can you elaborate?

[Krzmbrzl]

Murmur should let admins do what they want with logs.

Absolutely not. While it is true that from a legal point of view, it is the admin's responsibility and not the one of the Mumble Team, it is also true that we can easily lose the trust of our users if we implement something like this sloppy (in a way that let's admin do whatever they want). Thus we should take special care to prevent these scenarios. If you disagree with the implementation of a specific feature (like the logging of chat messages), you can always create a fork of the project and change the implementation. That is your right and that's also the beauty of FOSS.

[sugoidogo]

@Krzmbrzl I should clarify that I mean admins should be able to change settings like log expiry, encryption, permissions, and size without mumble enforcing restrictions on those settings. I'm a little confused by your comment because I don't know what you would be restricting, or what this scenario is.

[Krzmbrzl]

Ah then I guess there has been a misunderstanding. I was thinking you meant that admins should have unrestricted access to all log messages at all time. Of course I agree that the settings about how long chat-logs are saved, etc. are up to the admin. However in my opinion the option whether or not chat-logs are saved encrypted is not an option I see fit for being under the admin's control. This falls in the category of data-safety a Mumble user can expect when using the official client & server.

[sugoidogo]

I think encrypted by default would be the best implementation, as part of sensible defaults. For example:

log_expire_time=-1 # defaults to -1 for no logging, 0 for infinite, 1y for one year, 10d for ten days, 1h for one hour, 5m for five minutes, 30s for thirty seconds
log_max_size=1g # g,m,k,b
log_encrypted=true # DO NOT DISABLE THIS without otherwise protecting log files.

The user could receive a pop-up or otherwise flashy and non-optional warning upon entering an unencrypted logging channel, which advises them to treat it like posting on the open internet.

[Krzmbrzl]

But why would you want to be able to disable encryption in the first place? I actually don't see a valid use-case for that, nut maybe I just don't have enough imagination atm ^^

[Wadera]

Example: Enable true encryption on multi user chat isn't something easy to manage. You need to get some authority who can manage keys, authorise new users etc. It can be difficult to understand and use by "ordinary user".

Use case: Mumble server hosted on home server - I'm admin and have full access to hardware and software on root level. No one have access to this server anyway and I'm using it for gaming with my friends. It's just 5 people who joining and sending links or chat about World of Warcraft / PUBG etc. As we are only users on this host and we can see this chat anyway - there is no real risk and no reason to play with RSA key sharing, permission managing and learning my friends how to get access to mumble server from laptop and PC and get ability to read chat.

[sugoidogo]

An example might be a public server with a public channel with logging enabled, where the encryption is pointless because anyone can join the channel and scroll back up, might as well save the CPU cycles since such a server likely needs all the performance it can get.

[Krzmbrzl]

Yeah the example of a public server anyone can join to read the messages is a good one (might be more difficult though as one had to keep in mind that you might not be allowed to join certain channels and thus you can't read the chat history there).

I think that if anyhow possible the whole log-encryption stuff should work "automagically" and thus probably without most users even noticing. I don't know how feasible that is though (definitely no crypto expert here) :shrug:

[sugoidogo]

Log settings should be per channel so that same server could have a moderator channel with encrypted logs. The encryption could be done with the same keys used for tls, since log encryption would also be pointless if the messages are sent unencrypted in the first place.

[IIIXJokerXIII]

Simplicity Simplicity Simplicity. I repeat myself because the only way this project moves forward in a timely manner is to keep it simple! I've already seen many projects fail because they became too complicated. If it becomes too complicated Devs will not want to work on it and server owners will become frustrated for thinking the program lacks what they see as basic chat features now days. Losing more users to other programs is not something the murmur project needs right now.

Like my 1st comment but with more detail

1. Setup a Server Chat Log - Server Owner has control of how large this file can be.
2. Client has a history setting of days 0-All. - This is still limited to log size set by server owner.
3. When a client connects his chat is filled with the Server Chat Log AKA History of Users.

Now if a user had his history setting set at 3 only 3 days of his post are saved to the Server Chat Log. If that same user now changes that setting to 0 then the server will purge all of that users post when he disconnects or make a button to purge my messages from server. This would delete any of that users messages out of the Server Chat Log past his history setting value. <--Make this work and you could get more detailed with it later like the server log could be per channel and the purge could have options like only purge this channel of my messages or maybe per channel history setting so many options could be added. Discord would not have anything on this privacy & history WOW :dancing_men:

[toby63]

Feature-wise:

• End-to-End-Encryption should be considered: related to #1813 Like I already said, the perfect scenario would be for chat logs only to be visible to participants and not to the server owners and everyone else. I know this is a bit complicated, but there might be a solution for this, like seperate group-passwords (maybe in combination with a certificate) used for encryption.
• Another thing to consider is forward-secrecy (but I already hear the witchhunters with the forks :stuck_out_tongue_winking_eye: )

My opinions:

• I think that this feature should be implemented properly from the beginning (so all reasonable options: like encryption, per channel logging, limited access etc. should be implemented before it is included)
• I don't think unencrypted chat logs should be allowed. Just like mumble always uses encryption for the transport, it should also use encryption for chat logs.
• I also don't think scenarios like public servers/channels with chat logs should be allowed. This is bad practice and also for the server owners it will bring a lot of spam. I also don't see reasons for this; if I set up a server just for my friends, this server is of course access-restricted. Mumble is no forum.

@josephsmendoza One general thing: I don't understand why some people here see my comments as invalid or negative. All I say is that mumble should consider implementing options, so that server owners can comply with the law. And if you look at the law, you also see that this is (mostly) good practice (asking for permission, encryption, information for users etc.). Thankfully some users and devs see what I'm up to and agree :slightly_smiling_face: .

@Wadera

Enable true encryption on multi user chat isn't something easy to manage.

As @Krzmbrzl already pointed out, the whole encryption should run automatically in the background. There are of course some potencial challenges for the developers (if we want stuff like End to End Encryption etc.), but in general its all about the usual encryption (certificates or keys are created by mumble and then used in the process).

@IIIXJokerXIII

Like my 1st comment but with more detail

You still ignore what I already pointed out: guessing what users said is still possible if the messages of their correspondents are visible. Also the problem of Plausible Deniability remains. Per channel settings are much better.

[Kamori]

Hello everyone, unfortunately I'll be unsubscribing from this issue as I've moved on to just using Discord as it fits all my needs.

Having said this. I think there are a lot of good points here. But there seems to be a lot of over-engineering around a feature that doesn't exist yet. Perhaps we need to take a step back, create a functional prototype, and worry about non-technical issues ones we have a working base.

Keep up the good work, and I'm excited to see what Mumble turns into in the future!

[toby63]

@Kamori

Perhaps we need to take a step back, create a functional prototype

I am sure the developers will do exactly that. But having a concept before starting the work is mostly better, because changing the code structure afterwards (to for example fill in something that was not considered before) is maybe unpleasant work.

That said, if you think about it, the easiest way would be to just implement client logging.

[IIIXJokerXIII]

@toby63

• You still ignore what I already pointed out: guessing what users said is still possible if the messages of their correspondents are visible.
• Also the problem of Plausible Deniability remains.

You're missing my point you're over thinking it. If you need a system that secure you should not be using mumble or like someone said above make a new branch yourself with that level of detail as the average user will never need it. It's overkill for a dev to put that much work in to making it so a user can not guess what you may have said lol. In the end look at it from a dev point of view as to how much work will need to be done to get this in and as always keep it simple. I just like to offer up ideas but let the coder/dev decide how complex he/she wishes to make it.

[toby63]

@IIIXJokerXIII I don't think this has really something to do with complexity. This is all about a concept. Your concept ignores nearly every security and privacy aspect and as I have pointed out, this is not only valuable, but also a legal requirement (at least in many cases).

That said, of course the world is free.

[XP-Fan]

@toby63: " I don't think unencrypted chat logs should be allowed." If it was like that, I'd immediately leave and switch to discord out of principle. I would never accept having to encrypt something on my physical hard disk drive, be it on my desktop or my server. If I'm the only one having hardware access, then why the heck should I fuck up my processors like that? And yeah, I know that discord uses electron, but hey, there are IRC-gateways and unofficial clients, so, … Anyway keep in mind that with E2E it's getting hard for people like me who have about 30 different computers around, for the sake of being able to test handwritten assembly on different processors, and I want to have my main chat client everywhere. So, get over it, not everyone wants your encryption-nightmares, so if the mumble-devs really wanna code them, I'd beg them off to keep it optional, not enforced.

From: toby63 [mailto:notifications@github.com] Sent: Donnerstag, 30. April 2020 18:46 To: mumble-voip/mumble Cc: XP-Fan; Comment Subject: Re: [mumble-voip/mumble] [Feature Request] Log text chat messages (#2560)

@IIIXJokerXIIIhttps://github.com/IIIXJokerXIII I don't think this has really something to do with complexity. This is all about a concept. Your concept ignores nearly every security and privacy aspect and as I have pointed out, this is not only valuable, but also a legal requirement (at least in many cases).

That said, of course the world is free.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/mumble-voip/mumble/issues/2560#issuecomment-621970645, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKEBHTOVDRHDNNFHEGIUIMLRPGTMLANCNFSM4CQL3WYQ.

[Krzmbrzl]

@XP-Fan could you please check your tone in this discussion?

Anyways I don't think that the discussion continues to bring a lot of new arguments at this point. I guess everything important has been said now and lately the discussion seems to have drifted into a battle of opinions. Thus if you don't have new arguments, I'd ask you to just +1 the posts that you agree with (and perhaps -1 the ones you don't).

From my point of view (as a Dev), I agree with @toby63 that it is very beneficial to have an idea of where a feature might go in the future when one is beginning to implement it.

That being said though I think ultimately it'll come down to the Dev that ends up implementing this feature. We have listed a lot of pros and cons for various degrees of security/encryption and as this is an OpenSource project, that Dev will most likely implement this as he or she thinks it is best. And I think this issue now give s solid ground to make up an opinion about the topic :point_up: