search

signalapp / Signal-Desktop

A private messenger for Windows, macOS, and Linux.
https://signal.org/download
GNU Affero General Public License v3.0
14.66k stars 2.67k forks source link

Ridiculously slow startup times on both linux and windows #5030

Closed tanujkucp closed 3 years ago

tanujkucp commented 3 years ago

Bug Description

When i open signal app on ubuntu or windows after a day or two, it has too much messages to load. It stops on loading screen for 5 min to 30 mins and sometimes loads upto 5000 messages. It is very unusable experience.

Steps to Reproduce

Open signal app anyday after chatting for some time on your other device.

Actual Result:

Signal takes too long to load messages. sometimes half an hour.

Expected Result:

It should load instantaneously.

Screenshots

Not applicable

Platform Info

Signal Version: Signal 1.39.6

Operating System:

Linux ubuntu 20.04 Windows 20H2

Linked Device Version: Signal for android 5.4.4

Link to Debug Log

Uploading log from inside app didnt work. Uploading zip of log here did not work, it showes error that it does not support this type of file. So i pasted it in the dog bin.

https://del.dog/tupameryng

https://debuglogs.org/7f60412f0910d02072d69a96298a86ede78c219435b79f44d544abfd89b88021

Dyras commented 3 years ago

Out of curiosity, do you have an SSD on your PC? Does the message counter ever freeze or is it constantly going up?

I have an M.2 SSD and for me, it goes from 0 to 5000 in about 3 minutes.

tanujkucp commented 3 years ago

I have an SSD and the messages keep coming slowly. The message counter doesn't freeze. Probably this processing should be done in RAM if the disk access is the bottleneck. I also have write cache enabled on ubuntu ,but that also doesnt improve the startup time.

hiqua commented 3 years ago

If it's really unusable, you could try to disconnect, run Signal-Desktop and let it load, then connect again. Then I think the sync should happen in the background. Could you confirm this?

If that works I think we should just make a PR to prevent the sync from happening on start-up.

tanujkucp commented 3 years ago

From what I have read about this issue on forums is that the sync on startup is necessary to maintain consistent state between all linked devices. And it's slow because all the state and chat messages need to be loaded and processed in sequence. And if this process uses disk as someone pointed out that their SSD performs better, then this job should be improved and maybe process these requests in RAM and then batch write once on disk.

hiqua commented 3 years ago

You are completely right and in an ideal world that's what we should do. But if we can work around this inefficiency by loading messages after the interface is done loading, that could be good enough for now, until the real problem is addressed. My question was thus: does it load faster if you're not connected to the Internet?

If yes, then I (or another contributor) could try to make a PR to implement this workaround. In terms of UX, it would not be perfect because you would see Signal-Desktop synchronizing a lot of messages as you're using it, but you would have a better idea of the progress, and it would still be possible to send a message (e.g. if you already know about the last messages because you've read them from another device).

tanujkucp commented 3 years ago

Yes, it does load faster when i am not connected to internet. And when i connect to internet it starts loading messages. This time it feels a bit faster than loading on startup, but during this also app is not usable. It stutters and freezes a lot and the conversation messages are continuosly refreshing - loading, and sending a message at that time does not seem to be a good idea.

BarbossHack commented 3 years ago

Same problem for me on Windows (SSD), I have to clean my desktop app data (disconnect) every week to keep a decent loading time (less than 10 minutes). It's awfull, should be less than 30 sec (on my smartphone its instantaneous so why not on my desktop, what is the main difference)

Why does the app have to load all (old and new) messages for each conversation? I am 100% sure I have less than 500 messages in all my conversations (combined) because I have message expiration, and still the desktop app is loading 3000-4000 messages each time and it continues to grow a lot...

tanujkucp commented 3 years ago

A workaround I found and I use on both Linux and windows is to start signal on boot. I added it to start on boot and remain minized in tray.this way it loads messages in background before I want to start using it. But sometimes loading time is half an hour and even this workaround doesn't work.

hiqua commented 3 years ago

It's awfull, should be less than 30 sec (on my smartphone its instantaneous so why not on my desktop, what is the main difference)

One big difference is that the smartphone continuously synchronizes, so it never needs to have these synchronization bursts. If you had Signal-Desktop open most of the time, it would be faster to load.

Why does the app have to load all (old and new) messages for each conversation? I am 100% sure I have less than 500 messages in all my conversations (combined) because I have message expiration, and still the desktop app is loading 3000-4000 messages each time and it continues to grow a lot...

"Message" is a bit misleading, it's not necessarily some message you wrote yourself. See this thread for more info: https://community.signalusers.org/t/signal-desktop-startup-slow/24799/6. In particular, having self-destructing messages on should not change how many messages are loaded.

BarbossHack commented 3 years ago

"Message" is a bit misleading, it's not necessarily some message you wrote yourself. See this thread for more info: https://community.signalusers.org/t/signal-desktop-startup-slow/24799/6. In particular, having self-destructing messages on should not change how many messages are loaded.

Thank you for the link!

hiqua commented 3 years ago

wait for 10 minutes loading time, close the app

You wait for 10 minutes, after which the UI is fully accessible and you can send messages from Signal-Desktop?

Then I'd say it's indeed a bug, not expected behavior. Debug log?

Firefly74940 commented 3 years ago

If it's really unusable, you could try to disconnect, run Signal-Desktop and let it load, then connect again. Then I think the sync should happen in the background. Could you confirm this?

If that works I think we should just make a PR to prevent the sync from happening on start-up.

i do have the same problem since i started using signal and as history isn't synced with newly added devices at startup it's indeed instant after removing and re-adding the device.

there is probably different way to improve the sync logic ( conv by conv, media in last , background, more in memory work and less with disk etc )

hiqua commented 3 years ago

i do have the same problem since i started using signal and as history isn't synced with newly added devices at startup it's indeed instant after removing and re-adding the device.

You mean when you unlink / relink the device? I meant to keep it linked, but just e.g. disconnect from your wifi, launch signal-desktop, then connect to your wifi again.

But as you mention, unlinking and linking again could also help.

Nisc3d commented 3 years ago

I updated to v1.40.0 today. The first thing I noticed, that startup took longer than before. I changed nothing on my setup.

https://debuglogs.org/3203fe853dadcbb8e3d6501e6c8a1ad3b80b5a4e4dbd4e48bc9bae88753641c1

hiqua commented 3 years ago

I updated to v1.40.0 today. The first thing I noticed, that startup took longer than before. I changed nothing on my setup.

https://debuglogs.org/3203fe853dadcbb8e3d6501e6c8a1ad3b80b5a4e4dbd4e48bc9bae88753641c1

Was it only the case right after the update, or is it still the case now that you've closed the app and reopened it several times? I'd expect the first start-up right after an upgrade to be slightly slower (although your log does not seem to indicate this).

I can't say that I'm experiencing a slower start-up with v1.40.0, so it's also hard to tell what's subjective and what's not.

tanujkucp commented 3 years ago

I have added a new debug log link to original issue template. This is a bigger log file and maybe more helpful.

ferguslesley commented 3 years ago

I also have this issue. It really makes the desktop app unusable. This has been a problem since 2019 from what I have found, shocking that it hasn't been fixed yet.

ghost commented 3 years ago

Why doesn't the app just load the most recent messages and load more when users scroll up? Because most probably users have already seen older messages on other devices and no need to load them again.

Today the app has taken over 20 minutes to load on my device, Debian 10 with core i7 CPU. My average download speed is 20 Mbps. Messages loaded were almost 5000 because I haven't opened the app in a while.

hiqua commented 3 years ago

There's a certain dependency between messages, it's not possible to ignore everything before and decrypt only the last one.

BarbossHack commented 3 years ago

Maybe you can look at what Element (Riot) do. https://github.com/vector-im/element-web

You can open the desktop app in seconds, then they download and decrypt the messages of the current conversation (you can even watch the messages being decrypted, funny, and very fast), and messages of other conversations are processed in background

pepa65 commented 3 years ago

It seems to synchronize "new" messages about 1 per second, so that can take a while if you've been out on your phone using Signal all day...

limbusdev commented 3 years ago

Same here. It seems to download all messages at every startup (more than 5000) and takes up to 5 minutes Signal can be used. Same on Windows and Linux. Why not downloading only new messages?

BarbossHack commented 3 years ago

Same here. It seems to download all messages at every startup (more than 5000) and takes up to 5 minutes Signal can be used. Same on Windows and Linux. Why not downloading only new messages?

It download only new messages. Try to open Signal desktop app (it will take 5-30 minutes the first time in a while), then close the app, and open it again, it will load in seconds. The problem is not here, we need to find a way to load them more efficiently (and user friendly), maybe like Element (Riot) does

tanujkucp commented 3 years ago

Try to open Signal desktop app (it will take 5-30 minutes the first time in a while), then close the app, and open it again, it will load in seconds.

I can tell you by experience that you should not close the app while it's loading the messages. This results into some messages dropped out of sync completely and will never get synced. Happened with me few times and I have created thats issue too. #5034

BarbossHack commented 3 years ago

Try to open Signal desktop app (it will take 5-30 minutes the first time in a while), then close the app, and open it again, it will load in seconds.

I can tell you by experience that you should not close the app while it's loading the messages. This results into some messages dropped out of sync completely and will never get synced. Happened with me few times and I have created thats issue too. #5034

(If forgot to say to wait for the messages to load, and it is just for testing purpose)

balintbarna commented 3 years ago

Same issue

I have a strong dekstop PC running Windows with a very fast SSD and fast internet connection. Opening signal takes several minutes after I haven't used it on the PC for a few days. During this I can see the startup screen and the loaded messages counter going up at about a dozen or few dozens per second

nixsee commented 3 years ago

I have similar issue to @balintmaci - takes a few minutes if hasn't been used for a bit. If I close and re-open, it loads instantly.

Logs posted here: https://debuglogs.org/8e82a02373aa28fa9ea40f69c5dd81539859730564591996cbef0aef788bfc21

mcaap commented 3 years ago

I agree with this issue.. its very slow for these modern times and probably needs some better way to sync down all details.

scottnonnenberg-signal commented 3 years ago

@balintmaci Your experience seems to be particularly extreme - a fast computer and much slower performance than we see. Would you consider sharing your debug log? That would really help us understand what's going wrong for you.

balintbarna commented 3 years ago

@balintmaci Your experience seems to be particularly extreme - a fast computer and much slower performance than we see. Would you consider sharing your debug log? That would really help us understand what's going wrong for you.

https://debuglogs.org/32ea11b620696574c232f3071e44dcac6585497c164e201b9470c1caa81e5d85 Today I opened it after 2 days and I forgot to time it but from what I can see the startup took well over half a minute for somewhat more than a 1000 messages according to the counter. It seemed to increase with roughly 30 increments so the math checks out. Also I wouldn't think that I send a 1000 messages in 2 days but maybe I just don't notice...

BarbossHack commented 3 years ago

Also I wouldn't think that I send a 1000 messages in 2 days but maybe I just don't notice...

Every message reaction, every deleted message, every group update notification etc... count as one more message. So you can have 3 "text" messages in a discussion, and yet have to download 9 messages, for example

nixsee commented 3 years ago

No, it is not an incremental sync. It is a full, of nearly full, sync. I have maybe 3 conversation threads, which are pretty inactive. I logged into desktop yesterday, a couple days after my first report here, and it synced hundreds, maybe 1000, messages (I stopped looking), despite having only sent or received 20 at most in that period.

scottnonnenberg-signal commented 3 years ago

@nixsee Another thing that can cause the counter to go up is when we get a 'contact sync' with contact data from your primary device. If you provide debug logs for when you expect a low count but get a high count, we can tell you what's going on for sure.

oomek commented 3 years ago

If such a simple app is unable to start in 1 second from a ssd then it's poorly designed. Messenger and Discord open up immediately on my PC.

laicasaane commented 3 years ago

I think you need better synchronization strategy.

nixsee commented 3 years ago

@nixsee Another thing that can cause the counter to go up is when we get a 'contact sync' with contact data from your primary device. If you provide debug logs for when you expect a low count but get a high count, we can tell you what's going on for sure.

I already provided a link to a debug log above. Here it is again. https://debuglogs.org/8e82a02373aa28fa9ea40f69c5dd81539859730564591996cbef0aef788bfc21

balintbarna commented 3 years ago

@balintmaci Your experience seems to be particularly extreme - a fast computer and much slower performance than we see. Would you consider sharing your debug log? That would really help us understand what's going wrong for you.

It seems even slower on Ubuntu (same machine, dual boot) https://debuglogs.org/445790d687a87a0037c7bf506fcf9dd3ae9240e2d6807aa1c000634bf577cb0d

abeyaustin commented 3 years ago

This has been a problem since 2019 from what I have found, shocking that it hasn't been fixed yet.

This is exactly why I don't expect them to fix this terrible issue anytime sooner.

Sigh!

josh-signal commented 3 years ago

@tanujkucp can you please try the latest beta release and report back if there has been any improvement? We've made some changes which should net some small improvements.

BarbossHack commented 3 years ago

For curious people, the commits are https://github.com/signalapp/Signal-Desktop/commit/1622fdfa4e798fd419c7f95526385dbab6f49d92 and https://github.com/signalapp/Signal-Desktop/commit/d82ce079421c3fa08a0920a90b7abc19b1bb0e59 (among others) 😛 I will check next time without opening it for a few days, thank you !

abeyaustin commented 3 years ago

For curious people, the commits are 1622fdf and d82ce07 (among others) 😛 I will check next time without opening it for a few days, thank you !

Hey, you meant the 1622fdf commit was actually implemented in the latest beta? But I think it's wrong because when I tried to cold start the app today morning, it was loading more than 50 messages for me.

image

BarbossHack commented 3 years ago

Hey, you meant the 1622fdf commit was actually implemented in the latest beta?

But I think it's wrong because when I tried to cold start the app today morning, it was loading more than 50 messages for me.

Read the commit, it's 50 messages "by conversation"

breversa commented 3 years ago

Would regularly vaccuming the database speed up the start ?

https://github.com/signalapp/Signal-Desktop/issues/3713#issuecomment-804074415

josh-signal commented 3 years ago

Hey, you meant the 1622fdf commit was actually implemented in the latest beta? But I think it's wrong because when I tried to cold start the app today morning, it was loading more than 50 messages for me.

Read the commit, it's 50 messages "by conversation"

Sorry that commit you've linked to is regarding opening a conversation.

From the release notes:

Performance improvements let you switch between chats faster than ever.

abeyaustin commented 3 years ago

Hey, you meant the 1622fdf commit was actually implemented in the latest beta? But I think it's wrong because when I tried to cold start the app today morning, it was loading more than 50 messages for me.

Read the commit, it's 50 messages "by conversation"

Sorry that commit you've linked to is regarding opening a conversation.

From the release notes:

Performance improvements let you switch between chats faster than ever.

Oh, that's a bummer. I just have ONE chat and this is what my cold start looks like every evening. image

And it takes absolutely FOREVER!

Wouldn't it be possible to introduce an option to limit the loading of cold start messages to just 50 ?

PiFlUn commented 3 years ago

I don't understand why signal has to load all the messages of EVERY SINGLE chat at all. I have currently something like 3-5 chats which I actively use and god knows how many which haven't read in months or even years.

stephematician commented 3 years ago

I have the same issue; I find that on my new laptop with SSD the performance is somewhat reasonable (a thousand messages sync in a few seconds) whereas on an older laptop with magnetic HDD it takes ... ten minutes? If I were looking at this bottleneck; I'd absolutely be looking at access to storage - the difference in CPU, memory, etc, is not a factor of 100 for my two machines.

Caveat: I haven't tried running the Signal beta version

Additional edit: I also noticed that the system app jdb2 (via iotop) was absolutely crushing I/O, with multiple 'signal' threads writing, perhaps multiple threads competing for access and the file system is having a rough go of resolving conflict?

Kwasow commented 3 years ago

Start up times seem to be greatly improved after 5c8cb6b89be45a1c1dd790f87c4aa25fd0f749b4 It's over twice as fast in my case, but there is still room for improvement.

scottnonnenberg-signal commented 3 years ago

@PiFlUn Because this question comes up a lot, I'm going to step in and provide an answer. We must load all messages in order, because they are all encrypted in a stateful way, with progression forward in time. If we decrypt things out of order, things might go wrong. Note that because of the encryption, we don't even know what conversation a message is going to end up in until the decryption completes.

So we cannot possibly load just the 50 most recent messages in a given conversation. But we are absolutely working on improving performance, as you've seen from recent commits. Stay tuned!

BarbossHack commented 3 years ago

@PiFlUn Because this question comes up a lot, I'm going to step in and provide an answer. We must load all messages in order, because they are all encrypted in a stateful way, with progression forward in time. If we decrypt things out of order, things might go wrong. Note that because of the encryption, we don't even know what conversation a message is going to end up in until the decryption completes.

So we cannot possibly load just the 50 most recent messages in a given conversation. But we are absolutely working on improving performance, as you've seen from recent commits. Stay tuned!

First really good answer, thank you, now it's crystal clear for everyone that Signal prefers security over performance (good thing) 👍