Whatsapp bridging stopped working: Client outdated

[Vetgans]

Describe the bug Bridge has been running succesfully for months and months Whatsapp bridging stopped working last night with log message that the client is outdated

To Reproduce Steps to reproduce the behavior: Build latest development version with go install -tags nomsteams,nozulip,whatsappmulti github.com/42wim/matterbridge@master Start the bridge. It will fail. The message of the client being outdated will appear.

Expected behavior The bridge running succesfully like before

Screenshots/debug logs

[0000]  INFO main:         Running version 1.26.1-dev 
[0000]  INFO main:         WARNING: THIS IS A DEVELOPMENT VERSION. Things may break.
.
.
.
[0000]  INFO router:       Starting bridge: whatsapp.mywhatsapp 
11:56:00.501 [Client ERROR] Client outdated (405) connect failure (client version: 2.2332.15)
[0015]  INFO whatsapp:     QR channel result: err-client-outdated
11:56:00.521 [Client/Socket ERROR] Error reading from websocket: websocket: close 1006 (abnormal closure): unexpected EOF
[0026]  INFO whatsapp:     WhatsApp connection successful
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0xf241c0]

goroutine 1 [running]:
github.com/42wim/matterbridge/bridge/whatsappmulti.(*Bwhatsapp).Connect(0x438e900)
        /home/pi/go/pkg/mod/github.com/42wim/matterbridge@v1.26.1-0.20230805184319-56e7bd01ca09/bridge/whatsappmulti/whatsapp.go:125 +0x62c
github.com/42wim/matterbridge/gateway.(*Router).Start(0x438e500)
        /home/pi/go/pkg/mod/github.com/42wim/matterbridge@v1.26.1-0.20230805184319-56e7bd01ca09/gateway/router.go:76 +0x300
main.main()
        /home/pi/go/pkg/mod/github.com/42wim/matterbridge@v1.26.1-0.20230805184319-56e7bd01ca09/matterbridge.go:65 +0x3f0

[tzagim]

I have the same problem, it seems to be a problem with the following module: https://github.com/tulir/whatsmeow/issues/584

[Vetgans]

I have the same problem, it seems to be a problem with the following module: tulir/whatsmeow#584

In that topic they discuss login problems with the QR-code, but matterbridge stops even before showing a QR-code.

I don't know but it is probably just a matter of the whatsmeow version (and therefore the web version) being too old. After the bridge stopped working this night, i built a new one this morning, but that is still exactly the same as the one that was running since september 2023.

So it's not unlikely for the used whatsmeow version to be outdated. @42wim, is it possible for you to update the whatsmeow version for the self-build process?

[david-botelho-mariano]

Same here...

[Vetgans]

@tzagim : I tried to build from your forked repo, but get the following error:

# go.mau.fi/whatsmeow/socket
../go/pkg/mod/go.mau.fi/whatsmeow@v0.0.0-20240520180327-81f8f07f1dfb/socket/noisesocket.go:27:22: undefined: atomic.Bool
note: module requires Go 1.21

So can you please update to the latest Go module as well?

[Schattenkanzler]

Same here ☝🏽

[tzagim]

@Vetgans

@tzagim : I tried to build from your forked repo, but get the following error:

# go.mau.fi/whatsmeow/socket
../go/pkg/mod/go.mau.fi/whatsmeow@v0.0.0-20240520180327-81f8f07f1dfb/socket/noisesocket.go:27:22: undefined: atomic.Bool
note: module requires Go 1.21

So can you please update to the latest Go module as well?

Try now

[rasysdev]

For me it compiled ok:

root@sede-automations:~/go/bin# CGO_ENABLED=0 go install -tags whatsappmulti github.com/tzagim/matterbridge@master go: finding module for package github.com/42wim/matterbridge/version go: finding module for package github.com/42wim/matterbridge/gateway go: finding module for package github.com/42wim/matterbridge/gateway/bridgemap go: finding module for package github.com/42wim/matterbridge/bridge/config go: found github.com/42wim/matterbridge/bridge/config in github.com/42wim/matterbridge v1.26.0 go: found github.com/42wim/matterbridge/gateway in github.com/42wim/matterbridge v1.26.0 go: found github.com/42wim/matterbridge/gateway/bridgemap in github.com/42wim/matterbridge v1.26.0 go: found github.com/42wim/matterbridge/version in github.com/42wim/matterbridge v1.26.0 go: finding module for package go.mau.fi/util/jsontime go: finding module for package go.mau.fi/util/retryafter go: finding module for package github.com/rs/zerolog go: finding module for package go.mau.fi/util/random go: found github.com/rs/zerolog in github.com/rs/zerolog v1.32.0 go: found go.mau.fi/util/random in go.mau.fi/util v0.4.2 go: found go.mau.fi/util/retryafter in go.mau.fi/util v0.4.2 go: found go.mau.fi/util/jsontime in go.mau.fi/util v0.4.2

But when running i get this error:

[0000] INFO router: Starting bridge: whatsapp.mywhatsapp [0000] FATAL main: Starting gateway failed: Bridge whatsapp.mywhatsapp failed to start: failed to connect to database: failed to upgrade database: foreign keys are not enabled

Something about this: https://github.com/tulir/whatsmeow/issues/454

I'm not really a coder, so can't help too much.

[Vetgans]

Compiled OK here as well.

I can confirm what @rasysdev already reported and got the same errors:

[0012]  INFO router:       Starting bridge: whatsapp.mywhatsapp 
[0012] FATAL main:         Starting gateway failed: Bridge whatsapp.mywhatsapp failed to start: failed to connect to database: failed to upgrade database: foreign keys are not enabled

So it's more complicated...

[rasysdev]

Really need this wpp bridge to work, so i asked a developer i know to fix it: https://github.com/42wim/matterbridge/pull/2148 I tested locally and is running 100%

[Vetgans]

Really need this wpp bridge to work, so i asked a developer i know to fix it: #2148 I tested locally and is running 100%

Nice!

But did you build it before https://github.com/42wim/matterbridge/commit/3b31b5466d1566104376c57acc5ffb2df255790d ?

Because when trying to build it, it will give this error:

go: downloading github.com/rodrigozietlow/matterbridge v0.0.0-20240522031003-3b31b5466d15
go: github.com/rodrigozietlow/matterbridge@master: version constraints conflict:
        github.com/rodrigozietlow/matterbridge@v0.0.0-20240522031003-3b31b5466d15: parsing go.mod:
        module declares its path as: github.com/42wim/matterbridge
                but was required as: github.com/rodrigozietlow/matterbridge

[tzagim]

But did you build it before 3b31b54 ?

@Vetgans You can use:

github.com/rodrigozietlow/matterbridge@f056afa3e578c2ba2765ab2cb33922971516c375

You will get the same error (https://github.com/42wim/matterbridge/issues/2146#issuecomment-2123034942).

[Vetgans]

@Vetgans You can use:

github.com/rodrigozietlow/matterbridge@f056afa3e578c2ba2765ab2cb33922971516c375

You will get the same error (#2146 (comment)).

I can confirm this...

@rasysdev: You mentioned having it working 100%. Can you explain how you built it exactly, preferably with the exact command for the used repository and its state (in terms of commits)?

[rasysdev]

git clone https://github.com/rodrigozietlow/matterbridge.git enter in the folder CGO_ENABLED=0 go install -tags whatsappmulti ./

then i went in /root/go/bin and the "mattermost" there worked.

Maybe some URLs need to be changed to compile in the normal way...

But i did this way and is still running.

Edit: I cloned from the branch master.

[Vetgans]

Edit: I cloned from the branch master.

Building from master gives me the following error:

go: github.com/rodrigozietlow/matterbridge@master: version constraints conflict:
        github.com/rodrigozietlow/matterbridge@v0.0.0-20240522031003-3b31b5466d15: parsing go.mod:
        module declares its path as: github.com/42wim/matterbridge
                but was required as: github.com/rodrigozietlow/matterbridge

Understandable, as @rodrigozietlow made commit https://github.com/42wim/matterbridge/commit/3b31b5466d1566104376c57acc5ffb2df255790d for the Pull request.

Building without that last commit by using github.com/rodrigozietlow/matterbridge@f056afa3e578c2ba2765ab2cb33922971516c375 results in a proper file.

Running that file however still throws the error that it fails to connect to the database:

[0000]  INFO router:       Starting bridge: whatsapp.mywhatsapp 
[0000] FATAL main:         Starting gateway failed: Bridge whatsapp.mywhatsapp failed to start: failed to connect to database: failed to upgrade database: foreign keys are not enabled

I've done everything all over again, but still the same errors.

Is it perhaps possible for you to upload the resulting file?

[rasysdev]

Did you build id with ./ ? CGO_ENABLED=0 go install -tags whatsappmulti ./

What resulting file are you refering to? The "mattermost" file from go bin?

[Vetgans]

Did you build id with ./ ? CGO_ENABLED=0 go install -tags whatsappmulti ./

Yep

What resulting file are you refering to? The "mattermost" file from go bin?

The resulting matterbridge file

[rasysdev]

See the history:

353 cd /tmp/ 354 git clone https://github.com/rodrigozietlow/matterbridge.git 355 cd matterbridge/ 356 CGO_ENABLED=0 go install -tags whatsappmulti ./ 357 git pull 358 CGO_ENABLED=0 go install -tags whatsappmulti ./ 359 ls 360 cd /root/go/bin/ 361 ls 362 ./matterbridge

The "matterbridge" file isn't it a binary? "File size too big: 25 MB are allowed, 31 MB were attempted to upload."

Can you debug by the binary? In that case i will somehow send it here.

[rasysdev]

have you tryed deleting the old database file?

[pegelf]

What didn't worked was just using the go install command and just replacing the url like this: go install -tags whatsappmulti github.com/rodrigozietlow/matterbridge@master

The issue is, that the URL is also referenced in other files.

But the instructions of @rasysdev worked well:

cd ~
mv go go_old
git clone https://github.com/rodrigozietlow/matterbridge.git
cd matterbridge
CGO_ENABLED=0 go install -tags whatsappmulti,nomsteams,nozulip ./
cp ~/go_old/bin/matterbridge.toml ~/go/bin/matterbridge.toml

Then you will have a new matterbridge executable in the ~/go/bin/matterbridge folder that works again until the pull request is merged.

[pegelf]

@rasysdev In my commands, I removed msteams and zulip. Maybe this could also work for you?

But after I reinstalled the fork and used a new database, I still get an error after logging in:

[0011] INFO main: Gateway(s) started successfully. Now relaying messages 16:12:56.406 [Client WARN] Received stream end frame 16:12:56.406 [Client INFO] Got device removed stream error, sending LoggedOut event and deleting session 16:12:56.507 [Client/Socket ERROR] Error reading from websocket: websocket: close 1006 (abnormal closure): unexpected EOF

So I think I have to wait for a few errors, because I'm not able to resolve that.

[naveenyeluru]

What didn't worked was just using the go install command and just replacing the url like this: go install -tags whatsappmulti github.com/rodrigozietlow/matterbridge@master

The issue is, that the URL is also referenced in other files.

But the instructions of @rasysdev worked well:

cd ~
mv go go_old
git clone https://github.com/rodrigozietlow/matterbridge.git
cd matterbridge
CGO_ENABLED=0 go install -tags whatsappmulti,nomsteams,nozulip ./
cp ~/go_old/bin/matterbridge.toml ~/go/bin/matterbridge.toml

Then you will have a new matterbridge executable in the ~/go/bin/matterbridge folder that works again until the pull request is merged.

These steps worked perfectly. Thank you @pegelf

[Vetgans]

I started from scratch on another machine, and finally got it running.

Still some errors are being reported:

20:22:40.502 [Client ERROR] Failed to save push name of XXXXXXXXXXX@s.whatsapp.net in device store: database is locked (5) (SQLITE_BUSY)
20:22:40.685 [Client ERROR] Failed to store app state sync key 00000000347D: database is locked (5) (SQLITE_BUSY)
20:22:40.686 [Client ERROR] Failed to do initial fetch of app state critical_block: failed to get app state critical_block version: database is locked (5) (SQLITE_BUSY)
20:22:40.686 [Client ERROR] Failed to do initial fetch of app state critical_unblock_low: failed to get app state critical_unblock_low version: database is locked (5) (SQLITE_BUSY)
20:22:40.687 [Client ERROR] Failed to do initial fetch of app state regular_high: failed to get app state regular_high version: database is locked (5) (SQLITE_BUSY)
20:22:40.688 [Client ERROR] Failed to do initial fetch of app state regular: failed to get app state regular version: database is locked (5) (SQLITE_BUSY)
20:22:40.689 [Client ERROR] Failed to do initial fetch of app state regular_low: failed to get app state regular_low version: database is locked (5) (SQLITE_BUSY)
20:22:41.341 [Client INFO] Sending key request for app state keys [00000000347d]
20:22:41.350 [Client ERROR] Failed to sync app state after notification: failed to decode app state critical_block patches: failed to get key 00000000347D to verify patch v1 MACs: didn't find app state key
20:22:41.513 [Client ERROR] Failed to sync app state after notification: failed to decode app state critical_unblock_low patches: failed to get key 00000000347D to verify patch v1 MACs: didn't find app state key

Apart from that it seems to be running fine.

[d33psky]

See the history: ...

I can confirm that this works. Thanks !

[Vetgans]

Everything still working fine.

I did get these errors later last night

22:37:17.202 [Client/Socket ERROR] Error reading from websocket: websocket: close 1006 (abnormal closure): unexpected EOF
22:37:17.203 [Client/Socket ERROR] Error closing websocket: tls: failed to send closeNotify alert (but connection was closed anyway): write tcp 192.168.1.12:56036->XXX.XXX.XXX.XX:443: write: connection reset by peer
22:37:27.458 [Client INFO] Successfully authenticated

but it continued to run.

Later last night i restarted with the session file, and ever since the bridge has been running well without any errors.

Thanks to everyone who helped out, and in particular of course @rodrigozietlow for the vital commits.