Upgrade to 5.0.2 - settings-omnichannel only gives me an empty grey screen

tom666-debug commented 2 years ago

Omnichannel feature cannot be activated or configured as the according settings cannot be opened. When I try to open the Omnichannel setting page RC just gives me blank grey screen. The remaining other setting items are working as expected.

Steps to reproduce:

Go to Administration - Settings
Click on Omnichannel

Expected behavior:

Omnichannel Settings page will be opened

Actual behavior:

RC only gives me blank grey screen

Server Setup Information:

Version of Rocket.Chat Server: 5.0.2
Operating System: Debian 10.7
Deployment Method: Docker
Number of Running Instances: 1
DB Replicaset Oplog: ?
NodeJS Version: v14.19.3
MongoDB Version: 5.0.6 / wiredTiger (oplog Enabled)

Client Setup Information

Desktop App or Browser Version: Both - Desktop App and Browser version
Operating System: Windows 11

Relevant logs:

rc Docker container: _rocketchat_rocketchat_1_logs.txt

sleeplessnight2 commented 2 years ago

this is happening since 5.0.0 at my installation...

hibobmaster commented 2 years ago

Same here.

tom666-debug commented 2 years ago

Still broken with version 5.0.3

hibobmaster commented 2 years ago

@sampaiodiego

sampaiodiego commented 2 years ago

@KevLehman @murtaza98

murtaza98 commented 2 years ago

Hi @tom666-debug @sleeplessnight2 @hibobmaster Can you guys please share some console logs from the browser when you get the grey screen? Because I just checked to access the settings within a few env running the 5.0.2 version & they all seem to be loading the settings page without any issue

hibobmaster commented 2 years ago

@murtaza98 Thanks for your reply, here is the error log.

Error: Minified React error #130; visit https://reactjs.org/docs/error-decoder.html?invariant=130&args[]=undefined&args[]= for the full message or use the non-minified dev environment for full errors and additional helpful warnings.

tom666-debug commented 2 years ago

here you go:

Screenshot 2022-08-14 110055

urangsabah commented 2 years ago

also experiencing similar issue with our omnichannel rollout

Server Setup Information:

Version of Rocket.Chat Server: 5.0.3 Operating System: Ubuntu 20.04.2 LTS Deployment Method: Docker Number of Running Instances: 1 DB Replicaset Oplog: (default) NodeJS Version: v14.19.3 MongoDB Version: 4.4.12 / wiredTiger (oplog Enabled)

Client Setup Information

Desktop App or Browser Version: Chrome 104.0.5112.81, Firefox 103.0.2 (x64) Operating System: Windows 10

Additional context

Last known working version was rocketchat 4.8.1 on mongodb 4.0.x - upgraded to current config on 2022-08-14 01:00:00 GMT+0

Also broken on Android App > Admin Panel > Settings > "Omnichannel" > Click Button >> "white screen"

Relevant logs:

Chrome Dev Console

31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:13 fuselage: 0.31.16
31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1558 MessageType.render is deprecated. Use MessageType.message instead. livechat_webrtc_video_call
registerType @ 31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1558
31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124 Error: Minified React error #130; visit https://reactjs.org/docs/error-decoder.html?invariant=130&args[]=undefined&args[]= for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at vu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:107534)
    at d (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:53373)
    at g (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:54643)
    at 31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:56704
    at ui (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:67706)
    at Rs (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:113409)
    at nu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98840)
    at tu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98768)
    at eu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98631)
    at Gs (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:95485)
Ti @ 31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124
31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124 Uncaught Error: Minified React error #130; visit https://reactjs.org/docs/error-decoder.html?invariant=130&args[]=undefined&args[]= for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
    at vu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:107534)
    at d (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:53373)
    at g (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:54643)
    at 31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:56704
    at ui (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:67706)
    at Rs (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:113409)
    at nu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98840)
    at tu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98768)
    at eu (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:98631)
    at Gs (31420213d29e97049165764c3dedd3969b6b7b63.js?meteor_js_resource=true:1124:95485)

Firefox Dev Console

tom666-debug commented 2 years ago

Still broken with version 5.0.4

tom666-debug commented 2 years ago

@murtaza98 Just wondering why the status of this ticket is still "need more info"?! What additional info can we provide?

PlusaN commented 2 years ago

Description:

Omnichannel settings don't work.

Steps to reproduce:

Go to Administration - Settings
Click on Omnichannel

Expected behavior:

Omnichannel Settings page will be opened

Actual behavior:

RC only gives me blank grey screen

Server Setup Information:

Version of Rocket.Chat Server: 5.0.4
Operating System: Debian 10.12
Deployment Method: tar
Number of Running Instances: 1
DB Replicaset Oplog: -
NodeJS Version: 14.19.3
MongoDB Version: 4.4.16

Client Setup Information

Desktop App or Browser Version: Rocket.Chat.Electron 3.8.8, Firefox 104.0, Chromium 104.0.5112.81
Operating System: Windows 10

Additional context

Last known working version was rocketchat 4.8.2

Relevant logs:

chromium.log

tom666-debug commented 2 years ago

Version 5.0.5 still broken ... ridiculous

localshot commented 2 years ago

Acknowledged. 5.0.5 - grey screen in Omnichannel settings

KevLehman commented 2 years ago

Hey! Small update We have been trying to reproduce this issue for quite a while, with multiple envs running versions from 4.8 to 5.1 (candidate) and issue doesn't appear.

A few questions:

Do this happen only with Omnichannel settings, or does it happen with other sections too?
Does the same happen on incognito windows? Different browsers? After clearing cache/local storage?
What was the last version that was working for you?

We understand this issue prevents you from using that part of the product. As workaround, you can update settings individually by using the REST API. We'll keep trying to find why this is happening.

urangsabah commented 2 years ago

@KevLehman

greetings - not OP - but to anwer your questions since everybody here is reporting the same browser blank screen (and console logs) on this. i should note that if we leave the blank screen open, it will eventually hang that specific tab/browser session and make it unresponsive

re: your questions

Do this happen only with Omnichannel settings, or does it happen with other sections too? ONLY ON OMNICHANNEL SETTINGS, that is Administration > Settings > "Omnichannel" > Open
Does the same happen on incognito windows? Different browsers? After clearing cache/local storage? YES, we have tested this on Firefox, Chrome, Edge, on a variety of OSes (Win10, Win11, Chromebook, Ubuntu). Same error on incognito/private modes, with cleared storage.
Last known working version is rocketchat 4.8.1, mongodb 4.2 (see upgrade timeline below)

timeline

For clarity, we planned an upgrade of our docker-based rocketchat instance and had a planned roll-out of omnichannel - which has since stalled due to this bug (all steps below occurred on same day mid-August over 3 hours):

rocketchat version was initially 4.8.1 - at this point Administration > Settings > "Omnichannel" was working
mongodb upgraded from 4.0 mmapv1 to 4.0 wiredtiger then 4.2 wiredtiger
the following omnichannel settings were changed via the web gui: omnichannel > enable omnichannel > routing algorithm: manual omnichannel > business hours > (set all to 24x7 - all days, 12am-12pm, but was not enabled) omnichannel > time for livechat abandoment > 3 hours
omnichannel was tested at this point (on rc 4.8.1, mongo 4.2 wiredtiger), no issues were noted
rocketchat upgraded to 5.0.3
mongodb upgraded from 4.2 to 4.4
it was at this point the blank screen on the web gui for Administration > Settings > "Omnichannel" was observed

additional notes

unfortunately for (5), we don't have a complete list of the initial modifications to the administration settings, but these were the most signficant settings that we recall, since we intended to tweak and confirm them after the final rc upgrade to 5.x
our omnichannel had no integrations configured (i.e. no facebook, etc)
original report is here

further investigation via API?

noted on the REST API, and was considering this as a workaround - in fact, if you need us to check specific omnichannel settings we are using, let us know which keys to dump via API for further investigation (we are not running livechat in production yet)

tom666-debug commented 2 years ago

Hi,

As requested:

Do this happen only with Omnichannel settings, or does it happen with other sections too? As stated in my initial post, only Omnichannel seems to be affected

Does the same happen on incognito windows? Different browsers? After clearing cache/local storage? Different browsers (Chrome (incl. incognito), Edge, FireFox) - with cleared cache and even on a fresh installation - allways the same result

What was the last version that was working for you? RC 4.8.4 - Mongo 5.0.6

imahbub commented 2 years ago

Same! I vote!

tom666-debug commented 2 years ago

Version 5.1.0 still broken - of course. It's becoming embarrassing.

fabiosurrage commented 2 years ago

I have exactly the same problem. Working omnichannel settings page with version 4.8.2. Empty page with version 5.0.5 and 5.1.1.

But I have one test instance with version 5.0.5 that works fine, then I am searching for some proxy problem or diference between instalations ...

tom666-debug commented 2 years ago

Confirmed, version 5.1.1 still broken.

I know it is really tricky to analyse an issue that you can't hardly reproduce. But no feedback or further questions at all regarding to this issue? Wow! For me it just feels like they dont care at all - pretty sad.

billynoah commented 2 years ago

This is affecting us as well.

billynoah commented 2 years ago

Hey! Small update We have been trying to reproduce this issue for quite a while, with multiple envs running versions from 4.8 to 5.1 (candidate) and issue doesn't appear.

A few questions:
1. Do this happen _only_ with Omnichannel settings, or does it happen with other sections too?

2. Does the same happen on incognito windows? Different browsers? After clearing cache/local storage?

3. What was the last version that was working for you?
We understand this issue prevents you from using that part of the product. As workaround, you can update settings individually by using the REST API. We'll keep trying to find why this is happening.

Hi @KevLehman - Since you're having trouble reproducing the issue, I could provide access to our instance (where Omnichannel settings are broken) if someone from your team wants to use it as a case example. Would that help?

fabiosurrage commented 2 years ago

I got this erro in log when accessing the page /admin/settings/Omnichannel

{"level":50,"time":"2022-09-12T12:58:20.713Z","pid":1,"hostname":"rocketchat-server-deployment-6475c587d5-z2bww","name":"System","msg":"Exception while invoking method autoTranslate.getSupportedLanguages 'Auto-Translate is disabled [error-autotranslate-disabled]'"}

dudanogueira commented 2 years ago

{"level":50,"time":"2022-09-12T12:58:20.713Z","pid":1,"hostname":"rocketchat-server-deployment-6475c587d5-z2bww","name":"System","msg":"Exception while invoking method autoTranslate.getSupportedLanguages 'Auto-Translate is disabled [error-autotranslate-disabled]'"}

not sure this error is related, as I also see this error with a working instance, FWIW

KevLehman commented 2 years ago

Hey! We have a possible fix. Thanks to @fabiosurrage that provided an env with the issue happening 😬 (and also thanks to @billynoah for offering help too!). We were able to reproduce it locally and investigate it.

The actual reason of the issue is still unclear. Possibly, a really old DB migration created a setting (an EE setting) that shouldn't be there. This invalid setting was causing issues, as it didn't have all the data we expect from a setting and therefore, broke the UI. This seems to have only happened to envs that have been fully community always, without any touch of EE features (cause that would've fixed the setting automatically).

The problem:

UI was receiving an invalid setting

    {
        "_id": "Canned_Responses_Enable",
        "_updatedAt": "2021-10-26T10:59:20.368Z",
        "group": "Omnichannel"
    },

As this setting doesn't have all the data required, UI broke (here's a valid setting for reference)

    {
        "_id": "Livechat_Allow_collect_and_store_HTTP_header_informations",
        "_updatedAt": "2021-11-12T09:54:14.872Z",
        "autocomplete": true,
        "blocked": false,
        "createdAt": "2021-11-12T09:54:14.872Z",
        "group": "Omnichannel",
        "hidden": false,
        "i18nDescription": "Allow_collect_and_store_HTTP_header_informations_description",
        "i18nLabel": "Allow_collect_and_store_HTTP_header_informations",
        "packageValue": false,
        "public": true,
        "sorter": 8000,
        "ts": "2021-11-12T09:54:14.872Z",
        "type": "boolean",
        "value": false,
        "valueSource": "packageValue",
        "secret": false,
        "section": "GDPR",
        "enterprise": false,
        "requiredOnWizard": false,
        "enableQuery": "{\"_id\":\"Livechat_enabled\",\"value\":true}",
        "env": false
    },

The solution

Remove the offending setting from the DB. Sadly, this will involve some DB touching (I'll work on doing a proper code fix for a following release, so consider this as a "workaround")

// Remove setting with ID canned_responses_enable from rocketchat_settings collection
// Note: driver syntax may vary
db.collection('rocketchat_settings').deleteOne({ _id: 'Canned_Responses_Enable' }) // From a MongoDB shell

Then, clear browser data (Go to Dev tools => Application => Storage => Clear data) and refresh.

That should work 😬

Thanks for your patience, just pls have us a little bit more of patience while we develop the actual fix (for cases where you don't have DB access or other special cases too)

billynoah commented 2 years ago

@KevLehman - thanks that worked!

urangsabah commented 2 years ago

@KevLehman - the workaround works - thank you!

here's what we did:

rs0:PRIMARY> db.rocketchat_settings.find({ _id: 'Canned_Responses_Enable'  })
{ "_id" : "Canned_Responses_Enable", "_updatedAt" : ISODate("2021-12-20T03:42:28.590Z"), "group" : "Omnichannel" }

rs0:PRIMARY> db.rocketchat_settings.deleteOne({ _id: 'Canned_Responses_Enable'  })
{ "acknowledged" : true, "deletedCount" : 1 }

then...

Clear browser cache/storage
- Chrome > Dev Tools > Application > Storage > Clear site data
- Firefox > Lock Icon (URL bar) > Clear cookies and site data...
Login again and check on Administration > Settings > Omnichannel > Open

only happened to envs that have been fully community always

...is right - we've been running on community since 3.x back in 2019, and progressively upgraded on community till 5.x recently

fabiosurrage commented 2 years ago

Thanks a lot ! Worked!

privnote42 commented 1 year ago

Hello, is there an official solution to this in the meantime?

the workaround does not work for me.

At the command db.collection('rocketchat_settings').deleteOne({ _id: 'Canned_Responses_Enable' })

I get an error message

uncaught exception: TypeError: db.collection is not a function : @(shell):1:1

and when I use the commands from @urangsabah then I get no output on the first command and the following on the second command: { "acknowledged" : true, "deletedCount" : 0 }

deleting the cache of the page did not help any more than the private mode of the browser.

I am using rocketchat 5.3.1 and mongodb 4.4.18

tom666-debug commented 1 year ago

and when I use the commands from @urangsabah then I get no output on the first command and the following on the second command: { "acknowledged" : true, "deletedCount" : 0 }

But you did a "use rocketchat" first after starting mongosh, right? And I can confirm that there is no official solution available yet, even if the root cause is known .... pretty sad.

privnote42 commented 1 year ago

I forgot that of course :sweat_smile: Now the workaround is working, thank you very much!

RocketChat / Rocket.Chat