After Slack import: Exception while invoking method 'loadHistory' TypeError: Cannot destructure property `username` of 'undefined' or 'null'. at app/utils/server/lib/normalizeMessagesForUser.js

[nooblag]

18764 # Description:

After successful Slack import, some channels are not available. Clicking on them produces "Internal Server Error [500]" or "Undefined" message at top right of Rocket Chat screen in red. Below is what is listed in the log.

Have tried a fresh reboot and still persists.

Update: After much trial and error, the setting that seems to break things for me is: Administration -> Layout -> User Interface:

Use Real Name YES

Slack imports seem to work fine for me with this setting off.

Steps to reproduce:

1. Import Slack
2. Click on channel, error

Expected behavior:

Should work fine

Actual behavior:

Clicking on them produces "Internal Server Error [500]" message at top right of Rocket Chat screen in red. Below is what is listed in the log.

Server Setup Information:

• Version of Rocket.Chat Server: 3.6.2
• Operating System: Ubuntu 18 LTS
• Deployment Method: Normal
• Number of Running Instances: 1
• Node Version | v12.14.0
• Mongo Version | 4.0.19

Client Setup Information

• Desktop App or Browser Version: Firefox 80.0.1
• Operating System: Linux Mint 19

Additional context

Relevant logs:

I20200922-16:50:06.410(10) Exception while invoking method 'loadHistory' TypeError: Cannot destructure property username of 'undefined' or 'null'. at app/utils/server/lib/normalizeMessagesForUser.js:34:37 at Array.forEach () at app/utils/server/lib/normalizeMessagesForUser.js:34:28 at Array.forEach () at normalizeMessagesForUser (app/utils/server/lib/normalizeMessagesForUser.js:26:11) at loadMessageHistory (app/lib/server/functions/loadMessageHistory.js:31:19) at MethodInvocation.loadHistory (server/methods/loadHistory.js:33:10) at MethodInvocation.methodsMap. (app/lib/server/lib/debug.js:67:34) at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1771:12) at packages/ddp-server/livedata_server.js:719:19 at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1234:12) at packages/ddp-server/livedata_server.js:717:46 at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1234:12) at packages/ddp-server/livedata_server.js:715:46 at new Promise () at Session.method (packages/ddp-server/livedata_server.js:689:23) at packages/ddp-server/livedata_server.js:559:43

[robertfromont]

I'm also getting this on version 3.6.3. It's not on all channels and not always at the latest messages.

• On two channels, I can't see any messages at all when I open the channel.
• On other channels, much of the message history displays fine, but if I go back far enough, this error appears and I can't go back any further.

[robertfromont]

By going through the channel history and searching for individual messages to see if they appear, I think I've found the culprit message.

In the search results pane, it shows up, but without the matching text, and when I try to 'Jump to Message' the same error as above appears in the logs.

[robertfromont]

After some sleuthing in MongoDB, I found that the message looked (somewhat) like this:

{
    "_id" : "slack-...",
    "ts" : ISODate("2020-04-06T21:19:14Z"),
    "msg" : "Thanks @robertfromont...",
    "rid" : "3WcfZG9K5Y8TBz39Y",
    "u" : {
        "_id" : "sAropWB2bfcCGa34s",
        "username" : "..."
    },
    "mentions" : [
        null
    ],
    "channels" : [ ],
    "_updatedAt" : ISODate("2020-09-14T21:08:30.290Z")
}

I thought "mentions" : [null] looked suspicious, so I deleted that attribute, and the channel now loads. That particular message has the @ mention broken.

Running this in mongo:

db.rocketchat_message.update( {"mentions" : [ null ] }, { $unset : { mentions : true } }, { multi : true } )

...seems to have fixed the issue on all the channels. It also breaks the @ mention links to the referred-to user, which is not too important in our case (it had happened in 26 out of almost 16K messages, which is tolerably low)

I checked whether the user_id mentioned in the file exported by Slack was linkable to the user record in Rocket Chat and it was - the Slack export included:

{
    "client_msg_id": "...",
    "type": "message",
    "text": "Thanks <@UDL8Z5VKM>...

...and the Rocket Chat user record includes:

{
    "_id" : "bCKST9viG9Xh3WEbx",
    "createdAt" : ISODate("2020-09-02T19:52:31.541Z"),
...
    "importIds" : [
        "UDL8Z5VKM"
    ]
}

So I guess there's an intermittent bug where mentions are failing the RC user lookup.

From my perspective, having a broken @ mention is less bad than having a channel that won't load, so maybe a fallback if it comes back as null is to not insert the "mentions" attribute is in order.

[nooblag]

I'm also getting this on version 3.6.3. It's not on all channels and not always at the latest messages.

• On two channels, I can't see any messages at all when I open the channel.
• On other channels, much of the message history displays fine, but if I go back far enough, this error appears and I can't go back any further.

I can also confirm that exact behaviour.

[nooblag]

After much trial and error, the setting that seems to break things for me is: Administration -> Layout -> User Interface:

Use Real Name YES

Slack imports seem to work fine for me with this setting off.

[robertfromont]

It turns out that my mongo hacking above is not quite sufficient, as there are cases where the "mentions" array contains several entries, of which only some are null, e.g.:

{
    "_id" : "slack-...",
    "ts" : ISODate("2020-02-17T03:15:16Z"),
    "msg" : "Just looking at schedules, @somebody, and it looks like two times we both might have free are Mondays @11am, or Weds @12pm.  Would either of those be good times?",
    "rid" : ",,,",
    "u" : {
        "_id" : "...", ...
    },
    "mentions" : [
        {
            "_id" : "...", ...
        },
        null,
        null
    ],
    "channels" : [ ],
    "_updatedAt" : ISODate("2020-09-14T21:07:47.463Z")
}

i.e. the text Mondays @11am, or Weds @12pm has been incorrectly identified as including two @-mentions.

The mongo command to repair these, which will remove the null elements even when there are other non-null elements in the same array, is:

db.rocketchat_message.update({"mentions" : { $elemMatch : { $eq : null} }}, { $pull : { mentions : null } }, { multi : true } )

[robertfromont]

To support what @nooblag found, I did have Use Real Name YES set when I did the import.

[nooblag]

Any ideas?

[nooblag]

Issue persists in 3.7.1, but error message appears as "[undefined]" and not "Internal Server Error [500]" as before.

[nooblag]

Issue persists in 3.9.3 and also 3.10.3.

[jasonbayton]

Still present in 3.11.1, and quite disheartening to see the issue entirely ignored up to this point.

[nooblag]

Aha! I think I have some more valuable information about this bug.

I think the problem might have something to do with the way the "rapid post" types of messages from Slack are handled by Rocket Chat (i.e. the messages that are sent close together time-wise and hence don't have a username/avatar and timestamp):

Slack

Rocket Chat

You'll notice in the latter that Rocket Chat is more lax with handling the name in rapid fire messages (and maybe the missing name is blank in the last message of the Slack import data or something?) and that's why we're getting exceptions/empties/null/whatever in Rocket Chat?

I think I've narrowed it down to these types of messages, because also, I noticed if I deleted the messages that were imported from Slack but have no names next to them in Rocket Chat, the channel loaded fine again and Use Real Names also works great throughout again. It seems these sorts of messages are throwing the error?

I hope this is helpful to reproduce the problem?!

[nooblag]

Closing, as gave up on Rocket Chat many years ago, bugs go unrecognised and unfixed.