MQTT disconnection issue. (also some questions)

[harmine21]

Main Issue

I'll keep this as short as possible. I'm using ig.realtime.on() to listen for messages 24/7 and print them on console. After running 2 or 3 days I noticed that the program stopped receiving new messages but it didn't error out. I would assume that if the connection got somehow terminated it would error.

I first assumed that my public IP changed (since I don't have a static one) and that messed it up. After further investigation I found out that it wasn't that since my IP had not changed for a week.

What could be causing this no-error disconnection? After restarting the script, it starts picking up the new messages fine again. I know for a fact it's not a login expired error since I'm using the same state login file the whole time. Is there some way to at least catch the error so I can restart my script when it happens?

Secondary-priority questions. Feel free to ignore :

• Would an IP change by my ISP cause the MQTT subscription any errors or would everything work as expected?

• When receiving messages you can identify the sender by the message.message.user_id but that doesn't work if you're trying to figure out if the message was sent to a group chat. Is there any way to identify group chat name from message.message.thread_id ?

_{Huge appreciation to Nerixyz and the other contributors. Love the project guys.}

[Nerixyz]

I think I heard of this issue before, but I never really got to the root cause. Realistically, as with any "realtime" service, the server would disconnect you at some point. I'm not sure why this isn't done here. For now, it's probably best to reconnect every X hours. You could also turn on debug logging and see if the server still accepts pings after some time.

Would an IP change by my ISP cause the MQTT subscription any errors or would everything work as expected?

This should cause at least a timeout of the pings to the server (⇒ reconnect).

When receiving messages you can identify the sender by the message.message.user_id but that doesn't work if you're trying to figure out if the message was sent to a group chat. Is there any way to identify group chat name from message.message.thread_id ?

You can get information about a thread using the thread-id, so this way you can get information about the group.

[harmine21]

Ok! I will settle with reconnect every X hours.

A couple of questions:

• Is there any function or way to get the information from message.message.thread_id ? You didn't specify how on your response.

• Is logging in with sessionid cookie supported? I've been using a weird way that made it work with no problems but I'd like to use an official way if there is one :

  const cookieValue = "SESSIONID HERE";
  const stateData = `{"cookies":"{\\\"cookies\\\":[{\\\"key\\\":\\\"sessionid\\\",\\\"value\\\":\\\"${cookieValue}\\\",\\\"domain\\\":\\\"instagram.com\\\",\\\"path\\\":\\\"/\\\",\\\"secure\\\":true}]}"}`;
  await ig.state.deserialize(stateData);

[harmine21]

@Nerixyz you there? Please respond in any way I'd like to close this issue. Thanks.

[Nerixyz]

Sorry, I missed that notification.

• Is there any function or way to get the information from message.message.thread_id? You didn't specify how on your response.

You can use it in the directThread feed (See interface).

• Is logging in with sessionid cookie supported? I've been using a weird way that made it work with no problems but I'd like to use an official way if there is one :

AFAIK, the login changed, and you use some other login method. But I'm not using the library anymore.

[harmine21]

You can use it in the directThread feed (See interface).

Could you provide a code example? (like ig.whatever.whatever(thread_id)) I am not that good with js and I don't understand the documentation.

[harmine21]

nevermind. figured everything out

[diragb]

@harmine21 Can you provide a reference to what you figured out to help future users?

[harmine21]

@harmine21 Can you provide a reference to what you figured out to help future users?

Yes sure.

Regarding the thread_id.

I managed to find out that the information for the thread_id comes from the API endpoint : https://i.instagram.com/api/v1/direct_v2/threads/{thread_id}

From the thread_id you can get a lot of information about a DM conversation like if it's a group chat (its username), which are the members and more.

I wrote an axios function myself to just fetch the data from the endpoint. To do that all you have to do is visit the URL I provided above with an acceptable UserAgent like this one :

Mozilla/5.0 (iPhone; CPU iPhone OS 12_3_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 Instagram 105.0.0.11.118 (iPhone11,8; iOS 12_3_1; en_US; en-US; scale=2.00; 828x1792; 165586599)

Regarding the disconnection issue.

If a user's session expires while the mqtt is listening for messages, we don't receive a fatal error even though the connection gets terminated (this behavior is not replicated with any disconnection, some disconnections do actually produce fatal errors as expected). I tried running my script on debug mode and I can clearly see the warning/error that says that the client got disconnected but no errors are thrown. There must be something wrong in the code of one of these 3 :

• https://github.com/dilame/instagram-private-api
• https://github.com/Nerixyz/instagram_mqtt
• https://github.com/Nerixyz/mqtts

Personal workaround for managing disconnections.

In general, disconnections are expected. But we'd like to reconnect when they happen.

I have made my script close every time as expected when having a disconnection by adding just 1 line to the function connect() from this file node_modules\mqtts\dist\mqtt.client.js as follows :

async connect(options) {
    try {
        await this._connect(options);
    } catch (e) {
        this.mqttDebug(`Connection error`, e);
        this.emitError(e);
        process.exit(1); //this is the only added line.
    }
}

That catch block gets triggered when the connection is not successful therefore it should throw an error. It seems to do so by this.emitError(e); but it doesn't actually close the script as it should (only prints stuff when using debug mode).

Now I've just made a parent.js file that spawns and restarts the child.js script (the main script) whenever it exits. This approach helps maintain a reliable and uninterrupted connection.

[harmine21]

Update on my workaround.

It seems my previous workaround only fixes the problem of not producing an error when a re-connection has failed. But I've also discovered a new problem. Use both this and the previous fixes if you'd like to avoid these problems.

After letting the mqtt client run for at least some days, a disconnection will happen. After that disconnection the client will try and re-connect and it will succeed but the connection will not be truly successful.

Take a look at this debug log :

As you can see, a successful connection was made but the ig_sub_iris_response has answered with a {"succeeded":false}. When this is false, our client receives no more responses from the server so there is no point in staying connected.

To make your client disconnect whenever that response is false we can modify the file node_modules\instagram_mqtt\dist\realtime\realtime.client.js.

Open the file and find the function connect() (it should be around line 80-90).

Here is the modified connect() function (I've added comments on the changed lines so you can know what to change. Explanation of each change will be below the code block):

async connect(initOptions) {
    var _a;
    this.realtimeDebug('Connecting to realtime-broker...');
    this.setInitOptions(initOptions);
    this.realtimeDebug(`Overriding: ${Object.keys(this.initOptions.connectOverrides || {}).join(', ')}`);
    this._mqtt = new mqttot_1.MQTToTClient({
        url: constants_1.REALTIME.HOST_NAME_V6,
        payloadProvider: () => {
            this.constructConnection();
            return shared_1.compressDeflate(this.connection.toThrift());
        },
        enableTrace: this.initOptions.enableTrace,
        autoReconnect: (_a = this.initOptions.autoReconnect) !== null && _a !== void 0 ? _a : true,
        requirePayload: false,
        socksOptions: this.initOptions.socksOptions,
        additionalOptions: this.initOptions.additionalTlsOptions,
    });
    this.commands = new commands_1.Commands(this.mqtt);
    this.direct = new commands_1.DirectCommands(this.mqtt);
    this.mqtt.on('message', async (msg) => {
        var _a;
        const unzipped = await shared_1.tryUnzipAsync(msg.payload);
        const topic = constants_1.RealtimeTopicsArray.find(t => t.id === msg.topic);
        if (topic && topic.parser && !topic.noParse) {
            const parsedMessages = topic.parser.parseMessage(topic, unzipped);
            const parsedMessagesJson = JSON.stringify(Array.isArray(parsedMessages) ? parsedMessages.map((x) => x.data) : parsedMessages.data); // CHANGE NUMBER 1
            this.messageDebug(`Received on ${topic.path}: ${parsedMessagesJson}`); // CHANGE NUMBER 2
            try { if (JSON.parse(parsedMessagesJson).succeeded === false) process.exit(1); } catch (error) {} // CHANGE NUMBER 3
            this.emit('receive', topic, Array.isArray(parsedMessages) ? parsedMessages : [parsedMessages]);
        }
        else {
            this.messageDebug(`Received raw on ${(_a = topic === null || topic === void 0 ? void 0 : topic.path) !== null && _a !== void 0 ? _a : msg.topic}: (${unzipped.byteLength} bytes) ${shared_1.prepareLogString(unzipped.toString())}`);
            this.emit('receiveRaw', msg);
        }
    });
    this.mqtt.on('error', e => this.emitError(e));
    this.mqtt.on('warning', w => this.emitWarning(w));
    this.mqtt.on('disconnect', () => this.safeDisconnect
        ? this.emit('disconnect')
        : this.emitError(new errors_1.ClientDisconnectedError('MQTToTClient got disconnected.')));
    return new Promise((resolve, reject) => {
        this.mqtt.on('connect', async () => {
            this.realtimeDebug('Connected. Checking initial subs.');
            const { graphQlSubs, skywalkerSubs, irisData } = this.initOptions;
            await Promise.all([
                graphQlSubs && graphQlSubs.length > 0 ? this.graphQlSubscribe(graphQlSubs) : null,
                skywalkerSubs && skywalkerSubs.length > 0 ? this.skywalkerSubscribe(skywalkerSubs) : null,
                irisData ? this.irisSubscribe(irisData) : null,
            ]).then(resolve);
        });
        this.mqtt
            .connect({
            keepAlive: 20,
            protocolLevel: 3,
            clean: true,
            connectDelay: 60 * 1000,
        })
            .catch(e => {
            this.emitError(e);
            reject(e);
        });
    });
}

Changes :

No. 1: I've added the variable parsedMessagesJson that contains the response we get from the server to make the code more readable. No. 2: This line already existed before but now we're just making use of the variable parsedMessagesJson. No. 3: This is the only line that actually changes something in the functionality of the code. It's a one-liner try-catch block that will just close our program if the response is {"succeeded":false}.

_{Note: These are just simple temporary fixes to a problem and are not actual fixes. To fix the actual code, the repo owner should implement a fix on the source code correctly.}