jaraco / irc

Full-featured Python IRC library for Python.
MIT License
396 stars 87 forks source link

DCC Transfers getting stuck #201

Open Splitframe opened 2 years ago

Splitframe commented 2 years ago

Sometimes, mostly around the 500MB mark, the DCC transfers seem to get stuck at

connection.send_bytes(struct.pack("!I", self.received_bytes))

The previous package is received and then execution just stops at the above function. It's a little hard to debug since a few thousand packages go through so manually stepping through is not feasable.

Is this related to the warning that TCP Buffers can block?

Here are my relevant functions. Any Idea would be appreciated.

def on_ctcp(self, connection, event):
        print(event)
        if (event.arguments[0] == "VERSION"):
            connection.ctcp_reply(event.source, "VERSION Niblette 0.1a")
            return

        payload = event.arguments[1]
        parts = shlex.split(payload)
        command, filename, peer_address, peer_port, size = parts

        if (command != "SEND"):
            return

        showName, season = determineStructure(filename)

        if (isWindows):
            fullpath   = filename
        else:
            showPath   = f"/usr/download/Anime/{showName}"
            seasonPath = f"/usr/download/Anime/{showName}/{season}"
            fullpath   = f"/usr/download/Anime/{showName}/{season}/{filename}"
            if (not os.path.exists(showPath)):
                os.mkdir(showPath)
            if (not os.path.exists(seasonPath)):
                os.mkdir(seasonPath)

        print("Receiving a file. Potential location: ", fullpath)
        if (os.path.exists(fullpath)):
            print("A file named", fullpath, "already exists. Refusing to save it.")
            return
        print("Downloading ", filename)
        self.total_bytes = int(size)
        print(f"Size: {self.total_bytes}")
        self.received_bytes = 0
        if (self.file is not None):
            self.file.close()
        self.file = open(fullpath, "wb")
        peer_address = irc.client.ip_numstr_to_quad(peer_address)
        peer_port = int(peer_port)
        self.downloader = self.dcc("raw")
        self.downloader = self.downloader.connect(peer_address, peer_port)

    def on_dccmsg(self, connection, event):
        try:
            if (len(event.arguments) != 1):
                print(f"Error while downloading, error: {event.arguments}")
                self.file.close()
                self.downloader.disconnect()
            data = event.arguments[0]
            self.file.write(data)
            self.received_bytes = self.received_bytes + len(data)

            connection.send_bytes(struct.pack("!I", self.received_bytes))

            if (self.received_bytes == self.total_bytes):
                print("")
                print("Finished, disconnecting.")
                self.file.close()
                self.downloader.disconnect()
        except Exception as ex:
            print(f"Error: {ex}")
Splitframe commented 2 years ago

I just cannot find a solution. I tried buffering the file in RAM and writing it after it finishes to reduce socket answer times and I also tried adding a few nanoseconds of delay to don't flood the server, nothing worked. The error is "Connection Reset by Peer" and only occurs on Linux (Docker), but not on Windows and always around the 500MB mark. I am at my wits end.