How to handle larger file download or upload?

[Laevon]

Hi, I am also working on ftp over quic using aioquic recently. It's lucky to see your nice work. I have a question, how to handle larger file download or upload? In your demo, the files are all around 1 KB. If I want to download or upload larger files, e.g. a 100 MB file. How to handle this?

I wrote a demo, server reads the file content and send it over one stream using quic.send_stream_data: self.protocol._quic.send_stream_data(stream_id, data, end_stream=True) and what I found is I don't need to slice the data, aioquic will finish this.

Cilent will received data in `quic_event_received'.

I tried to download one ~80 MB .exe file, it took ~70 sec, which was too long compared with TCP, ~2 sec only. I thought I/O operations took too much time, so I didn't execute any I/O operations, like this:

    def quic_event_received(self, event):
        if isinstance(event, HandshakeCompleted):
            asyncio.create_task(aioconsole.aprint("Handshake completed!"))

        elif isinstance(event, StreamDataReceived):
            stream_id = event.stream_id
            data = event.data  
            if self.control_stream_id is None:
                self.control_stream_id = stream_id
            if stream_id == self.control_stream_id: 
                self.control_queue.put_nowait(data.decode())
            else:
                self.times += 1
                if event.end_stream:
                    end = time.time()
                    print('time:', end - self.start, 'times:', self.times)

Amazingly, it still took ~70sec. and self.times was 71553. I don't know why aioquic works that bad. It seems the reason was 71553 StreamDataReceived events led to such a long time. Could you please help me with this, thanks!

[Subhrato20]

That's an interesting thoughts; I hadn't thought of that. Once I get some time, I'll definitely look into it and expeiment with that. Thanks for sharing these insights!

p.s. If you find any better alternative just let me know.

[Laevon]

That's an interesting thoughts; I hadn't thought of that. Once I get some time, I'll definitely look into it and expeiment with that. Thanks for sharing these insights!

p.s. If you find any better alternative just let me know.

I appreciate you taking the time to consider my issue. In the meantime, I'm also actively looking for some solutions. If I find anything promising, I'll be sure to share it with you.