search

nhorvath / Pyrebase4

A simple python wrapper for the Firebase API. ⛺
258 stars 64 forks source link

stream() is very slow with large messages #45

Open bringert opened 2 years ago

bringert commented 2 years ago

In current HEAD, with Python 3.6.5, I found that stream() is extremely slow. It takes many seconds on a Raspberry Pi 3 to parse a few tens of kilobytes, and it gets progressively slower.

It looks like pyre_sseclient._event_complete() searches the message from the start for every character read, which makes detecting the end of a message quadratic in the length of the message.

bringert commented 2 years ago

https://github.com/btubbs/sseclient/blob/master/sseclient.py doesn't seem to have this quadratic behavior, so one solution would be to change to that.

bringert commented 2 years ago

Changing it so that the search for end_of_field just searches the end of the buffer speeds it up a lot, but it's still a lot slower than https://github.com/btubbs/sseclient/blob/master/sseclient.py.

It looks like it's because pyre_sseclient reads one character at a time, whereas sseclient.py uses a default chunk size of 1024.