search

andyrocket8 / blacklist

Application for IP address blacklisting
0 stars 1 forks source link

Implement generator result generation #21

Closed andyrocket8 closed 8 months ago

andyrocket8 commented 9 months ago

On list fetching we use total data fetching with memory utilization. On blacklist and whitelist execution it's possible to use AsyncGenerator functions for address data. Need some refactoring for optimal memory utilization

andyrocket8 commented 8 months ago

During deployment of issue #35 it is noticed that StreamingResponse with Awaitable has two side effects 1) Before finishing fetch all injected dependencies are disposed on handler function exit. So if stream use database connection it is closed with sad consequences. 2) When pieces of fetched data are small it has bad streaming performance. So I have decided to make all streaming based on in memory lists with chunks of data sized in CHUNK_SIZE_BYTES