search

giampaolo / pyftpdlib

Extremely fast and scalable Python FTP server library
MIT License
1.65k stars 267 forks source link

Cache username/groupname results on LIST #520

Closed giampaolo closed 4 years ago

giampaolo commented 4 years ago

When LISTing a directory we call pwd.getpwuid and grp.getgrgid repeatedly. This patch avoids that by using a memoize decorator caching results. This simple benchmark reports a +3.7x speedup when listing a directory via LIST:

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.filesystems import AbstractedFS
import os
import time

here = os.getcwd()
fs = AbstractedFS(here, FTPHandler)
t = time.time()
for x in range(1000):
    list(fs.format_list(here, os.listdir(here)))
print(time.time() - t)

Before: 1.3308672904968262 After: 0.35703492164611816

Directory had 38 files, meaning in my system I'm able to list 38 * 1000 = 38000 files in 0.35 secs instead of 1.33.

coveralls commented 4 years ago

Coverage Status

Coverage remained the same at ?% when pulling 2e0b42a1c24db1ed23d180ce5518bd4186cc451c on cache-user-group-names into 298e48dd4d86f316925fa0997b4f750733f02a7f on master.