search

zopefoundation / ZODB

Python object-oriented database
https://zodb-docs.readthedocs.io/
Other
681 stars 92 forks source link

Blob usage depending on file size #361

Open georgpfolz opened 2 years ago

georgpfolz commented 2 years ago

Is there a best practice or some first hand experience for which file sizes it makes sense to use Blob storage?

I'm using the blob storage to store files and images, but I wonder if it's maybe better (performance-wise?) to keep small files in the ZODB.

d-maurer commented 2 years ago

Georg Pfolz wrote at 2022-4-24 03:14 -0700:

Is there a best practice or some first hand experience for which file sizes it makes sense to use Blob storage?

I'm using the blob storage to store files and images, but I wonder if it's maybe better (performance-wise?) to keep small files in the ZODB.

You could make some measurements.

jimfulton commented 2 years ago

I think of blobs as persistent files and storage of files, including images is the intended use case.

The work I did on blobs was motivated by the need to store media for which sizes were fairly large.

If you have lots of small files, then there might be a win of storing them directly in database records. The win would be highly use-case specific. As Dieter suggested, you could try to measure performance characteristics for your use case.

georgpfolz commented 2 years ago

Ok, I understand, it's too use-case specific so a general approach doesn't make sense. But your answer has reinforced my view that considering file size makes sense.

Thanks for the answers and I'll do some measurements! :)

georgpfolz commented 2 years ago

The first measurements I made suggest that the performance difference is negligible. So I'll probably store all image and file data in blobs, regardless of size.