search

inveniosoftware / invenio-files-rest

REST API for uploading/downloading files for Invenio.
https://invenio-files-rest.readthedocs.io
MIT License
9 stars 68 forks source link

storage: customizable checksum algorithms for fixity #187

Open mirekys opened 5 years ago

mirekys commented 5 years ago

Adds the the possibility to customize the checksums that are computed on files by using the following config options:

FILES_REST_SUPPORTED_CHECKSUM_ALGORITHMS = {
    'md5': hashlib.md5
}
"""Algorithms that can be used for file checksum compute and verify"""

FILES_REST_CHECKSUM_ALGORITHM = 'md5'
"""Checksum algorithm to be used on all newly uploaded files

.. note::
   Value of this variable must be a corresponding
   key in the ``FILES_REST_SUPPORTED_CHECKSUM_ALGORITHMS`` variable.
"""

When the current checksum algorithm changes, newly created files will have the new checksum. Fixity checking of old files having a checksum generated by a previous algorithm is still maintained (as long as the algorithm remains in the FILES_REST_SUPPORTED_CHECKSUM_ALGORITHMS option).

lnielsen commented 5 years ago

Thanks for the PR @mirekys.

I'm having a look at it now and will update it. There's a couple of things I'd like to fix like ensuring the FileStorage class is independent of the application state (i.e. current_files_rest) and having a look at what the base interface should look like because there's already Invenio-XRootD and Invenio-S3 which implement the interface.