Multiple SAWarning messages are triggered after uploading an empty file when
RECORDS_RESOURCES_ALLOW_EMPTY_FILES = False.
These warnings shows after waiting for scheduled jobs such as file-checks and file-integrity-report to complete.
Steps to Reproduce
Set RECORDS_RESOURCES_ALLOW_EMPTY_FILES = False in the configuration.
Upload an empty file.
Wait for the scheduled jobs (file-checks and file-integrity-report) to complete.
Observe the console for SAWarning messages related to ORM conflicts.
Expected behavior
The system should reject the empty file upload cleanly without triggering SAWarning messages. The scheduled jobs should be completed without errors or warnings related to ORM conflicts.
Logs
After uploading the empty file
[2024-08-24 17:41:39,415: ERROR/ForkPoolWorker-10] root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not exist
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum
real_checksum = self.storage(**kwargs).checksum(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum
fp = self.open(mode="rb")
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open
fs, path = self._get_fs(create_dir=create_dir)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs
opendir(filedir, writeable=True, create=create_dir),
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs
_fs, _path = self.open(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open
open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs
osfs = OSFS(path, create=create)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in __init__
raise errors.CreateFailed(message)
fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not exist
After the jobs have run.
[2024-08-27 11:35:23,594: WARNING/ForkPoolWorker-10] /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/tasks.py:46: LegacyAPIWarning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings. Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9) f = FileInstance.query.get(uuid.UUID(file_id))
[2024-08-27 11:35:23,595: INFO/ForkPoolWorker-9] Task invenio_files_rest.tasks.schedule_checksum_verification[865a1a96-0f5c-4abc-b23c-feeaad86c33b] succeeded in 0.10319974800040654s: None
[2024-08-27 11:35:23,596: WARNING/ForkPoolWorker-4] [2024-08-27 11:35:23,594] ERROR in models: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum
real_checksum = self.storage(**kwargs).checksum(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum
fp = self.open(mode="rb")
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open
fs, path = self._get_fs(create_dir=create_dir)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs
opendir(filedir, writeable=True, create=create_dir),
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs
_fs, _path = self.open(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open
open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs
osfs = OSFS(path, create=create)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init
raise errors.CreateFailed(message)
fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist
[2024-08-27 11:35:23,594: ERROR/ForkPoolWorker-4] root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum
real_checksum = self.storage(**kwargs).checksum(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum
fp = self.open(mode="rb")
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open
fs, path = self._get_fs(create_dir=create_dir)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs
opendir(filedir, writeable=True, create=create_dir),
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs
_fs, _path = self.open(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open
open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs
osfs = OSFS(path, create=create)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init
raise errors.CreateFailed(message)
fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist
[2024-08-27 11:35:23,609: INFO/ForkPoolWorker-7] POST http://localhost:9200/latest-build-stats-bookmarks/_doc [status:201 request:0.051s]
[2024-08-27 11:35:23,610: INFO/ForkPoolWorker-7] Task invenio_rdm_records.services.tasks.reindex_stats[23e5ca67-7a0b-458e-a179-59d42660e2be] succeeded in 0.11804539900003874s: '0 documents reindexed'
[2024-08-27 11:35:23,613: WARNING/ForkPoolWorker-4] [2024-08-27 11:35:23,613] ERROR in models: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not exist
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum
real_checksum = self.storage(**kwargs).checksum(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum
fp = self.open(mode="rb")
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open
fs, path = self._get_fs(create_dir=create_dir)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs
opendir(filedir, writeable=True, create=create_dir),
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs
_fs, _path = self.open(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open
open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs
osfs = OSFS(path, create=create)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init
raise errors.CreateFailed(message)
fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not exist
[2024-08-27 11:35:23,613: ERROR/ForkPoolWorker-4] root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not exist
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum
real_checksum = self.storage(**kwargs).checksum(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum
fp = self.open(mode="rb")
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open
fs, path = self._get_fs(create_dir=create_dir)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs
opendir(filedir, writeable=True, create=create_dir),
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs
_fs, _path = self.open(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open
open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs
osfs = OSFS(path, create=create)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init
raise errors.CreateFailed(message)
fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not exist
[2024-08-27 11:35:23,622: WARNING/ForkPoolWorker-10] [2024-08-27 11:35:23,621] ERROR in models: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum
real_checksum = self.storage(**kwargs).checksum(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum
fp = self.open(mode="rb")
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open
fs, path = self._get_fs(create_dir=create_dir)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs
opendir(filedir, writeable=True, create=create_dir),
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs
_fs, _path = self.open(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open
open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs
osfs = OSFS(path, create=create)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init
raise errors.CreateFailed(message)
fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist
[2024-08-27 11:35:23,621: ERROR/ForkPoolWorker-10] root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum
real_checksum = self.storage(**kwargs).checksum(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum
fp = self.open(mode="rb")
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open
fs, path = self._get_fs(create_dir=create_dir)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs
opendir(filedir, writeable=True, create=create_dir),
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs
_fs, _path = self.open(
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open
open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs
osfs = OSFS(path, create=create)
File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init
raise errors.CreateFailed(message)
fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist
[2024-08-27 11:35:23,627: INFO/ForkPoolWorker-10] Task invenio_files_rest.tasks.verify_checksum[148b9864-e3de-4fac-8a46-2839ea93ee57] succeeded in 0.034604151000166894s: None
[2024-08-27 11:35:23,635: INFO/ForkPoolWorker-2] POST http://localhost:9200/latest-build-requests/_search?scroll=5m&size=1000&version=true [status:200 request:0.038s]
[2024-08-27 11:35:23,640: INFO/ForkPoolWorker-5] POST http://localhost:9200/latest-build-rdmrecords-records/_search?scroll=5m&size=1000&version=true [status:200 request:0.039s]
[2024-08-27 11:35:23,645: INFO/ForkPoolWorker-5] DELETE http://localhost:9200/_search/scroll [status:200 request:0.004s]
[2024-08-27 11:35:23,645: INFO/ForkPoolWorker-2] DELETE http://localhost:9200/_search/scroll [status:200 request:0.009s]
[2024-08-27 11:35:23,646: INFO/ForkPoolWorker-5] Task invenio_rdm_records.services.tasks.update_expired_embargos[cca8b5ca-1d6d-417f-9c79-1cd1bfde6f08] succeeded in 0.15260031100115157s: None
[2024-08-27 11:35:23,649: INFO/ForkPoolWorker-2] Task invenio_requests.tasks.check_expired_requests[bd7be258-5035-40a4-b3f6-086f621500e9] succeeded in 0.15791969299971242s: None
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: Files integrity report [2024-08-27 09:35:23]
From: info@inveniosoftware.org
To: info@inveniosoftware.org
Date: Tue, 27 Aug 2024 11:35:23 +0200
Message-ID: 172475132365.50251.4887367087467600466@nvidia-2070.lan
The following files were flagged as 'unhealthy'. This means that the checksum check failed or timed out. Please take any action if needed.
ID: 77661a52-fff3-4a48-8e8a-1d3a6123d8c7
URI: /home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7/data
Name: emptyFile.empty
Created: 2024-08-23 11:07:01.948394
Checksum: md5:d41d8cd98f00b204e9800998ecf8427e
Last check date: 2024-08-27 09:35:23.613965
Last check FAILED with result: None
Draft: https://127.0.0.1/uploads/tjvdc-z8x81
ID: 1ebba22a-5ec5-4351-995b-9ff126ef42c1
URI: /home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1/data
Name: emptyFile.empty
Created: 2024-08-23 11:06:20.140607
Checksum: md5:d41d8cd98f00b204e9800998ecf8427e
Last check date: 2024-08-27 09:35:23.599688
Last check FAILED with result: None
Draft: https://127.0.0.1/uploads/tjvdc-z8x81
[2024-08-27 11:35:23,676: INFO/ForkPoolWorker-4] Task invenio_app_rdm.tasks.file_integrity_report[a42a6ee7-b2fd-43ee-98b6-2789b7019596] succeeded in 0.18464864300040063s: None
'FLASK_ENV' is deprecated and will not be used in Flask 2.3. Use 'FLASK_DEBUG' instead.
'FLASK_ENV' is deprecated and will not be used in Flask 2.3. Use 'FLASK_DEBUG' instead.
[2024-08-27 11:35:24,394] DEBUG in entrypoint: Loading config for entry point invenio_app_rdm = invenio_app_rdm.config
[2024-08-27 11:35:26,296] DEBUG in ext: Flask-DebugToolbar extension not installed.
'FLASK_ENV' is deprecated and will not be used in Flask 2.3. Use 'FLASK_DEBUG' instead.
[2024-08-27 11:35:26,814] DEBUG in entrypoint: Loading config for entry point invenio_app_rdm = invenio_app_rdm.config
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.RecordMetadataVersion, and will be replaced in the string-lookup table.
super(BindMetaMixin, cls).init(name, bases, d)
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.RDMRecordMetadataVersion, and will be replaced in the string-lookup table.
super(BindMetaMixin, cls).init(name, bases, d)
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.RDMFileRecordMetadataVersion, and will be replaced in the string-lookup table.
super(BindMetaMixin, cls).init(name, bases, d)
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.RDMMediaFileRecordMetadataVersion, and will be replaced in the string-lookup table.
super(BindMetaMixin, cls).init(name, bases, d)
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.PageModelVersion, and will be replaced in the string-lookup table.
super(BindMetaMixin, cls).init(name, bases, d)
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.PageListVersion, and will be replaced in the string-lookup table.
super(BindMetaMixin, cls).init(name, bases, d)
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class RecordMetadataVersion->records_metadata_version, primary key column 'records_metadata_version.transaction_id' is being combined with distinct primary key column 'records_metadata_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name.
for prop in sa.inspect(cls).iterate_properties:
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'RecordMetadataVersion.transaction' will copy column transaction.id to column records_metadata_version.transaction_id, which conflicts with relationship(s): 'RecordMetadataVersion.transaction' (copies transaction.id to records_metadata_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'RecordMetadataVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
for prop in sa.inspect(cls).iterate_properties:
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class RDMRecordMetadataVersion->rdm_records_metadata_version, primary key column 'rdm_records_metadata_version.transaction_id' is being combined with distinct primary key column 'rdm_records_metadata_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name.
for prop in sa.inspect(cls).iterate_properties:
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'RDMRecordMetadataVersion.transaction' will copy column transaction.id to column rdm_records_metadata_version.transaction_id, which conflicts with relationship(s): 'RDMRecordMetadataVersion.transaction' (copies transaction.id to rdm_records_metadata_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'RDMRecordMetadataVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
for prop in sa.inspect(cls).iterate_properties:
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class RDMFileRecordMetadataVersion->rdm_records_files_version, primary key column 'rdm_records_files_version.transaction_id' is being combined with distinct primary key column 'rdm_records_files_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name.
for prop in sa.inspect(cls).iterate_properties:
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'RDMFileRecordMetadataVersion.transaction' will copy column transaction.id to column rdm_records_files_version.transaction_id, which conflicts with relationship(s): 'RDMFileRecordMetadataVersion.transaction' (copies transaction.id to rdm_records_files_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'RDMFileRecordMetadataVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
for prop in sa.inspect(cls).iterate_properties:
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class RDMMediaFileRecordMetadataVersion->rdm_records_media_files_version, primary key column 'rdm_records_media_files_version.transaction_id' is being combined with distinct primary key column 'rdm_records_media_files_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name.
for prop in sa.inspect(cls).iterate_properties:
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'RDMMediaFileRecordMetadataVersion.transaction' will copy column transaction.id to column rdm_records_media_files_version.transaction_id, which conflicts with relationship(s): 'RDMMediaFileRecordMetadataVersion.transaction' (copies transaction.id to rdm_records_media_files_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'RDMMediaFileRecordMetadataVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
for prop in sa.inspect(cls).iterate_properties:
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class PageModelVersion->pages_page_version, primary key column 'pages_page_version.transaction_id' is being combined with distinct primary key column 'pages_page_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name.
for prop in sa.inspect(cls).iterate_properties:
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'PageModelVersion.transaction' will copy column transaction.id to column pages_page_version.transaction_id, which conflicts with relationship(s): 'PageModelVersion.transaction' (copies transaction.id to pages_page_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'PageModelVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
for prop in sa.inspect(cls).iterate_properties:
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class PageListVersion->pages_pagelist_version, primary key column 'pages_pagelist_version.transaction_id' is being combined with distinct primary key column 'pages_pagelist_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name.
for prop in sa.inspect(cls).iterate_properties:
/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'PageListVersion.transaction' will copy column transaction.id to column pages_pagelist_version.transaction_id, which conflicts with relationship(s): 'PageListVersion.transaction' (copies transaction.id to pages_pagelist_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'PageListVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx)
for prop in sa.inspect(cls).iterate_properties:
Package version (if known): v12 - 13
Describe the bug
Multiple SAWarning messages are triggered after uploading an empty file when
RECORDS_RESOURCES_ALLOW_EMPTY_FILES = False
. These warnings shows after waiting for scheduled jobs such as file-checks and file-integrity-report to complete.Steps to Reproduce
RECORDS_RESOURCES_ALLOW_EMPTY_FILES = False
in the configuration.Expected behavior
The system should reject the empty file upload cleanly without triggering SAWarning messages. The scheduled jobs should be completed without errors or warnings related to ORM conflicts.
Logs
After uploading the empty file
[2024-08-24 17:41:39,415: ERROR/ForkPoolWorker-10] root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not exist Traceback (most recent call last): File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum real_checksum = self.storage(**kwargs).checksum( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum fp = self.open(mode="rb") File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open fs, path = self._get_fs(create_dir=create_dir) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs opendir(filedir, writeable=True, create=create_dir), File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs _fs, _path = self.open( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs osfs = OSFS(path, create=create) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in __init__ raise errors.CreateFailed(message) fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not existAfter the jobs have run.
[2024-08-27 11:35:23,594: WARNING/ForkPoolWorker-10] /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/tasks.py:46: LegacyAPIWarning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings. Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9) f = FileInstance.query.get(uuid.UUID(file_id)) [2024-08-27 11:35:23,595: INFO/ForkPoolWorker-9] Task invenio_files_rest.tasks.schedule_checksum_verification[865a1a96-0f5c-4abc-b23c-feeaad86c33b] succeeded in 0.10319974800040654s: None [2024-08-27 11:35:23,596: WARNING/ForkPoolWorker-4] [2024-08-27 11:35:23,594] ERROR in models: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist Traceback (most recent call last): File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum real_checksum = self.storage(**kwargs).checksum( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum fp = self.open(mode="rb") File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open fs, path = self._get_fs(create_dir=create_dir) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs opendir(filedir, writeable=True, create=create_dir), File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs _fs, _path = self.open( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs osfs = OSFS(path, create=create) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init raise errors.CreateFailed(message) fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist [2024-08-27 11:35:23,594: ERROR/ForkPoolWorker-4] root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist Traceback (most recent call last): File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum real_checksum = self.storage(**kwargs).checksum( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum fp = self.open(mode="rb") File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open fs, path = self._get_fs(create_dir=create_dir) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs opendir(filedir, writeable=True, create=create_dir), File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs _fs, _path = self.open( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs osfs = OSFS(path, create=create) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init raise errors.CreateFailed(message) fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist [2024-08-27 11:35:23,609: INFO/ForkPoolWorker-7] POST http://localhost:9200/latest-build-stats-bookmarks/_doc [status:201 request:0.051s] [2024-08-27 11:35:23,610: INFO/ForkPoolWorker-7] Task invenio_rdm_records.services.tasks.reindex_stats[23e5ca67-7a0b-458e-a179-59d42660e2be] succeeded in 0.11804539900003874s: '0 documents reindexed' [2024-08-27 11:35:23,613: WARNING/ForkPoolWorker-4] [2024-08-27 11:35:23,613] ERROR in models: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not exist Traceback (most recent call last): File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum real_checksum = self.storage(**kwargs).checksum( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum fp = self.open(mode="rb") File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open fs, path = self._get_fs(create_dir=create_dir) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs opendir(filedir, writeable=True, create=create_dir), File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs _fs, _path = self.open( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs osfs = OSFS(path, create=create) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init raise errors.CreateFailed(message) fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not exist [2024-08-27 11:35:23,613: ERROR/ForkPoolWorker-4] root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not exist Traceback (most recent call last): File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum real_checksum = self.storage(**kwargs).checksum( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum fp = self.open(mode="rb") File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open fs, path = self._get_fs(create_dir=create_dir) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs opendir(filedir, writeable=True, create=create_dir), File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs _fs, _path = self.open( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs osfs = OSFS(path, create=create) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init raise errors.CreateFailed(message) fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7' does not exist [2024-08-27 11:35:23,622: WARNING/ForkPoolWorker-10] [2024-08-27 11:35:23,621] ERROR in models: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist Traceback (most recent call last): File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum real_checksum = self.storage(**kwargs).checksum( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum fp = self.open(mode="rb") File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open fs, path = self._get_fs(create_dir=create_dir) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs opendir(filedir, writeable=True, create=create_dir), File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs _fs, _path = self.open( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs osfs = OSFS(path, create=create) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init raise errors.CreateFailed(message) fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist [2024-08-27 11:35:23,621: ERROR/ForkPoolWorker-10] root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist Traceback (most recent call last): File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/models.py", line 811, in verify_checksum real_checksum = self.storage(**kwargs).checksum( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/base.py", line 146, in checksum fp = self.open(mode="rb") File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 59, in open fs, path = self._get_fs(create_dir=create_dir) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/invenio_files_rest/storage/pyfs.py", line 46, in _get_fs opendir(filedir, writeable=True, create=create_dir), File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 220, in open_fs _fs, _path = self.open( File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/registry.py", line 177, in open open_fs = opener.open_fs(fs_url, parse_result, writeable, create, cwd) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/opener/osfs.py", line 40, in open_fs osfs = OSFS(path, create=create) File "/home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/fs/osfs.py", line 141, in init raise errors.CreateFailed(message) fs.errors.CreateFailed: root path '/home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1' does not exist [2024-08-27 11:35:23,627: INFO/ForkPoolWorker-10] Task invenio_files_rest.tasks.verify_checksum[148b9864-e3de-4fac-8a46-2839ea93ee57] succeeded in 0.034604151000166894s: None [2024-08-27 11:35:23,635: INFO/ForkPoolWorker-2] POST http://localhost:9200/latest-build-requests/_search?scroll=5m&size=1000&version=true [status:200 request:0.038s] [2024-08-27 11:35:23,640: INFO/ForkPoolWorker-5] POST http://localhost:9200/latest-build-rdmrecords-records/_search?scroll=5m&size=1000&version=true [status:200 request:0.039s] [2024-08-27 11:35:23,645: INFO/ForkPoolWorker-5] DELETE http://localhost:9200/_search/scroll [status:200 request:0.004s] [2024-08-27 11:35:23,645: INFO/ForkPoolWorker-2] DELETE http://localhost:9200/_search/scroll [status:200 request:0.009s] [2024-08-27 11:35:23,646: INFO/ForkPoolWorker-5] Task invenio_rdm_records.services.tasks.update_expired_embargos[cca8b5ca-1d6d-417f-9c79-1cd1bfde6f08] succeeded in 0.15260031100115157s: None [2024-08-27 11:35:23,649: INFO/ForkPoolWorker-2] Task invenio_requests.tasks.check_expired_requests[bd7be258-5035-40a4-b3f6-086f621500e9] succeeded in 0.15791969299971242s: None Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: Files integrity report [2024-08-27 09:35:23] From: info@inveniosoftware.org To: info@inveniosoftware.org Date: Tue, 27 Aug 2024 11:35:23 +0200 Message-ID: 172475132365.50251.4887367087467600466@nvidia-2070.lan The following files were flagged as 'unhealthy'. This means that the checksum check failed or timed out. Please take any action if needed. ID: 77661a52-fff3-4a48-8e8a-1d3a6123d8c7 URI: /home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/77/66/1a52-fff3-4a48-8e8a-1d3a6123d8c7/data Name: emptyFile.empty Created: 2024-08-23 11:07:01.948394 Checksum: md5:d41d8cd98f00b204e9800998ecf8427e Last check date: 2024-08-27 09:35:23.613965 Last check FAILED with result: None Draft: https://127.0.0.1/uploads/tjvdc-z8x81 ID: 1ebba22a-5ec5-4351-995b-9ff126ef42c1 URI: /home/user/.pyenv/versions/3.9.19/envs/latest-build/var/instance/data/1e/bb/a22a-5ec5-4351-995b-9ff126ef42c1/data Name: emptyFile.empty Created: 2024-08-23 11:06:20.140607 Checksum: md5:d41d8cd98f00b204e9800998ecf8427e Last check date: 2024-08-27 09:35:23.599688 Last check FAILED with result: None Draft: https://127.0.0.1/uploads/tjvdc-z8x81 [2024-08-27 11:35:23,676: INFO/ForkPoolWorker-4] Task invenio_app_rdm.tasks.file_integrity_report[a42a6ee7-b2fd-43ee-98b6-2789b7019596] succeeded in 0.18464864300040063s: None 'FLASK_ENV' is deprecated and will not be used in Flask 2.3. Use 'FLASK_DEBUG' instead. 'FLASK_ENV' is deprecated and will not be used in Flask 2.3. Use 'FLASK_DEBUG' instead. [2024-08-27 11:35:24,394] DEBUG in entrypoint: Loading config for entry point invenio_app_rdm = invenio_app_rdm.config [2024-08-27 11:35:26,296] DEBUG in ext: Flask-DebugToolbar extension not installed. 'FLASK_ENV' is deprecated and will not be used in Flask 2.3. Use 'FLASK_DEBUG' instead. [2024-08-27 11:35:26,814] DEBUG in entrypoint: Loading config for entry point invenio_app_rdm = invenio_app_rdm.config /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.RecordMetadataVersion, and will be replaced in the string-lookup table. super(BindMetaMixin, cls).init(name, bases, d) /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.RDMRecordMetadataVersion, and will be replaced in the string-lookup table. super(BindMetaMixin, cls).init(name, bases, d) /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.RDMFileRecordMetadataVersion, and will be replaced in the string-lookup table. super(BindMetaMixin, cls).init(name, bases, d) /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.RDMMediaFileRecordMetadataVersion, and will be replaced in the string-lookup table. super(BindMetaMixin, cls).init(name, bases, d) /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.PageModelVersion, and will be replaced in the string-lookup table. super(BindMetaMixin, cls).init(name, bases, d) /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/flask_sqlalchemy/model.py:121: SAWarning: This declarative base already contains a class with the same class name and module name as sqlalchemy_continuum.model_builder.PageListVersion, and will be replaced in the string-lookup table. super(BindMetaMixin, cls).init(name, bases, d) /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class RecordMetadataVersion->records_metadata_version, primary key column 'records_metadata_version.transaction_id' is being combined with distinct primary key column 'records_metadata_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name. for prop in sa.inspect(cls).iterate_properties: /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'RecordMetadataVersion.transaction' will copy column transaction.id to column records_metadata_version.transaction_id, which conflicts with relationship(s): 'RecordMetadataVersion.transaction' (copies transaction.id to records_metadata_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'RecordMetadataVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx) for prop in sa.inspect(cls).iterate_properties: /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class RDMRecordMetadataVersion->rdm_records_metadata_version, primary key column 'rdm_records_metadata_version.transaction_id' is being combined with distinct primary key column 'rdm_records_metadata_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name. for prop in sa.inspect(cls).iterate_properties: /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'RDMRecordMetadataVersion.transaction' will copy column transaction.id to column rdm_records_metadata_version.transaction_id, which conflicts with relationship(s): 'RDMRecordMetadataVersion.transaction' (copies transaction.id to rdm_records_metadata_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'RDMRecordMetadataVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx) for prop in sa.inspect(cls).iterate_properties: /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class RDMFileRecordMetadataVersion->rdm_records_files_version, primary key column 'rdm_records_files_version.transaction_id' is being combined with distinct primary key column 'rdm_records_files_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name. for prop in sa.inspect(cls).iterate_properties: /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'RDMFileRecordMetadataVersion.transaction' will copy column transaction.id to column rdm_records_files_version.transaction_id, which conflicts with relationship(s): 'RDMFileRecordMetadataVersion.transaction' (copies transaction.id to rdm_records_files_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'RDMFileRecordMetadataVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx) for prop in sa.inspect(cls).iterate_properties: /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class RDMMediaFileRecordMetadataVersion->rdm_records_media_files_version, primary key column 'rdm_records_media_files_version.transaction_id' is being combined with distinct primary key column 'rdm_records_media_files_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name. for prop in sa.inspect(cls).iterate_properties: /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'RDMMediaFileRecordMetadataVersion.transaction' will copy column transaction.id to column rdm_records_media_files_version.transaction_id, which conflicts with relationship(s): 'RDMMediaFileRecordMetadataVersion.transaction' (copies transaction.id to rdm_records_media_files_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'RDMMediaFileRecordMetadataVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx) for prop in sa.inspect(cls).iterate_properties: /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class PageModelVersion->pages_page_version, primary key column 'pages_page_version.transaction_id' is being combined with distinct primary key column 'pages_page_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name. for prop in sa.inspect(cls).iterate_properties: /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'PageModelVersion.transaction' will copy column transaction.id to column pages_page_version.transaction_id, which conflicts with relationship(s): 'PageModelVersion.transaction' (copies transaction.id to pages_page_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'PageModelVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx) for prop in sa.inspect(cls).iterate_properties: /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: On mapper mapped class PageListVersion->pages_pagelist_version, primary key column 'pages_pagelist_version.transaction_id' is being combined with distinct primary key column 'pages_pagelist_version.transaction_id' in attribute 'transaction_id'. Use explicit properties to give each column its own mapped attribute name. for prop in sa.inspect(cls).iterate_properties: /home/user/.pyenv/versions/3.9.19/envs/latest-build/lib/python3.9/site-packages/sqlalchemy_continuum/builder.py:123: SAWarning: relationship 'PageListVersion.transaction' will copy column transaction.id to column pages_pagelist_version.transaction_id, which conflicts with relationship(s): 'PageListVersion.transaction' (copies transaction.id to pages_pagelist_version.transaction_id). If this is not the intention, consider if these relationships should be linked with back_populates, or if viewonly=True should be applied to one or more if they are read-only. For the less common case that foreign key constraints are partially overlapping, the orm.foreign() annotation can be used to isolate the columns that should be written towards. To silence this warning, add the parameter 'overlaps="transaction"' to the 'PageListVersion.transaction' relationship. (Background on this error at: https://sqlalche.me/e/14/qzyx) for prop in sa.inspect(cls).iterate_properties:Additional context
Shouldn't we delete the file after checking if it's empty so we can dismiss all this party? see: https://github.com/inveniosoftware/invenio-records-resources/blob/58f2052983773199c3dcaba9503d8c40b5faee18/invenio_records_resources/services/files/transfer.py#L90-L109