NSFS | S3 | Versioning: DeleteObjects with VersionId does not delete all object versions

[hseipp]

Environment info

• NooBaa Version: 5.15.4
• Platform: CentOS 8, deployed using https://github.com/IBM/StorageScaleVagrant

Actual behavior

DeleteObjects trying to delete 2 versions of a key with their VersionId only deletes one version.

    @pytest.mark.versioning
    def test_versioning_multi_object_delete():
        client = get_client()
        bucket_name = get_new_bucket(client=client)

        check_configure_versioning_retry(bucket_name, "Enabled", "Enabled", client=client)

        key = 'key'
        num_versions = 2

        (version_ids, contents) = create_multiple_versions(client, bucket_name, key, num_versions)
        assert len(version_ids) == 2

        # delete both versions
        objects = [{'Key': key, 'VersionId': v} for v in version_ids]
        client.delete_objects(Bucket=bucket_name, Delete={'Objects': objects})

        response = client.list_object_versions(Bucket=bucket_name)
>       assert not 'Versions' in response
E       AssertionError: assert not 'Versions' in {'Delimiter': 'undefined', 'EncodingType': 'url', 'IsTruncated': False, 'KeyMarker': 'undefined', ...}

Expected behavior

DeleteObjects deletes all versions specified with VersionId.

Steps to reproduce

Execute test_versioning_multi_object_delete() like shown above.

More information - Screenshots / Logs / Other output

Log with "all" log level attached noobaa_20240913_2.log.gz

[nadavMiz]

@hseipp seems to be caused because because dir_file in options_gpfs is undefined: Sep 13 09:16:24 m1.example.com node[9580]: Sep-13 9:16:24.565 [nsfs/10263] [ERROR] core.util.native_fs_utils:: safe_unlink: dir_file is undefined, cannot use it to call safe_unlink_gpfs

there is already a fix related to this issue made by @shirady: https://github.com/noobaa/noobaa-core/pull/8345/files

we should test it again on gpfs machine after this fix is applied

[nadavMiz]

closing as test passes on GPFS machine with mentioned fix:

[root@tmtscalets-master s3-tests]# S3TEST_CONF=s3tests.conf tox -- s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete
.pkg: _optional_hooks> python /usr/local/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_sdist> python /usr/local/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_wheel> python /usr/local/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: prepare_metadata_for_build_wheel> python /usr/local/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_sdist> python /usr/local/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
py: install_package> python -I -m pip install --force-reinstall --no-deps /root/s3-tests/.tox/.tmp/package/9/s3tests-0.0.1.tar.gz
py: commands[0]> pytest s3tests_boto3/functional/test_s3.py::test_versioning_multi_object_delete
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.9.18, pytest-8.3.3, pluggy-1.5.0
cachedir: .tox/py/.pytest_cache
rootdir: /root/s3-tests
configfile: pytest.ini
collected 1 item                                                                                                                                                                                           

s3tests_boto3/functional/test_s3.py .                                                                                                                                                                [100%]

============================================================================================ 1 passed in 1.25s =============================================================================================
  py: OK (5.44=setup[3.74]+cmd[1.70] seconds)
  congratulations :) (5.56 seconds)

fix is already merged and should be included on sundays version