search

iterative / dvc

đŸ¦‰ Data Versioning and ML Experiments
https://dvc.org
Apache License 2.0
13.9k stars 1.19k forks source link

unexpected error - Forbidden: An error occurred (403) when calling the HeadObject operation: Forbidden #7238

Closed davebulaval closed 2 years ago

davebulaval commented 2 years ago

Bug Report

I have a strange behaviour with a DVC repository (ccf-project) that does not happen on other repositories connected to an S3-like bucket. My other repo is connected to AWS S3 buckets while the one with a strange behaviour is connected to an institutional S3-like bucket. (ccf-project). I have verified with institutional S3-like bucket admin and my credentials and my access are working (as of today).

Pattern

dvc pull or dvc push when an endpoint URL (the only difference I see with my other repositories.)

Consequence: Cannot pull or push data to my S3-like bucket.

Description

My config file in the ccf-project is the following:

[core]
    remote = storage
['remote "storage"']
    url = s3://bucket-poc-juliendavid
    endpointurl = https://s3.valeria.science

and I also have a config.local file to identify the profile to use from my credentials (in that case valeria).

It has been working before but had a lot of problems (see issue #6293) with it. I did not use it for a while and recently had to do stuff in it for a research paper and I cannot pull data from it.

However, if I change to another repository connected without an endpoint URL, it works fine. Note, that this uses my default AWS credential (but even when setting a profile to the default one it works).

Reproduce

  1. Dvc init
  2. Set a remote with an endpointurl
  3. dvc pull

Expected

Data is pull.

Environment information

DVC version: 2.9.3 (conda)
---------------------------------
Platform: Python 3.9.7 on Linux-5.11.0-44-generic-x86_64-with-glibc2.33
Supports:
    webhdfs (fsspec = 2021.11.1),
    http (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
    https (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
    s3 (s3fs = 2021.11.1, boto3 = 1.19.8)
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme0n1p3
Caches: local
Remotes: s3
Workspace directory: ext4 on /dev/nvme0n1p3
Repo: dvc, git

Output of pip check

notify-run 0.0.14 has requirement requests==2.22.0, but you have requests 2.27.0.

Output of dvc pull -v

dvc pull -v
2022-01-07 14:28:29,797 DEBUG: Adding '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/config.local' to gitignore file.
2022-01-07 14:28:29,805 DEBUG: Adding '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/tmp' to gitignore file.
2022-01-07 14:28:29,806 DEBUG: Adding '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache' to gitignore file.
2022-01-07 14:28:30,050 DEBUG: Preparing to transfer data from 'bucket-poc-juliendavid' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-07 14:28:30,050 DEBUG: Preparing to collect status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-07 14:28:30,050 DEBUG: Collecting status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-07 14:28:30,051 DEBUG: Preparing to collect status from 'bucket-poc-juliendavid'                                      
2022-01-07 14:28:30,051 DEBUG: Collecting status from 'bucket-poc-juliendavid'
2022-01-07 14:28:30,051 DEBUG: Querying 1 hashes via object_exists
2022-01-07 14:28:30,179 ERROR: unexpected error - Forbidden: An error occurred (403) when calling the HeadObject operation: Forbidden
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/s3fs/core.py", line 261, in _call_s3
    out = await method(**additional_kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/aiobotocore/client.py", line 180, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/main.py", line 55, in main
    ret = cmd.do_run()
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/command/base.py", line 45, in do_run
    return self.run()
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/command/data_sync.py", line 30, in run
    stats = self.repo.pull(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/pull.py", line 29, in pull
    processed_files_count = self.fetch(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/fetch.py", line 40, in fetch
    used = self.used_objs(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/__init__.py", line 431, in used_objs
    for odb, objs in self.index.used_objs(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/index.py", line 186, in used_objs
    for odb, objs in stage.get_used_objs(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/stage/__init__.py", line 660, in get_used_objs
    for odb, objs in out.get_used_objs(*args, **kwargs).items():
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/output.py", line 940, in get_used_objs
    obj = self._collect_used_dir_cache(**kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/output.py", line 882, in _collect_used_dir_cache
    self.get_dir_cache(jobs=jobs, remote=remote)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/output.py", line 864, in get_dir_cache
    self.repo.cloud.pull([obj.hash_info], **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/data_cloud.py", line 114, in pull
    return transfer(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/transfer.py", line 153, in transfer
    status = compare_status(src, dest, obj_ids, check_deleted=False, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/status.py", line 164, in compare_status
    src_exists, src_missing = status(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/status.py", line 122, in status
    exists = hashes.intersection(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/status.py", line 48, in _indexed_dir_hashes
    dir_exists.update(odb.list_hashes_exists(dir_hashes - dir_exists))
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/db/base.py", line 421, in list_hashes_exists
    ret = list(itertools.compress(hashes, in_remote))
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/concurrent/futures/_base.py", line 608, in result_iterator
    yield fs.pop().result()
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/concurrent/futures/_base.py", line 445, in result
    return self.__get_result()
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
    raise self._exception
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/db/base.py", line 412, in exists_with_progress
    ret = self.fs.exists(fs_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/fsspec_wrapper.py", line 91, in exists
    return self.fs.exists(path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/fsspec/asyn.py", line 91, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/fsspec/asyn.py", line 71, in sync
    raise return_result
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/fsspec/asyn.py", line 25, in _runner
    result[0] = await coro
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/s3fs/core.py", line 835, in _exists
    await self._info(path, bucket, key, version_id=version_id)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/s3fs/core.py", line 1029, in _info
    out = await self._call_s3(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/s3fs/core.py", line 281, in _call_s3
    raise err
PermissionError: Forbidden
------------------------------------------------------------
2022-01-07 14:28:30,233 DEBUG: Adding '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/config.local' to gitignore file.
2022-01-07 14:28:30,235 DEBUG: Adding '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/tmp' to gitignore file.
2022-01-07 14:28:30,236 DEBUG: Adding '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache' to gitignore file.
2022-01-07 14:28:30,236 DEBUG: [Errno 95] no more link types left to try out: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>: [Errno 95] Operation not supported
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/s3fs/core.py", line 261, in _call_s3
    out = await method(**additional_kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/aiobotocore/client.py", line 180, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/main.py", line 55, in main
    ret = cmd.do_run()
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/command/base.py", line 45, in do_run
    return self.run()
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/command/data_sync.py", line 30, in run
    stats = self.repo.pull(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/pull.py", line 29, in pull
    processed_files_count = self.fetch(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/fetch.py", line 40, in fetch
    used = self.used_objs(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/__init__.py", line 431, in used_objs
    for odb, objs in self.index.used_objs(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/index.py", line 186, in used_objs
    for odb, objs in stage.get_used_objs(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/stage/__init__.py", line 660, in get_used_objs
    for odb, objs in out.get_used_objs(*args, **kwargs).items():
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/output.py", line 940, in get_used_objs
    obj = self._collect_used_dir_cache(**kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/output.py", line 882, in _collect_used_dir_cache
    self.get_dir_cache(jobs=jobs, remote=remote)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/output.py", line 864, in get_dir_cache
    self.repo.cloud.pull([obj.hash_info], **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/data_cloud.py", line 114, in pull
    return transfer(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/transfer.py", line 153, in transfer
    status = compare_status(src, dest, obj_ids, check_deleted=False, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/status.py", line 164, in compare_status
    src_exists, src_missing = status(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/status.py", line 122, in status
    exists = hashes.intersection(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/status.py", line 48, in _indexed_dir_hashes
    dir_exists.update(odb.list_hashes_exists(dir_hashes - dir_exists))
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/db/base.py", line 421, in list_hashes_exists
    ret = list(itertools.compress(hashes, in_remote))
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/concurrent/futures/_base.py", line 608, in result_iterator
    yield fs.pop().result()
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/concurrent/futures/_base.py", line 445, in result
    return self.__get_result()
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
    raise self._exception
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/objects/db/base.py", line 412, in exists_with_progress
    ret = self.fs.exists(fs_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/fsspec_wrapper.py", line 91, in exists
    return self.fs.exists(path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/fsspec/asyn.py", line 91, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/fsspec/asyn.py", line 71, in sync
    raise return_result
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/fsspec/asyn.py", line 25, in _runner
    result[0] = await coro
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/s3fs/core.py", line 835, in _exists
    await self._info(path, bucket, key, version_id=version_id)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/s3fs/core.py", line 1029, in _info
    out = await self._call_s3(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/s3fs/core.py", line 281, in _call_s3
    raise err
PermissionError: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 28, in _link
    func(from_path, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/local.py", line 148, in reflink
    System.reflink(from_info, to_info)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 112, in reflink
    System._reflink_linux(source, link_name)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 96, in _reflink_linux
    fcntl.ioctl(d.fileno(), FICLONE, s.fileno())
OSError: [Errno 95] Operation not supported

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 69, in _try_links
    return _link(link, from_fs, from_path, to_fs, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 32, in _link
    raise OSError(
OSError: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 124, in _test_link
    _try_links([link], from_fs, from_file, to_fs, to_file)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 77, in _try_links
    raise OSError(
OSError: [Errno 95] no more link types left to try out
------------------------------------------------------------
2022-01-07 14:28:30,238 DEBUG: Removing '/home/despacitov3/Github/.M4RUi8Ag52aWdZdrdf52W6.tmp'
2022-01-07 14:28:30,238 DEBUG: Removing '/home/despacitov3/Github/.M4RUi8Ag52aWdZdrdf52W6.tmp'
2022-01-07 14:28:30,238 DEBUG: Removing '/home/despacitov3/Github/.M4RUi8Ag52aWdZdrdf52W6.tmp'
2022-01-07 14:28:30,238 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.65UFbt5gKA6rDsQVm3NdCA.tmp'
2022-01-07 14:28:30,245 DEBUG: Version info for developers:
DVC version: 2.9.3 (conda)
---------------------------------
Platform: Python 3.9.7 on Linux-5.11.0-44-generic-x86_64-with-glibc2.33
Supports:
    webhdfs (fsspec = 2021.11.1),
    http (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
    https (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
    s3 (s3fs = 2021.11.1, boto3 = 1.19.8)
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme0n1p3
Caches: local
Remotes: s3
Workspace directory: ext4 on /dev/nvme0n1p3
Repo: dvc, git

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2022-01-07 14:28:30,247 DEBUG: Analytics is enabled.
2022-01-07 14:28:30,289 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpz9rtnzin']'
2022-01-07 14:28:30,290 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpz9rtnzin']'
efiop commented 2 years ago

Hi @davebulaval đŸ™‚

Let's try to narrow it down a bit. Does aws cli work with it?

davebulaval commented 2 years ago

Hi @efiop

It is not an AWS bucket the one I have an error message on (the ccf-project). It's an S3-like bucket given by my university.

efiop commented 2 years ago

@davebulaval Right, I'm just saying that it also operates (at least it should) through s3 protocol, so if aws cli tool (https://aws.amazon.com/cli/ , e.g. aws s3 ...) doesn't work with it - dvc also won't.

davebulaval commented 2 years ago

@efiop ahhhh did not know. Good to know.

So I've tested the following command

aws s3 ls s3://bucket-poc-juliendavid --endpoint-url https://s3.valeria.science --profile valeria

And I get a list of elements in the bucket as expected (hashed files).

efiop commented 2 years ago

@davebulaval And could you try head-object on one of those objects that you got? E.g.

aws s3api head-object --bucket bucket-poc-juliendavid --key mykey --endpoint-url https://s3.valeria.science --profile valeria

Also, are you sure you've configured endpoint_url and profile in dvc config? Just making sure.

If s3api works, could you also try using some particular directory instead of the whole bucket as a remote in dvc? E.g. s3://bucket-poc-juliendavid/dvcremote

davebulaval commented 2 years ago
  1. So, I have a test-folder in the bucket and tested with the following command:
aws s3api head-object --bucket bucket-poc-juliendavid --key test-folder/ --endpoint-url https://s3.valeria.science --profile valeria

and I got the following .json return:

{
    "AcceptRanges": "bytes",
    "LastModified": "2020-08-19T18:07:17+00:00",
    "ContentLength": 0,
    "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"",
    "ContentType": "binary/octet-stream",
    "Metadata": {}
}
  1. Yes, it is configured in the project .config file. Here is a copy-paste of it.
[core]
    remote = storage
['remote "storage"']
    url = s3://bucket-poc-juliendavid
    endpointurl = https://s3.valeria.science
  1. Interesting, I have changed the remote URL to url = s3://bucket-poc-juliendavid/test-folder and I get different errors. For sure, It said some data are missing. Here is the complete output of dvc pull -v.
2022-01-08 08:32:38,094 DEBUG: Adding '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/config.local' to gitignore file.
2022-01-08 08:32:38,102 DEBUG: Adding '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/tmp' to gitignore file.
2022-01-08 08:32:38,103 DEBUG: Adding '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache' to gitignore file.
2022-01-08 08:32:38,343 DEBUG: Preparing to transfer data from 'bucket-poc-juliendavid/test-folder' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:38,343 DEBUG: Preparing to collect status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:38,343 DEBUG: Collecting status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:38,344 DEBUG: Preparing to collect status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:38,344 DEBUG: Collecting status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:38,344 DEBUG: Querying 1 hashes via object_exists
2022-01-08 08:32:38,478 DEBUG: Querying 1 hashes via object_exists                            
2022-01-08 08:32:38,514 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:
name: None, md5: 615a71e1d3db93779cd14151895b5548.dir
2022-01-08 08:32:38,537 DEBUG: Preparing to transfer data from 'bucket-poc-juliendavid/test-folder' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:38,537 DEBUG: Preparing to collect status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:38,537 DEBUG: Collecting status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:38,538 DEBUG: Preparing to collect status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:38,538 DEBUG: Collecting status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:38,538 DEBUG: Querying 1 hashes via object_exists
2022-01-08 08:32:38,650 DEBUG: Querying 1 hashes via object_exists                            
2022-01-08 08:32:38,686 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:
name: None, md5: a20b5b7040d54bc9ff5867428f4fdbfa.dir
2022-01-08 08:32:38,700 DEBUG: Preparing to transfer data from 'bucket-poc-juliendavid/test-folder' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:38,701 DEBUG: Preparing to collect status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:38,701 DEBUG: Collecting status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:38,701 DEBUG: Preparing to collect status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:38,701 DEBUG: Collecting status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:38,701 DEBUG: Querying 1 hashes via object_exists
2022-01-08 08:32:40,846 DEBUG: Querying 1 hashes via object_exists                            
2022-01-08 08:32:40,880 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:
name: None, md5: be0ba2c149da0f9f7aaf22dad2cee95e.dir
2022-01-08 08:32:40,922 DEBUG: Preparing to transfer data from 'bucket-poc-juliendavid/test-folder' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:40,922 DEBUG: Preparing to collect status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:40,922 DEBUG: Collecting status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:40,923 DEBUG: Preparing to collect status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:40,923 DEBUG: Collecting status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:40,923 DEBUG: Querying 1 hashes via object_exists
2022-01-08 08:32:41,048 DEBUG: Querying 1 hashes via object_exists                            
2022-01-08 08:32:41,080 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:
name: None, md5: a5c1b6f7bae00bf8780f580a8213c54e.dir
2022-01-08 08:32:41,099 DEBUG: Preparing to transfer data from 'bucket-poc-juliendavid/test-folder' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:41,099 DEBUG: Preparing to collect status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:41,100 DEBUG: Collecting status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:41,100 DEBUG: Preparing to collect status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:41,100 DEBUG: Collecting status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:41,100 DEBUG: Querying 1 hashes via object_exists
2022-01-08 08:32:41,224 DEBUG: Querying 1 hashes via object_exists                            
2022-01-08 08:32:41,262 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:
name: None, md5: 48847aeb2c2393e923b0dac00476e18b.dir
2022-01-08 08:32:41,276 DEBUG: Preparing to transfer data from 'bucket-poc-juliendavid/test-folder' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:41,276 DEBUG: Preparing to collect status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:41,276 DEBUG: Collecting status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:41,277 DEBUG: Preparing to collect status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:41,277 DEBUG: Collecting status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:41,277 DEBUG: Querying 1 hashes via object_exists
2022-01-08 08:32:41,391 DEBUG: Querying 1 hashes via object_exists                            
2022-01-08 08:32:41,426 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:
name: None, md5: 777a51328c70efff4a6f709da0f4dd92.dir
2022-01-08 08:32:41,440 DEBUG: Preparing to transfer data from 'bucket-poc-juliendavid/test-folder' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:41,441 DEBUG: Preparing to collect status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:41,441 DEBUG: Collecting status from '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache'
2022-01-08 08:32:41,442 DEBUG: Preparing to collect status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:41,442 DEBUG: Collecting status from 'bucket-poc-juliendavid/test-folder'
2022-01-08 08:32:41,442 DEBUG: Querying 2 hashes via object_exists
2022-01-08 08:32:41,618 DEBUG: Estimated remote size: 4096 files                              
2022-01-08 08:32:41,619 DEBUG: Querying '94' hashes via traverse                              
2022-01-08 08:32:41,638 DEBUG: 'bucket-poc-juliendavid/test-folder/' doesn't look like a cache file, skipping
2022-01-08 08:32:41,638 DEBUG: 'bucket-poc-juliendavid/test-folder/test.txt' doesn't look like a cache file, skipping
2022-01-08 08:32:41,638 DEBUG: 'bucket-poc-juliendavid/test-folder/testAD.txt' doesn't look like a cache file, skipping
2022-01-08 08:32:41,639 DEBUG: 'bucket-poc-juliendavid/test-folder/tools.py' doesn't look like a cache file, skipping
2022-01-08 08:32:41,639 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:
name: data/raw/spla_offre_iid_finance-date.json, md5: 161abc98b97fe854442128ae9b5643dc
name: data/raw/Analyse de bases de données_Offre d'emploi des compagnies d'assurance.xlsx, md5: 8256338beabb49fc2013ea4f2cf348bf
name: data/raw/KornFerry-Competences.xlsx, md5: f1cc4fd8870fff3953e907e20b1b6cf1
name: data/raw/spla_offre_iid_finance-utf8si-pandas.json, md5: 02c7669f4d6da6dc3cc721db1f0d11f4
name: data/raw/spla_offre_iid_finance-date-utf8-pandas.json, md5: 49c96e58396c8cd35c04d1d818d94605
name: data/raw/spla_offre_iid_finance-utf8si.json, md5: ff67721c13297cb2cadcc7ce3c1db20e
2022-01-08 08:32:41,654 DEBUG: [Errno 95] no more link types left to try out: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>: [Errno 95] Operation not supported
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 28, in _link
    func(from_path, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/local.py", line 148, in reflink
    System.reflink(from_info, to_info)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 112, in reflink
    System._reflink_linux(source, link_name)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 96, in _reflink_linux
    fcntl.ioctl(d.fileno(), FICLONE, s.fileno())
OSError: [Errno 95] Operation not supported

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 69, in _try_links
    return _link(link, from_fs, from_path, to_fs, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 32, in _link
    raise OSError(
OSError: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 124, in _test_link
    _try_links([link], from_fs, from_file, to_fs, to_file)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 77, in _try_links
    raise OSError(
OSError: [Errno 95] no more link types left to try out
------------------------------------------------------------
2022-01-08 08:32:41,656 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.oPGgGL2Bt7NWoS9VZNJaSp.tmp'
2022-01-08 08:32:41,656 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.aDSN6wtV5rbN2FGeaRDjbR.tmp' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.oPGgGL2Bt7NWoS9VZNJaSp.tmp'
2022-01-08 08:32:41,657 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.oPGgGL2Bt7NWoS9VZNJaSp.tmp'                                            
2022-01-08 08:32:41,658 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.aDSN6wtV5rbN2FGeaRDjbR.tmp'
2022-01-08 08:32:41,658 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/02/c7669f4d6da6dc3cc721db1f0d11f4' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-utf8si-pandas.json'
2022-01-08 08:32:41,660 DEBUG: [Errno 95] no more link types left to try out: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>: [Errno 95] Operation not supported
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 28, in _link
    func(from_path, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/local.py", line 148, in reflink
    System.reflink(from_info, to_info)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 112, in reflink
    System._reflink_linux(source, link_name)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 96, in _reflink_linux
    fcntl.ioctl(d.fileno(), FICLONE, s.fileno())
OSError: [Errno 95] Operation not supported

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 69, in _try_links
    return _link(link, from_fs, from_path, to_fs, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 32, in _link
    raise OSError(
OSError: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 124, in _test_link
    _try_links([link], from_fs, from_file, to_fs, to_file)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 77, in _try_links
    raise OSError(
OSError: [Errno 95] no more link types left to try out
------------------------------------------------------------
2022-01-08 08:32:41,660 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.GPLoVohD5VPG9FnWLrssDw.tmp'
2022-01-08 08:32:41,661 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.P8KkxXjs6PJqCMxNzsfXhv.tmp' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.GPLoVohD5VPG9FnWLrssDw.tmp'
2022-01-08 08:32:41,662 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.GPLoVohD5VPG9FnWLrssDw.tmp'                                            
2022-01-08 08:32:41,662 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.P8KkxXjs6PJqCMxNzsfXhv.tmp'
2022-01-08 08:32:41,663 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/82/56338beabb49fc2013ea4f2cf348bf' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/Analyse de bases de données_Offre d'emploi des compagnies d'assurance.xlsx'
2022-01-08 08:32:41,664 DEBUG: [Errno 95] no more link types left to try out: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>: [Errno 95] Operation not supported
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 28, in _link
    func(from_path, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/local.py", line 148, in reflink
    System.reflink(from_info, to_info)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 112, in reflink
    System._reflink_linux(source, link_name)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 96, in _reflink_linux
    fcntl.ioctl(d.fileno(), FICLONE, s.fileno())
OSError: [Errno 95] Operation not supported

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 69, in _try_links
    return _link(link, from_fs, from_path, to_fs, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 32, in _link
    raise OSError(
OSError: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 124, in _test_link
    _try_links([link], from_fs, from_file, to_fs, to_file)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 77, in _try_links
    raise OSError(
OSError: [Errno 95] no more link types left to try out
------------------------------------------------------------
2022-01-08 08:32:41,665 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.kVV7uEU7YApYKFQpmwvubi.tmp'
2022-01-08 08:32:41,665 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.eAyrzMxeigbak7Re9Bj3Bn.tmp' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.kVV7uEU7YApYKFQpmwvubi.tmp'
2022-01-08 08:32:41,666 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.kVV7uEU7YApYKFQpmwvubi.tmp'                                            
2022-01-08 08:32:41,667 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.eAyrzMxeigbak7Re9Bj3Bn.tmp'
2022-01-08 08:32:41,667 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/49/c96e58396c8cd35c04d1d818d94605' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-date-utf8-pandas.json'
2022-01-08 08:32:41,669 DEBUG: [Errno 95] no more link types left to try out: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>: [Errno 95] Operation not supported
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 28, in _link
    func(from_path, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/local.py", line 148, in reflink
    System.reflink(from_info, to_info)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 112, in reflink
    System._reflink_linux(source, link_name)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 96, in _reflink_linux
    fcntl.ioctl(d.fileno(), FICLONE, s.fileno())
OSError: [Errno 95] Operation not supported

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 69, in _try_links
    return _link(link, from_fs, from_path, to_fs, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 32, in _link
    raise OSError(
OSError: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 124, in _test_link
    _try_links([link], from_fs, from_file, to_fs, to_file)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 77, in _try_links
    raise OSError(
OSError: [Errno 95] no more link types left to try out
------------------------------------------------------------
2022-01-08 08:32:41,670 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.inJKRRhTHk93KxYvBDdVZF.tmp'
2022-01-08 08:32:41,670 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.jjhjCSmMznPNnAXhKm4DJA.tmp' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.inJKRRhTHk93KxYvBDdVZF.tmp'
2022-01-08 08:32:41,671 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.inJKRRhTHk93KxYvBDdVZF.tmp'                                            
2022-01-08 08:32:41,671 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.jjhjCSmMznPNnAXhKm4DJA.tmp'
2022-01-08 08:32:41,672 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/f1/cc4fd8870fff3953e907e20b1b6cf1' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/KornFerry-Competences.xlsx'
2022-01-08 08:32:41,674 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/test'. It won't be created.                  
2022-01-08 08:32:41,677 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/intermediate/models'. It won't be created.
2022-01-08 08:32:41,680 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/trained/models'. It won't be created.
2022-01-08 08:32:41,681 DEBUG: [Errno 95] no more link types left to try out: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>: [Errno 95] Operation not supported
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 28, in _link
    func(from_path, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/local.py", line 148, in reflink
    System.reflink(from_info, to_info)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 112, in reflink
    System._reflink_linux(source, link_name)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 96, in _reflink_linux
    fcntl.ioctl(d.fileno(), FICLONE, s.fileno())
OSError: [Errno 95] Operation not supported

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 69, in _try_links
    return _link(link, from_fs, from_path, to_fs, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 32, in _link
    raise OSError(
OSError: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 124, in _test_link
    _try_links([link], from_fs, from_file, to_fs, to_file)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 77, in _try_links
    raise OSError(
OSError: [Errno 95] no more link types left to try out
------------------------------------------------------------
2022-01-08 08:32:41,682 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.GrZ5Am5hxxm4knnrfRptts.tmp'
2022-01-08 08:32:41,682 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.5vDiCG9UAXvNfeadG3DzM4.tmp' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.GrZ5Am5hxxm4knnrfRptts.tmp'
2022-01-08 08:32:41,683 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.GrZ5Am5hxxm4knnrfRptts.tmp'                                            
2022-01-08 08:32:41,683 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.5vDiCG9UAXvNfeadG3DzM4.tmp'
2022-01-08 08:32:41,684 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/16/1abc98b97fe854442128ae9b5643dc' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-date.json'
2022-01-08 08:32:41,685 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/ccf/experiments/model'. It won't be created.                
2022-01-08 08:32:41,687 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/train'. It won't be created.
2022-01-08 08:32:41,689 DEBUG: [Errno 95] no more link types left to try out: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>: [Errno 95] Operation not supported
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 28, in _link
    func(from_path, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/local.py", line 148, in reflink
    System.reflink(from_info, to_info)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 112, in reflink
    System._reflink_linux(source, link_name)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/system.py", line 96, in _reflink_linux
    fcntl.ioctl(d.fileno(), FICLONE, s.fileno())
OSError: [Errno 95] Operation not supported

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 69, in _try_links
    return _link(link, from_fs, from_path, to_fs, to_path)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 32, in _link
    raise OSError(
OSError: [Errno 95] 'reflink' is not supported by <class 'dvc.fs.local.LocalFileSystem'>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 124, in _test_link
    _try_links([link], from_fs, from_file, to_fs, to_file)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/fs/utils.py", line 77, in _try_links
    raise OSError(
OSError: [Errno 95] no more link types left to try out
------------------------------------------------------------
2022-01-08 08:32:41,689 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.dnoEpTgNCe9PANLuhjiUt6.tmp'
2022-01-08 08:32:41,690 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.Dq853wm686dH2AKCp6AfST.tmp' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.dnoEpTgNCe9PANLuhjiUt6.tmp'
2022-01-08 08:32:41,691 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/.dnoEpTgNCe9PANLuhjiUt6.tmp'                                            
2022-01-08 08:32:41,691 DEBUG: Removing '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/.Dq853wm686dH2AKCp6AfST.tmp'
2022-01-08 08:32:41,692 DEBUG: Uploading '/home/despacitov3/Github/CCF-transformation-numerique-competences/.dvc/cache/ff/67721c13297cb2cadcc7ce3c1db20e' to '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-utf8si.json'
2022-01-08 08:32:41,693 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/anonymized'. It won't be created.                      
12 files failed                                                                               
2022-01-08 08:32:41,693 ERROR: failed to pull data from the cloud - Checkout failed for following targets:
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-utf8si-pandas.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/Analyse de bases de données_Offre d'emploi des compagnies d'assurance.xlsx
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-date-utf8-pandas.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/KornFerry-Competences.xlsx
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/test
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/intermediate/models
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/trained/models
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-date.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/ccf/experiments/model
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/train
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-utf8si.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/anonymized
Is your cache up to date?
<https://error.dvc.org/missing-files>
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/command/data_sync.py", line 30, in run
    stats = self.repo.pull(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/pull.py", line 40, in pull
    stats = self.checkout(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/checkout.py", line 110, in checkout
    raise CheckoutError(stats["failed"], stats)
dvc.exceptions.CheckoutError: Checkout failed for following targets:
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-utf8si-pandas.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/Analyse de bases de données_Offre d'emploi des compagnies d'assurance.xlsx
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-date-utf8-pandas.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/KornFerry-Competences.xlsx
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/test
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/intermediate/models
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/trained/models
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-date.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/ccf/experiments/model
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/train
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-utf8si.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/anonymized
Is your cache up to date?
<https://error.dvc.org/missing-files>
------------------------------------------------------------
2022-01-08 08:32:41,703 DEBUG: Analytics is enabled.
2022-01-08 08:32:41,746 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpgrv0_0s9']'
2022-01-08 08:32:41,748 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpgrv0_0s9']'
efiop commented 2 years ago

Interesting, I have changed the remote URL to url = s3://bucket-poc-juliendavid/test-folder and I get different errors. For sure, It said some data are missing. Here is the complete output of dvc pull -v.

And could you try dvc push, please? pull is saying that there are missing files on remote, which is no wonder if you didn't push them yet.

davebulaval commented 2 years ago

dvc push work. It has push the file in the test_folder within the bucket.

efiop commented 2 years ago

@davebulaval And dvc pull after dvc push doesn't?

davebulaval commented 2 years ago

I got the following

2022-01-10 16:50:53,778 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:     
name: None, md5: 615a71e1d3db93779cd14151895b5548.dir
2022-01-10 16:50:56,007 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:     
name: None, md5: a20b5b7040d54bc9ff5867428f4fdbfa.dir
2022-01-10 16:50:56,241 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:     
name: None, md5: be0ba2c149da0f9f7aaf22dad2cee95e.dir
2022-01-10 16:50:56,448 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:     
name: None, md5: a5c1b6f7bae00bf8780f580a8213c54e.dir
2022-01-10 16:50:56,660 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:     
name: None, md5: 48847aeb2c2393e923b0dac00476e18b.dir
2022-01-10 16:50:56,866 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:     
name: None, md5: 777a51328c70efff4a6f709da0f4dd92.dir
2022-01-10 16:50:57,167 WARNING: Some of the cache files do not exist neither locally nor on remote. Missing cache files:     
name: data/raw/spla_offre_iid_finance-date.json, md5: 161abc98b97fe854442128ae9b5643dc
name: data/raw/KornFerry-Competences.xlsx, md5: f1cc4fd8870fff3953e907e20b1b6cf1
name: data/raw/spla_offre_iid_finance-utf8si-pandas.json, md5: 02c7669f4d6da6dc3cc721db1f0d11f4
name: data/raw/Analyse de bases de données_Offre d'emploi des compagnies d'assurance.xlsx, md5: 8256338beabb49fc2013ea4f2cf348bf
name: data/raw/spla_offre_iid_finance-utf8si.json, md5: ff67721c13297cb2cadcc7ce3c1db20e
name: data/raw/spla_offre_iid_finance-date-utf8-pandas.json, md5: 49c96e58396c8cd35c04d1d818d94605
2022-01-10 16:50:57,171 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/train'. It won't be created.
2022-01-10 16:50:57,184 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/ccf/experiments/model'. It won't be created.                                                                                
2022-01-10 16:50:57,234 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/anonymized'. It won't be created.                                                                                      
2022-01-10 16:50:57,253 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/test'. It won't be created.                                                                                  
2022-01-10 16:50:57,264 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/intermediate/models'. It won't be created.                                                                             
2022-01-10 16:50:57,267 WARNING: No file hash info found for '/home/despacitov3/Github/CCF-transformation-numerique-competences/data/trained/models'. It won't be created.
2022-01-10 16:50:57,268 ERROR: failed to pull data from the cloud - Checkout failed for following targets:                    
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/train
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/Analyse de bases de données_Offre d'emploi des compagnies d'assurance.xlsx
/home/despacitov3/Github/CCF-transformation-numerique-competences/ccf/experiments/model
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-utf8si.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-utf8si-pandas.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-date-utf8-pandas.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/anonymized
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-date.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/test
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/KornFerry-Competences.xlsx
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/intermediate/models
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/trained/models
Is your cache up to date?
<https://error.dvc.org/missing-files>
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/command/data_sync.py", line 30, in run
    stats = self.repo.pull(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/pull.py", line 40, in pull
    stats = self.checkout(
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/despacitov3/anaconda3/envs/py3.9/lib/python3.9/site-packages/dvc/repo/checkout.py", line 110, in checkout
    raise CheckoutError(stats["failed"], stats)
dvc.exceptions.CheckoutError: Checkout failed for following targets:
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/train
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/Analyse de bases de données_Offre d'emploi des compagnies d'assurance.xlsx
/home/despacitov3/Github/CCF-transformation-numerique-competences/ccf/experiments/model
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-utf8si.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-utf8si-pandas.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-date-utf8-pandas.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/anonymized
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/spla_offre_iid_finance-date.json
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/annotated/test
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/raw/KornFerry-Competences.xlsx
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/intermediate/models
/home/despacitov3/Github/CCF-transformation-numerique-competences/data/trained/models
Is your cache up to date?
<https://error.dvc.org/missing-files>
efiop commented 2 years ago

@davebulaval Looks like you either didn't push something or there is some serious bug. Try to check if 16/1abc98b97fe854442128ae9b5643dc exists in remote using webui and aws cli.

davebulaval commented 2 years ago

At this point, I think it is on the institutional side of the problem. And it become too long to solve, so I've used the classic "send me data by email" for now. I will close the issue for that reason.

fengwang commented 2 years ago

@davebulaval I have almost the same error as yours. Have you figured out a way to solve this issue?

davebulaval commented 2 years ago

@fengwang I did not find a solution other than handling data by unconventional and unsecured email.

DendiHust commented 2 years ago

The server time may be different from the local time