danielbellsa commented 1 year ago

Only as of recently, have I had this error with DVC.

Tracback:

(venv) me@ubuntu-pcs:~/PycharmProjects/project$ dvc push
ERROR: unexpected error - _register_s3_control_events() takes 2 positional arguments but 6 were given                                                                                                                                                          

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

(venv) me@ubuntu-pcs:~/PycharmProjects/project$ dvc doctor
DVC version: 2.28.0 (pip)
---------------------------------
Platform: Python 3.9.13 on Linux-5.15.0-52-generic-x86_64-with-glibc2.35
Subprojects:
        dvc_data = 0.13.0
        dvc_objects = 0.5.0
        dvc_render = 0.0.11
        dvc_task = 0.1.2
        dvclive = 0.12.1
        scmrepo = 0.1.1
Supports:
        http (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
        s3 (s3fs = 2022.10.0, boto3 = 1.24.59)
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme0n1p5
Caches: local
Remotes: s3
Workspace directory: ext4 on /dev/nvme0n1p5
Repo: dvc, git

.dvc/config/:

[core]
    remote = storage
    autostage = true
['remote "storage"']
    url = s3://bucket/project/dvcstore/

Let me know if I should add anything else to the post.

skshetry commented 1 year ago

Looks like an issue with a recent release of botocore>=1.28.0. If you can pass the output of dvc push -v, we can trace where it's being called (most likely in aiobotocore). Meanwhile, you can install botocore<1.28.0.

See https://github.com/aio-libs/aiobotocore/blob/fbfeb1242ab1f378f6d4fe84b5d3dacdd8874189/aiobotocore/client.py#L82 compared to https://github.com/boto/botocore/blob/3d3f40ca854b90e5bb8eae88686bc3c6dcb85839/botocore/client.py#L161.

danielbellsa commented 1 year ago

(venv) me@ubuntu-pcs:~/PycharmProjects/project $ dvc push -v
2022-11-01 16:13:01,299 DEBUG: Preparing to transfer data from '/home/daniel/PycharmProjects/pdl1-lung/.dvc/cache' to 'nihr-data-restricted/pdl1-lung/dvcstore/'
2022-11-01 16:13:01,299 DEBUG: Preparing to collect status from 'nihr-data-restricted/pdl1-lung/dvcstore/'
2022-11-01 16:13:01,302 DEBUG: Collecting status from 'nihr-data-restricted/pdl1-lung/dvcstore/'
2022-11-01 16:13:01,406 DEBUG: Querying 20 oids via object_exists                                                                                                                                                                                              
2022-11-01 16:13:01,603 ERROR: unexpected error - _register_s3_control_events() takes 2 positional arguments but 6 were given                                                                                                                                  
------------------------------------------------------------
Traceback (most recent call last):
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc/cli/__init__.py", line 185, in main
    ret = cmd.do_run()
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc/cli/command.py", line 22, in do_run
    return self.run()
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc/commands/data_sync.py", line 59, in run
    processed_files_count = self.repo.push(
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc/repo/__init__.py", line 49, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc/repo/push.py", line 116, in push
    pushed += self.cloud.push(
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc/data_cloud.py", line 135, in push
    return self.transfer(
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc/data_cloud.py", line 114, in transfer
    return transfer(src_odb, dest_odb, objs, **kwargs)
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc_data/hashfile/transfer.py", line 158, in transfer
    status = compare_status(
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc_data/hashfile/status.py", line 179, in compare_status
    dest_exists, dest_missing = status(
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc_data/hashfile/status.py", line 136, in status
    exists = hashes.intersection(
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc_data/hashfile/status.py", line 43, in _indexed_dir_hashes
    indexed_dir_exists.update(hashes)
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/tqdm/std.py", line 1195, in __iter__
    for obj in iterable:
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc_objects/db.py", line 300, in list_oids_exists
    yield from itertools.compress(oids, in_remote)
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/concurrent/futures/_base.py", line 609, in result_iterator
    yield fs.pop().result()
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/concurrent/futures/_base.py", line 446, in result
    return self.__get_result()
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/dvc_objects/fs/base.py", line 275, in exists
    return self.fs.exists(path)
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/fsspec/asyn.py", line 111, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/fsspec/asyn.py", line 96, in sync
    raise return_result
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/fsspec/asyn.py", line 53, in _runner
    result[0] = await coro
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/s3fs/core.py", line 929, in _exists
    await self._info(path, bucket, key, version_id=version_id)
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/s3fs/core.py", line 1193, in _info
    out = await self._call_s3(
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/s3fs/core.py", line 331, in _call_s3
    await self.set_session()
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/s3fs/core.py", line 502, in set_session
    self._s3 = await s3creator.get_client()
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/s3fs/utils.py", line 66, in get_client
    self._client = await self._stack.enter_async_context(
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/contextlib.py", line 575, in enter_async_context
    result = await _cm_type.__aenter__(cm)
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/aiobotocore/session.py", line 26, in __aenter__
    self._client = await self._coro
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/aiobotocore/session.py", line 193, in _create_client
    client = await client_creator.create_client(
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/aiobotocore/client.py", line 82, in create_client
    self._register_s3_control_events(
TypeError: _register_s3_control_events() takes 2 positional arguments but 6 were given
------------------------------------------------------------
2022-11-01 16:13:01,650 DEBUG: link type reflink is not available ([Errno 95] no more link types left to try out)
2022-11-01 16:13:01,650 DEBUG: Removing '/home/daniel/PycharmProjects/.ZJBRYiBYFZfn7GyrGQRheL.tmp'
2022-11-01 16:13:01,650 DEBUG: Removing '/home/daniel/PycharmProjects/.ZJBRYiBYFZfn7GyrGQRheL.tmp'
2022-11-01 16:13:01,650 DEBUG: Removing '/home/daniel/PycharmProjects/.ZJBRYiBYFZfn7GyrGQRheL.tmp'
2022-11-01 16:13:01,651 DEBUG: Removing '/home/daniel/PycharmProjects/pdl1-lung/.dvc/cache/.EKkW9MupYBZgSUbqTLzcru.tmp'
2022-11-01 16:13:01,659 DEBUG: Version info for developers:
DVC version: 2.28.0 (pip)
---------------------------------
Platform: Python 3.9.13 on Linux-5.15.0-52-generic-x86_64-with-glibc2.35
Subprojects:
        dvc_data = 0.13.0
        dvc_objects = 0.5.0
        dvc_render = 0.0.11
        dvc_task = 0.1.2
        dvclive = 0.12.1
        scmrepo = 0.1.1
Supports:
        http (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.8.3, aiohttp-retry = 2.8.3),
        s3 (s3fs = 2022.10.0, boto3 = 1.24.59)
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme0n1p5
Caches: local
Remotes: s3
Workspace directory: ext4 on /dev/nvme0n1p5
Repo: dvc, git

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2022-11-01 16:13:01,661 DEBUG: Analytics is enabled.
2022-11-01 16:13:01,724 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmp9k8dx5ud']'
2022-11-01 16:13:01,726 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmp9k8dx5ud']'

pared commented 1 year ago

@danielbellsa could you also run pip freeze | grep aiobotocore?

danielbellsa commented 1 year ago

@danielbellsa could you also run pip freeze | grep aiobotocore?

(pdl1lung) daniel@ubuntu-pcs:~/PycharmProjects/pdl1-lung$ pip freeze | grep aiobotocore
aiobotocore==2.4.0

pared commented 1 year ago

Sorry, should have asked for botocore too. I have an error on 1.28.0 but completely different.

danielbellsa commented 1 year ago

(pdl1lung) daniel@ubuntu-pcs:~/PycharmProjects/pdl1-lung$ pip freeze | grep botocore
aiobotocore==2.4.0
botocore==1.28.3

pared commented 1 year ago

I am able to reproduce for botocore==1.28.3

efiop commented 1 year ago

aiobotocore requires a particular botocore version. What most likely happened is you probably installed some other package that conflicts with aiobotocore. Check that pip check doesn't complain about anything.

danielbellsa commented 1 year ago

aiobotocore requires a particular botocore version. What most likely happened is you probably installed some other package that conflicts with aiobotocore. Check that pip check doesn't complain about anything.

(pdl1lung) daniel@ubuntu-pcs:~/PycharmProjects/pdl1-lung$ pip check
ERROR: Exception:
Traceback (most recent call last):
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2681, in version
    return self._version
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _version

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 228, in _main
    status = self.run(options, args)
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/pip/_internal/commands/check.py", line 28, in run
    package_set, parsing_probs = create_package_set_from_installed()
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/pip/_internal/operations/check.py", line 50, in create_package_set_from_installed
    package_set[name] = PackageDetails(dist.version, dist.requires())
  File "/home/daniel/miniconda3/envs/pdl1lung/lib/python3.9/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2689, in version
    raise ValueError(msg, self)
ValueError: ("Missing 'Version:' header and/or PKG-INFO file at path: /home/daniel/PycharmProjects/pdl1-lung/pdl1_lung.egg-info/PKG-INFO", pdl1-lung [unknown version] (/home/daniel/PycharmProjects/pdl1-lung))

efiop commented 1 year ago

Looks like some other package is messed up in your environment that is not related to dvc at all. Consider recreating the env or installing dvc through pipx/snap/conda/etc.

Closing since this is env issue.

danielbellsa commented 1 year ago

Sure thing. Thanks

danielbellsa commented 1 year ago

Recreating venv didn't work but pip check is chill now:

(pdl1lung) daniel@ubuntu-pcs:~/PycharmProjects/pdl1-lung$ pip check
tables 4.3.2 has requirement pandas<1.4,>=1.0, but you have pandas 1.5.1.
tables 4.3.2 has requirement psutil==5.8.0, but you have psutil 5.9.3.
tables 4.3.2 has requirement shapely==1.8.0, but you have shapely 1.8.2.
sonpy 1.2.0 has requirement scikit-learn==0.24.2, but you have scikit-learn 1.0.2.
sonpy 1.2.0 has requirement scipy==1.8.1, but you have scipy 1.9.3.
indra-imaging 19.3.0 has requirement bidict==0.21.2, but you have bidict 0.22.0.
indra-imaging 19.3.0 has requirement numpy==1.23.0, but you have numpy 1.21.6.
indra-imaging 19.3.0 has requirement pillow==9.2.0, but you have pillow 9.3.0.
indra-imaging 19.3.0 has requirement python-decouple==3.4, but you have python-decouple 3.6.
indra-imaging 19.3.0 has requirement shapely==1.8.0, but you have shapely 1.8.2.
indra-dash 10.2.5 has requirement python-decouple==3.4, but you have python-decouple 3.6.
indra-dash 10.2.5 has requirement requests==2.26.0, but you have requests 2.28.1.
images 0.5.1 has requirement geopandas==0.10.2, but you have geopandas 0.11.1.
images 0.5.1 has requirement numpy==1.23, but you have numpy 1.21.6.
images 0.5.1 has requirement pillow==9.2.0, but you have pillow 9.3.0.
images 0.5.1 has requirement python-decouple==3.4, but you have python-decouple 3.6.
images 0.5.1 has requirement shapely==1.8.0, but you have shapely 1.8.2.
dvclive 0.12.1 has requirement dvc-render[table]>=0.0.12, but you have dvc-render 0.0.11.
distributed 2022.9.1 has requirement tornado<6.2,>=6.0.3, but you have tornado 6.2.
cytomine-python-client 2.5.1 has requirement Pillow<7.0.0,>=5.3.0, but you have pillow 9.3.0.
boto3 1.24.59 has requirement botocore<1.28.0,>=1.27.59, but you have botocore 1.29.0.
boto3 1.24.59 has requirement s3transfer<0.7.0,>=0.6.0, but you have s3transfer 0.5.2.
appdata 1.0.2 has requirement bidict==0.21.2, but you have bidict 0.22.0.
appdata 1.0.2 has requirement geopandas==0.10.2, but you have geopandas 0.11.1.
appdata 1.0.2 has requirement pandas<1.4,>=1.0, but you have pandas 1.5.1.
appdata 1.0.2 has requirement shapely==1.8.0, but you have shapely 1.8.2.
appdata 1.0.2 has requirement tables==4.3.1, but you have tables 4.3.2.
aiobotocore 2.4.0 has requirement botocore<1.27.60,>=1.27.59, but you have botocore 1.29.0.

efiop commented 1 year ago

@danielbellsa here is the problem I was talking about. You've installed packages with conflicting dependencies.

aiobotocore 2.4.0 has requirement botocore<1.27.60,>=1.27.59, but you have botocore 1.29.0.

danielbellsa commented 1 year ago

Solution

conda install botocore==1.27.59
dvc push

iterative / dvc

unexpected error - _register_s3_control_events() takes 2 positional arguments but 6 were given #8513

Solution