visual-layer / fastdup

fastdup is a powerful free tool designed to rapidly extract valuable insights from your image & video datasets. Assisting you to increase your dataset images & labels quality and reduce your data operations costs at an unparalleled scale.
Other
1.51k stars 74 forks source link

[Bug]: Can't install fastdup on Mac OSX, Python 3.10.12 #295

Closed guy4261 closed 4 days ago

guy4261 commented 8 months ago

What happened?

> uname -a
Darwin Guys-MacBook-Pro-2.local 22.6.0 Darwin Kernel Version 22.6.0: Wed Jul  5 22:21:56 PDT 2023; root:xnu-8796.141.3~6/RELEASE_X86_64 x86_64

> python -V
Python 3.10.12

 > pip install fastdup
Collecting fastdup
  Obtaining dependency information for fastdup from https://files.pythonhosted.org/packages/79/5d/00383e9f153c57680e37038d022e45c4f5cf95d3bdd55a16d80bede55328/fastdup-1.44-cp310-cp310-macosx_10_14_x86_64.whl.metadata
  Using cached fastdup-1.44-cp310-cp310-macosx_10_14_x86_64.whl.metadata (1.4 kB)
Requirement already satisfied: pandas in /Users/guyr/miniconda3/envs/wwjs/lib/python3.10/site-packages (from fastdup) (1.5.3)
Collecting numpy==1.23 (from fastdup)
  Using cached numpy-1.23.0-cp310-cp310-macosx_10_9_x86_64.whl (18.1 MB)
Requirement already satisfied: tqdm in /Users/guyr/miniconda3/envs/wwjs/lib/python3.10/site-packages (from fastdup) (4.65.0)
Requirement already satisfied: pyyaml in /Users/guyr/miniconda3/envs/wwjs/lib/python3.10/site-packages (from fastdup) (6.0.1)
Collecting pillow-heif (from fastdup)
  Obtaining dependency information for pillow-heif from https://files.pythonhosted.org/packages/3d/4d/ef01c601745206a93bd3a166dad89c555850667ee721189659fa1e7027e0/pillow_heif-0.13.1-cp310-cp310-macosx_10_9_x86_64.whl.metadata
  Using cached pillow_heif-0.13.1-cp310-cp310-macosx_10_9_x86_64.whl.metadata (9.3 kB)
Collecting requests==2.28.1 (from fastdup)
  Using cached requests-2.28.1-py3-none-any.whl (62 kB)
Requirement already satisfied: packaging in /Users/guyr/miniconda3/envs/wwjs/lib/python3.10/site-packages (from fastdup) (23.1)
Collecting sentry-sdk (from fastdup)
  Obtaining dependency information for sentry-sdk from https://files.pythonhosted.org/packages/63/25/d22e1e152e4eac10d39d9132d7b5f1ea4bdfa0b9a1d65fc606a7b90aeefb/sentry_sdk-1.32.0-py2.py3-none-any.whl.metadata
  Using cached sentry_sdk-1.32.0-py2.py3-none-any.whl.metadata (9.8 kB)
INFO: pip is looking at multiple versions of fastdup to determine which version is compatible with other requirements. This could take a while.
Collecting fastdup
  Obtaining dependency information for fastdup from https://files.pythonhosted.org/packages/aa/98/698f8dbced46bb7550cb8c2889d94e5e6789f9ebb1f1c4f824103f29d451/fastdup-1.43-cp310-cp310-macosx_10_14_x86_64.whl.metadata
  Using cached fastdup-1.43-cp310-cp310-macosx_10_14_x86_64.whl.metadata (1.4 kB)
  Obtaining dependency information for fastdup from https://files.pythonhosted.org/packages/06/e4/4369e4667ab5fabf89e2ce4903134db1bbec2716c3d40a3a92e2bd01f456/fastdup-1.42-cp310-cp310-macosx_10_14_x86_64.whl.metadata
  Using cached fastdup-1.42-cp310-cp310-macosx_10_14_x86_64.whl.metadata (1.4 kB)
  Obtaining dependency information for fastdup from https://files.pythonhosted.org/packages/68/2b/f2447452e25cb7906d4de3a7d60bd5795417e057e68db60fe681ec69628d/fastdup-1.41-cp310-cp310-macosx_10_14_x86_64.whl.metadata
  Using cached fastdup-1.41-cp310-cp310-macosx_10_14_x86_64.whl.metadata (1.4 kB)
  Obtaining dependency information for fastdup from https://files.pythonhosted.org/packages/36/b7/42eb54990e0c872b44832806750087f4d5c8467d47514e857368e7cd30ff/fastdup-1.39-cp310-cp310-macosx_10_14_x86_64.whl.metadata
  Using cached fastdup-1.39-cp310-cp310-macosx_10_14_x86_64.whl.metadata (1.4 kB)
ERROR: Cannot install fastdup==1.39, fastdup==1.41, fastdup==1.42, fastdup==1.43 and fastdup==1.44 because these package versions have conflicting dependencies.

The conflict is caused by:
    fastdup 1.44 depends on opencv-python-headless==4.1.2.30
    fastdup 1.43 depends on opencv-python-headless==4.1.2.30
    fastdup 1.42 depends on opencv-python-headless==4.1.2.30
    fastdup 1.41 depends on opencv-python-headless==4.1.2.30
    fastdup 1.39 depends on opencv-python-headless==4.1.2.30

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

What did you expect to see?

fastdup installed :(

What version of fastdup were you runnning on?

I was installing it

What version of Python were you running on?

Python 3.10

Operating System

Mac OS Ventura 13.5.2 (22G91)

Reproduction steps

As in the preview. I also retried after pip uninstall opencv-python - to no avail :(

Relevant log output

No response

Attach a screenshot [Optional]

No response

Contact Details [Optional]

guy4261@gmail.com

guy4261 commented 8 months ago

Same for trying on a fresh Python 3.9.18 venv created using miniconda:

> pip install fastdup                                                                                                                                                                                                                                                                                                                                                                                                 py fastdup 20:09:27
Collecting fastdup
  Downloading fastdup-1.44-cp39-cp39-macosx_10_14_x86_64.whl.metadata (1.4 kB)
Collecting pandas (from fastdup)
  Downloading pandas-2.1.1-cp39-cp39-macosx_10_9_x86_64.whl.metadata (18 kB)
Collecting numpy==1.23 (from fastdup)
  Downloading numpy-1.23.0-cp39-cp39-macosx_10_9_x86_64.whl (18.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.1/18.1 MB 27.6 MB/s eta 0:00:00
Collecting tqdm (from fastdup)
  Downloading tqdm-4.66.1-py3-none-any.whl.metadata (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.6/57.6 kB 2.1 MB/s eta 0:00:00
Collecting pyyaml (from fastdup)
  Downloading PyYAML-6.0.1-cp39-cp39-macosx_10_9_x86_64.whl.metadata (2.1 kB)
Collecting pillow-heif (from fastdup)
  Downloading pillow_heif-0.13.1-cp39-cp39-macosx_10_9_x86_64.whl.metadata (9.3 kB)
Collecting requests==2.28.1 (from fastdup)
  Downloading requests-2.28.1-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 2.2 MB/s eta 0:00:00
Collecting packaging (from fastdup)
  Downloading packaging-23.2-py3-none-any.whl.metadata (3.2 kB)
Collecting sentry-sdk (from fastdup)
  Downloading sentry_sdk-1.32.0-py2.py3-none-any.whl.metadata (9.8 kB)
INFO: pip is looking at multiple versions of fastdup to determine which version is compatible with other requirements. This could take a while.
Collecting fastdup
  Downloading fastdup-1.43-cp39-cp39-macosx_10_14_x86_64.whl.metadata (1.4 kB)
  Downloading fastdup-1.42-cp39-cp39-macosx_10_14_x86_64.whl.metadata (1.4 kB)
  Downloading fastdup-1.41-cp39-cp39-macosx_10_14_x86_64.whl.metadata (1.4 kB)
  Downloading fastdup-1.39-cp39-cp39-macosx_10_14_x86_64.whl.metadata (1.4 kB)
ERROR: Cannot install fastdup==1.39, fastdup==1.41, fastdup==1.42, fastdup==1.43 and fastdup==1.44 because these package versions have conflicting dependencies.

The conflict is caused by:
    fastdup 1.44 depends on opencv-python-headless==4.1.2.30
    fastdup 1.43 depends on opencv-python-headless==4.1.2.30
    fastdup 1.42 depends on opencv-python-headless==4.1.2.30
    fastdup 1.41 depends on opencv-python-headless==4.1.2.30
    fastdup 1.39 depends on opencv-python-headless==4.1.2.30

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
WARNING: There was an error checking the latest version of pip.
dnth commented 8 months ago

@guy4261 can you please try this?

pip install fastdup --no-deps
pip install pandas numpy opencv-python-headless packaging pillow pillow-heif pyyaml requests==2.28.1 sentry-sdk tqdm
kmandeville commented 4 days ago

I'm on Mac OS Sonoma 14.5. I have pip installed (I'm new to python I'm a Java dev) and the command pip install fastdup returns this error: ERROR: Could not find a version that satisfies the requirement fastdup (from versions: none) ERROR: No matching distribution found for fastdup

What am I missing? If this is the only step in the instructions aside from installing python then why doesn't it work?

dnth commented 4 days ago

Hello @kmandeville there's nothing wrong on your side! This output means that pip is unable to find a version of fastdup that fits your hardware and software configurations.

What is the output on your terminal if you run

uname -a

and

python -V

dbickson commented 4 days ago

Hi @kmandeville please also share the output of pip3 debug --verbose

kmandeville commented 4 days ago

uname -a (redacted machine name): Darwin *** 23.5.0 Darwin Kernel Version 23.5.0: Wed May 1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000 arm64

python: command not found

python3: Python 3.12.4

pip3 debug --verbose: WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice. pip version: pip 24.1.1 from /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip (python 3.12) sys.version: 3.12.4 (v3.12.4:8e8a4baf65, Jun 6 2024, 17:33:18) [Clang 13.0.0 (clang-1300.0.29.30)] sys.executable: /Library/Frameworks/Python.framework/Versions/3.12/bin/python3.12 sys.getdefaultencoding: utf-8 sys.getfilesystemencoding: utf-8 locale.getpreferredencoding: UTF-8 sys.platform: darwin sys.implementation: name: cpython 'cert' config value: Not specified REQUESTS_CA_BUNDLE: None CURL_CA_BUNDLE: None pip._vendor.certifi.where(): /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/certifi/cacert.pem pip._vendor.DEBUNDLED: False vendored library versions: CacheControl==0.14.0 distlib==0.3.8 distro==1.9.0 msgpack==1.0.8 packaging==24.1 platformdirs==4.2.1 pyproject-hooks==1.0.0 requests==2.32.3 certifi==2024.02.02 idna==3.7 urllib3==1.26.18 rich==13.7.1 (Unable to locate actual module version, using vendor.txt specified version) pygments==2.17.2 typing_extensions==4.11.0 (Unable to locate actual module version, using vendor.txt specified version) resolvelib==1.0.1 setuptools==69.5.1 (Unable to locate actual module version, using vendor.txt specified version) tenacity==8.2.3 (Unable to locate actual module version, using vendor.txt specified version) tomli==2.0.1 truststore==0.9.1 Compatible tags: 582 cp312-cp312-macosx_14_0_arm64 cp312-cp312-macosx_14_0_universal2 cp312-cp312-macosx_13_0_arm64 cp312-cp312-macosx_13_0_universal2 cp312-cp312-macosx_12_0_arm64 cp312-cp312-macosx_12_0_universal2 cp312-cp312-macosx_11_0_arm64 cp312-cp312-macosx_11_0_universal2 cp312-cp312-macosx_10_16_universal2 cp312-cp312-macosx_10_15_universal2 cp312-cp312-macosx_10_14_universal2 cp312-cp312-macosx_10_13_universal2 cp312-cp312-macosx_10_12_universal2 cp312-cp312-macosx_10_11_universal2 cp312-cp312-macosx_10_10_universal2 cp312-cp312-macosx_10_9_universal2 cp312-cp312-macosx_10_8_universal2 cp312-cp312-macosx_10_7_universal2 cp312-cp312-macosx_10_6_universal2 cp312-cp312-macosx_10_5_universal2 cp312-cp312-macosx_10_4_universal2 cp312-abi3-macosx_14_0_arm64 cp312-abi3-macosx_14_0_universal2 cp312-abi3-macosx_13_0_arm64 cp312-abi3-macosx_13_0_universal2 cp312-abi3-macosx_12_0_arm64 cp312-abi3-macosx_12_0_universal2 cp312-abi3-macosx_11_0_arm64 cp312-abi3-macosx_11_0_universal2 cp312-abi3-macosx_10_16_universal2 cp312-abi3-macosx_10_15_universal2 cp312-abi3-macosx_10_14_universal2 cp312-abi3-macosx_10_13_universal2 cp312-abi3-macosx_10_12_universal2 cp312-abi3-macosx_10_11_universal2 cp312-abi3-macosx_10_10_universal2 cp312-abi3-macosx_10_9_universal2 cp312-abi3-macosx_10_8_universal2 cp312-abi3-macosx_10_7_universal2 cp312-abi3-macosx_10_6_universal2 cp312-abi3-macosx_10_5_universal2 cp312-abi3-macosx_10_4_universal2 cp312-none-macosx_14_0_arm64 cp312-none-macosx_14_0_universal2 cp312-none-macosx_13_0_arm64 cp312-none-macosx_13_0_universal2 cp312-none-macosx_12_0_arm64 cp312-none-macosx_12_0_universal2 cp312-none-macosx_11_0_arm64 cp312-none-macosx_11_0_universal2 cp312-none-macosx_10_16_universal2 cp312-none-macosx_10_15_universal2 cp312-none-macosx_10_14_universal2 cp312-none-macosx_10_13_universal2 cp312-none-macosx_10_12_universal2 cp312-none-macosx_10_11_universal2 cp312-none-macosx_10_10_universal2 cp312-none-macosx_10_9_universal2 cp312-none-macosx_10_8_universal2 cp312-none-macosx_10_7_universal2 cp312-none-macosx_10_6_universal2 cp312-none-macosx_10_5_universal2 cp312-none-macosx_10_4_universal2 cp311-abi3-macosx_14_0_arm64 cp311-abi3-macosx_14_0_universal2 cp311-abi3-macosx_13_0_arm64 cp311-abi3-macosx_13_0_universal2 cp311-abi3-macosx_12_0_arm64 cp311-abi3-macosx_12_0_universal2 cp311-abi3-macosx_11_0_arm64 cp311-abi3-macosx_11_0_universal2 cp311-abi3-macosx_10_16_universal2 cp311-abi3-macosx_10_15_universal2 cp311-abi3-macosx_10_14_universal2 cp311-abi3-macosx_10_13_universal2 cp311-abi3-macosx_10_12_universal2 cp311-abi3-macosx_10_11_universal2 cp311-abi3-macosx_10_10_universal2 cp311-abi3-macosx_10_9_universal2 cp311-abi3-macosx_10_8_universal2 cp311-abi3-macosx_10_7_universal2 cp311-abi3-macosx_10_6_universal2 cp311-abi3-macosx_10_5_universal2 cp311-abi3-macosx_10_4_universal2 cp310-abi3-macosx_14_0_arm64 cp310-abi3-macosx_14_0_universal2 cp310-abi3-macosx_13_0_arm64 cp310-abi3-macosx_13_0_universal2 cp310-abi3-macosx_12_0_arm64 cp310-abi3-macosx_12_0_universal2 cp310-abi3-macosx_11_0_arm64 cp310-abi3-macosx_11_0_universal2 cp310-abi3-macosx_10_16_universal2 cp310-abi3-macosx_10_15_universal2 cp310-abi3-macosx_10_14_universal2 cp310-abi3-macosx_10_13_universal2 cp310-abi3-macosx_10_12_universal2 cp310-abi3-macosx_10_11_universal2 cp310-abi3-macosx_10_10_universal2 cp310-abi3-macosx_10_9_universal2 cp310-abi3-macosx_10_8_universal2 cp310-abi3-macosx_10_7_universal2 cp310-abi3-macosx_10_6_universal2 cp310-abi3-macosx_10_5_universal2 cp310-abi3-macosx_10_4_universal2 cp39-abi3-macosx_14_0_arm64 cp39-abi3-macosx_14_0_universal2 cp39-abi3-macosx_13_0_arm64 cp39-abi3-macosx_13_0_universal2 cp39-abi3-macosx_12_0_arm64 cp39-abi3-macosx_12_0_universal2 cp39-abi3-macosx_11_0_arm64 cp39-abi3-macosx_11_0_universal2 cp39-abi3-macosx_10_16_universal2 cp39-abi3-macosx_10_15_universal2 cp39-abi3-macosx_10_14_universal2 cp39-abi3-macosx_10_13_universal2 cp39-abi3-macosx_10_12_universal2 cp39-abi3-macosx_10_11_universal2 cp39-abi3-macosx_10_10_universal2 cp39-abi3-macosx_10_9_universal2 cp39-abi3-macosx_10_8_universal2 cp39-abi3-macosx_10_7_universal2 cp39-abi3-macosx_10_6_universal2 cp39-abi3-macosx_10_5_universal2 cp39-abi3-macosx_10_4_universal2 cp38-abi3-macosx_14_0_arm64 cp38-abi3-macosx_14_0_universal2 cp38-abi3-macosx_13_0_arm64 cp38-abi3-macosx_13_0_universal2 cp38-abi3-macosx_12_0_arm64 cp38-abi3-macosx_12_0_universal2 cp38-abi3-macosx_11_0_arm64 cp38-abi3-macosx_11_0_universal2 cp38-abi3-macosx_10_16_universal2 cp38-abi3-macosx_10_15_universal2 cp38-abi3-macosx_10_14_universal2 cp38-abi3-macosx_10_13_universal2 cp38-abi3-macosx_10_12_universal2 cp38-abi3-macosx_10_11_universal2 cp38-abi3-macosx_10_10_universal2 cp38-abi3-macosx_10_9_universal2 cp38-abi3-macosx_10_8_universal2 cp38-abi3-macosx_10_7_universal2 cp38-abi3-macosx_10_6_universal2 cp38-abi3-macosx_10_5_universal2 cp38-abi3-macosx_10_4_universal2 cp37-abi3-macosx_14_0_arm64 cp37-abi3-macosx_14_0_universal2 cp37-abi3-macosx_13_0_arm64 cp37-abi3-macosx_13_0_universal2 cp37-abi3-macosx_12_0_arm64 cp37-abi3-macosx_12_0_universal2 cp37-abi3-macosx_11_0_arm64 cp37-abi3-macosx_11_0_universal2 cp37-abi3-macosx_10_16_universal2 cp37-abi3-macosx_10_15_universal2 cp37-abi3-macosx_10_14_universal2 cp37-abi3-macosx_10_13_universal2 cp37-abi3-macosx_10_12_universal2 cp37-abi3-macosx_10_11_universal2 cp37-abi3-macosx_10_10_universal2 cp37-abi3-macosx_10_9_universal2 cp37-abi3-macosx_10_8_universal2 cp37-abi3-macosx_10_7_universal2 cp37-abi3-macosx_10_6_universal2 cp37-abi3-macosx_10_5_universal2 cp37-abi3-macosx_10_4_universal2 cp36-abi3-macosx_14_0_arm64 cp36-abi3-macosx_14_0_universal2 cp36-abi3-macosx_13_0_arm64 cp36-abi3-macosx_13_0_universal2 cp36-abi3-macosx_12_0_arm64 cp36-abi3-macosx_12_0_universal2 cp36-abi3-macosx_11_0_arm64 cp36-abi3-macosx_11_0_universal2 cp36-abi3-macosx_10_16_universal2 cp36-abi3-macosx_10_15_universal2 cp36-abi3-macosx_10_14_universal2 cp36-abi3-macosx_10_13_universal2 cp36-abi3-macosx_10_12_universal2 cp36-abi3-macosx_10_11_universal2 cp36-abi3-macosx_10_10_universal2 cp36-abi3-macosx_10_9_universal2 cp36-abi3-macosx_10_8_universal2 cp36-abi3-macosx_10_7_universal2 cp36-abi3-macosx_10_6_universal2 cp36-abi3-macosx_10_5_universal2 cp36-abi3-macosx_10_4_universal2 cp35-abi3-macosx_14_0_arm64 cp35-abi3-macosx_14_0_universal2 cp35-abi3-macosx_13_0_arm64 cp35-abi3-macosx_13_0_universal2 cp35-abi3-macosx_12_0_arm64 cp35-abi3-macosx_12_0_universal2 cp35-abi3-macosx_11_0_arm64 cp35-abi3-macosx_11_0_universal2 cp35-abi3-macosx_10_16_universal2 cp35-abi3-macosx_10_15_universal2 cp35-abi3-macosx_10_14_universal2 cp35-abi3-macosx_10_13_universal2 cp35-abi3-macosx_10_12_universal2 cp35-abi3-macosx_10_11_universal2 cp35-abi3-macosx_10_10_universal2 cp35-abi3-macosx_10_9_universal2 cp35-abi3-macosx_10_8_universal2 cp35-abi3-macosx_10_7_universal2 cp35-abi3-macosx_10_6_universal2 cp35-abi3-macosx_10_5_universal2 cp35-abi3-macosx_10_4_universal2 cp34-abi3-macosx_14_0_arm64 cp34-abi3-macosx_14_0_universal2 cp34-abi3-macosx_13_0_arm64 cp34-abi3-macosx_13_0_universal2 cp34-abi3-macosx_12_0_arm64 cp34-abi3-macosx_12_0_universal2 cp34-abi3-macosx_11_0_arm64 cp34-abi3-macosx_11_0_universal2 cp34-abi3-macosx_10_16_universal2 cp34-abi3-macosx_10_15_universal2 cp34-abi3-macosx_10_14_universal2 cp34-abi3-macosx_10_13_universal2 cp34-abi3-macosx_10_12_universal2 cp34-abi3-macosx_10_11_universal2 cp34-abi3-macosx_10_10_universal2 cp34-abi3-macosx_10_9_universal2 cp34-abi3-macosx_10_8_universal2 cp34-abi3-macosx_10_7_universal2 cp34-abi3-macosx_10_6_universal2 cp34-abi3-macosx_10_5_universal2 cp34-abi3-macosx_10_4_universal2 cp33-abi3-macosx_14_0_arm64 cp33-abi3-macosx_14_0_universal2 cp33-abi3-macosx_13_0_arm64 cp33-abi3-macosx_13_0_universal2 cp33-abi3-macosx_12_0_arm64 cp33-abi3-macosx_12_0_universal2 cp33-abi3-macosx_11_0_arm64 cp33-abi3-macosx_11_0_universal2 cp33-abi3-macosx_10_16_universal2 cp33-abi3-macosx_10_15_universal2 cp33-abi3-macosx_10_14_universal2 cp33-abi3-macosx_10_13_universal2 cp33-abi3-macosx_10_12_universal2 cp33-abi3-macosx_10_11_universal2 cp33-abi3-macosx_10_10_universal2 cp33-abi3-macosx_10_9_universal2 cp33-abi3-macosx_10_8_universal2 cp33-abi3-macosx_10_7_universal2 cp33-abi3-macosx_10_6_universal2 cp33-abi3-macosx_10_5_universal2 cp33-abi3-macosx_10_4_universal2 cp32-abi3-macosx_14_0_arm64 cp32-abi3-macosx_14_0_universal2 cp32-abi3-macosx_13_0_arm64 cp32-abi3-macosx_13_0_universal2 cp32-abi3-macosx_12_0_arm64 cp32-abi3-macosx_12_0_universal2 cp32-abi3-macosx_11_0_arm64 cp32-abi3-macosx_11_0_universal2 cp32-abi3-macosx_10_16_universal2 cp32-abi3-macosx_10_15_universal2 cp32-abi3-macosx_10_14_universal2 cp32-abi3-macosx_10_13_universal2 cp32-abi3-macosx_10_12_universal2 cp32-abi3-macosx_10_11_universal2 cp32-abi3-macosx_10_10_universal2 cp32-abi3-macosx_10_9_universal2 cp32-abi3-macosx_10_8_universal2 cp32-abi3-macosx_10_7_universal2 cp32-abi3-macosx_10_6_universal2 cp32-abi3-macosx_10_5_universal2 cp32-abi3-macosx_10_4_universal2 py312-none-macosx_14_0_arm64 py312-none-macosx_14_0_universal2 py312-none-macosx_13_0_arm64 py312-none-macosx_13_0_universal2 py312-none-macosx_12_0_arm64 py312-none-macosx_12_0_universal2 py312-none-macosx_11_0_arm64 py312-none-macosx_11_0_universal2 py312-none-macosx_10_16_universal2 py312-none-macosx_10_15_universal2 py312-none-macosx_10_14_universal2 py312-none-macosx_10_13_universal2 py312-none-macosx_10_12_universal2 py312-none-macosx_10_11_universal2 py312-none-macosx_10_10_universal2 py312-none-macosx_10_9_universal2 py312-none-macosx_10_8_universal2 py312-none-macosx_10_7_universal2 py312-none-macosx_10_6_universal2 py312-none-macosx_10_5_universal2 py312-none-macosx_10_4_universal2 py3-none-macosx_14_0_arm64 py3-none-macosx_14_0_universal2 py3-none-macosx_13_0_arm64 py3-none-macosx_13_0_universal2 py3-none-macosx_12_0_arm64 py3-none-macosx_12_0_universal2 py3-none-macosx_11_0_arm64 py3-none-macosx_11_0_universal2 py3-none-macosx_10_16_universal2 py3-none-macosx_10_15_universal2 py3-none-macosx_10_14_universal2 py3-none-macosx_10_13_universal2 py3-none-macosx_10_12_universal2 py3-none-macosx_10_11_universal2 py3-none-macosx_10_10_universal2 py3-none-macosx_10_9_universal2 py3-none-macosx_10_8_universal2 py3-none-macosx_10_7_universal2 py3-none-macosx_10_6_universal2 py3-none-macosx_10_5_universal2 py3-none-macosx_10_4_universal2 py311-none-macosx_14_0_arm64 py311-none-macosx_14_0_universal2 py311-none-macosx_13_0_arm64 py311-none-macosx_13_0_universal2 py311-none-macosx_12_0_arm64 py311-none-macosx_12_0_universal2 py311-none-macosx_11_0_arm64 py311-none-macosx_11_0_universal2 py311-none-macosx_10_16_universal2 py311-none-macosx_10_15_universal2 py311-none-macosx_10_14_universal2 py311-none-macosx_10_13_universal2 py311-none-macosx_10_12_universal2 py311-none-macosx_10_11_universal2 py311-none-macosx_10_10_universal2 py311-none-macosx_10_9_universal2 py311-none-macosx_10_8_universal2 py311-none-macosx_10_7_universal2 py311-none-macosx_10_6_universal2 py311-none-macosx_10_5_universal2 py311-none-macosx_10_4_universal2 py310-none-macosx_14_0_arm64 py310-none-macosx_14_0_universal2 py310-none-macosx_13_0_arm64 py310-none-macosx_13_0_universal2 py310-none-macosx_12_0_arm64 py310-none-macosx_12_0_universal2 py310-none-macosx_11_0_arm64 py310-none-macosx_11_0_universal2 py310-none-macosx_10_16_universal2 py310-none-macosx_10_15_universal2 py310-none-macosx_10_14_universal2 py310-none-macosx_10_13_universal2 py310-none-macosx_10_12_universal2 py310-none-macosx_10_11_universal2 py310-none-macosx_10_10_universal2 py310-none-macosx_10_9_universal2 py310-none-macosx_10_8_universal2 py310-none-macosx_10_7_universal2 py310-none-macosx_10_6_universal2 py310-none-macosx_10_5_universal2 py310-none-macosx_10_4_universal2 py39-none-macosx_14_0_arm64 py39-none-macosx_14_0_universal2 py39-none-macosx_13_0_arm64 py39-none-macosx_13_0_universal2 py39-none-macosx_12_0_arm64 py39-none-macosx_12_0_universal2 py39-none-macosx_11_0_arm64 py39-none-macosx_11_0_universal2 py39-none-macosx_10_16_universal2 py39-none-macosx_10_15_universal2 py39-none-macosx_10_14_universal2 py39-none-macosx_10_13_universal2 py39-none-macosx_10_12_universal2 py39-none-macosx_10_11_universal2 py39-none-macosx_10_10_universal2 py39-none-macosx_10_9_universal2 py39-none-macosx_10_8_universal2 py39-none-macosx_10_7_universal2 py39-none-macosx_10_6_universal2 py39-none-macosx_10_5_universal2 py39-none-macosx_10_4_universal2 py38-none-macosx_14_0_arm64 py38-none-macosx_14_0_universal2 py38-none-macosx_13_0_arm64 py38-none-macosx_13_0_universal2 py38-none-macosx_12_0_arm64 py38-none-macosx_12_0_universal2 py38-none-macosx_11_0_arm64 py38-none-macosx_11_0_universal2 py38-none-macosx_10_16_universal2 py38-none-macosx_10_15_universal2 py38-none-macosx_10_14_universal2 py38-none-macosx_10_13_universal2 py38-none-macosx_10_12_universal2 py38-none-macosx_10_11_universal2 py38-none-macosx_10_10_universal2 py38-none-macosx_10_9_universal2 py38-none-macosx_10_8_universal2 py38-none-macosx_10_7_universal2 py38-none-macosx_10_6_universal2 py38-none-macosx_10_5_universal2 py38-none-macosx_10_4_universal2 py37-none-macosx_14_0_arm64 py37-none-macosx_14_0_universal2 py37-none-macosx_13_0_arm64 py37-none-macosx_13_0_universal2 py37-none-macosx_12_0_arm64 py37-none-macosx_12_0_universal2 py37-none-macosx_11_0_arm64 py37-none-macosx_11_0_universal2 py37-none-macosx_10_16_universal2 py37-none-macosx_10_15_universal2 py37-none-macosx_10_14_universal2 py37-none-macosx_10_13_universal2 py37-none-macosx_10_12_universal2 py37-none-macosx_10_11_universal2 py37-none-macosx_10_10_universal2 py37-none-macosx_10_9_universal2 py37-none-macosx_10_8_universal2 py37-none-macosx_10_7_universal2 py37-none-macosx_10_6_universal2 py37-none-macosx_10_5_universal2 py37-none-macosx_10_4_universal2 py36-none-macosx_14_0_arm64 py36-none-macosx_14_0_universal2 py36-none-macosx_13_0_arm64 py36-none-macosx_13_0_universal2 py36-none-macosx_12_0_arm64 py36-none-macosx_12_0_universal2 py36-none-macosx_11_0_arm64 py36-none-macosx_11_0_universal2 py36-none-macosx_10_16_universal2 py36-none-macosx_10_15_universal2 py36-none-macosx_10_14_universal2 py36-none-macosx_10_13_universal2 py36-none-macosx_10_12_universal2 py36-none-macosx_10_11_universal2 py36-none-macosx_10_10_universal2 py36-none-macosx_10_9_universal2 py36-none-macosx_10_8_universal2 py36-none-macosx_10_7_universal2 py36-none-macosx_10_6_universal2 py36-none-macosx_10_5_universal2 py36-none-macosx_10_4_universal2 py35-none-macosx_14_0_arm64 py35-none-macosx_14_0_universal2 py35-none-macosx_13_0_arm64 py35-none-macosx_13_0_universal2 py35-none-macosx_12_0_arm64 py35-none-macosx_12_0_universal2 py35-none-macosx_11_0_arm64 py35-none-macosx_11_0_universal2 py35-none-macosx_10_16_universal2 py35-none-macosx_10_15_universal2 py35-none-macosx_10_14_universal2 py35-none-macosx_10_13_universal2 py35-none-macosx_10_12_universal2 py35-none-macosx_10_11_universal2 py35-none-macosx_10_10_universal2 py35-none-macosx_10_9_universal2 py35-none-macosx_10_8_universal2 py35-none-macosx_10_7_universal2 py35-none-macosx_10_6_universal2 py35-none-macosx_10_5_universal2 py35-none-macosx_10_4_universal2 py34-none-macosx_14_0_arm64 py34-none-macosx_14_0_universal2 py34-none-macosx_13_0_arm64 py34-none-macosx_13_0_universal2 py34-none-macosx_12_0_arm64 py34-none-macosx_12_0_universal2 py34-none-macosx_11_0_arm64 py34-none-macosx_11_0_universal2 py34-none-macosx_10_16_universal2 py34-none-macosx_10_15_universal2 py34-none-macosx_10_14_universal2 py34-none-macosx_10_13_universal2 py34-none-macosx_10_12_universal2 py34-none-macosx_10_11_universal2 py34-none-macosx_10_10_universal2 py34-none-macosx_10_9_universal2 py34-none-macosx_10_8_universal2 py34-none-macosx_10_7_universal2 py34-none-macosx_10_6_universal2 py34-none-macosx_10_5_universal2 py34-none-macosx_10_4_universal2 py33-none-macosx_14_0_arm64 py33-none-macosx_14_0_universal2 py33-none-macosx_13_0_arm64 py33-none-macosx_13_0_universal2 py33-none-macosx_12_0_arm64 py33-none-macosx_12_0_universal2 py33-none-macosx_11_0_arm64 py33-none-macosx_11_0_universal2 py33-none-macosx_10_16_universal2 py33-none-macosx_10_15_universal2 py33-none-macosx_10_14_universal2 py33-none-macosx_10_13_universal2 py33-none-macosx_10_12_universal2 py33-none-macosx_10_11_universal2 py33-none-macosx_10_10_universal2 py33-none-macosx_10_9_universal2 py33-none-macosx_10_8_universal2 py33-none-macosx_10_7_universal2 py33-none-macosx_10_6_universal2 py33-none-macosx_10_5_universal2 py33-none-macosx_10_4_universal2 py32-none-macosx_14_0_arm64 py32-none-macosx_14_0_universal2 py32-none-macosx_13_0_arm64 py32-none-macosx_13_0_universal2 py32-none-macosx_12_0_arm64 py32-none-macosx_12_0_universal2 py32-none-macosx_11_0_arm64 py32-none-macosx_11_0_universal2 py32-none-macosx_10_16_universal2 py32-none-macosx_10_15_universal2 py32-none-macosx_10_14_universal2 py32-none-macosx_10_13_universal2 py32-none-macosx_10_12_universal2 py32-none-macosx_10_11_universal2 py32-none-macosx_10_10_universal2 py32-none-macosx_10_9_universal2 py32-none-macosx_10_8_universal2 py32-none-macosx_10_7_universal2 py32-none-macosx_10_6_universal2 py32-none-macosx_10_5_universal2 py32-none-macosx_10_4_universal2 py31-none-macosx_14_0_arm64 py31-none-macosx_14_0_universal2 py31-none-macosx_13_0_arm64 py31-none-macosx_13_0_universal2 py31-none-macosx_12_0_arm64 py31-none-macosx_12_0_universal2 py31-none-macosx_11_0_arm64 py31-none-macosx_11_0_universal2 py31-none-macosx_10_16_universal2 py31-none-macosx_10_15_universal2 py31-none-macosx_10_14_universal2 py31-none-macosx_10_13_universal2 py31-none-macosx_10_12_universal2 py31-none-macosx_10_11_universal2 py31-none-macosx_10_10_universal2 py31-none-macosx_10_9_universal2 py31-none-macosx_10_8_universal2 py31-none-macosx_10_7_universal2 py31-none-macosx_10_6_universal2 py31-none-macosx_10_5_universal2 py31-none-macosx_10_4_universal2 py30-none-macosx_14_0_arm64 py30-none-macosx_14_0_universal2 py30-none-macosx_13_0_arm64 py30-none-macosx_13_0_universal2 py30-none-macosx_12_0_arm64 py30-none-macosx_12_0_universal2 py30-none-macosx_11_0_arm64 py30-none-macosx_11_0_universal2 py30-none-macosx_10_16_universal2 py30-none-macosx_10_15_universal2 py30-none-macosx_10_14_universal2 py30-none-macosx_10_13_universal2 py30-none-macosx_10_12_universal2 py30-none-macosx_10_11_universal2 py30-none-macosx_10_10_universal2 py30-none-macosx_10_9_universal2 py30-none-macosx_10_8_universal2 py30-none-macosx_10_7_universal2 py30-none-macosx_10_6_universal2 py30-none-macosx_10_5_universal2 py30-none-macosx_10_4_universal2 cp312-none-any py312-none-any py3-none-any py311-none-any py310-none-any py39-none-any py38-none-any py37-none-any py36-none-any py35-none-any py34-none-any py33-none-any py32-none-any py31-none-any py30-none-any

dbickson commented 4 days ago

Hi @kmandeville you have a non-standard configuration, python 3.12 macosx 10 with arm64 processor, we support macosx 10 and up on intel processor and macosx 12+ with arm processor. My suggestion is to upgrade your OS, otherwise we can not support this configuration at this point.

guy4261 commented 4 days ago

@dbickson

https://github.com/visual-layer/fastdup/?tab=readme-ov-file#getting-started

says:

More installation options are available here.

Perhaps add here these 3 commands as they can clarify future issue reports:

uname -a
python -V
pip3 debug --verbose
kmandeville commented 4 days ago

Hi @kmandeville you have a non-standard configuration, python 3.12 macosx 10 with arm64 processor, we support macosx 10 and up on intel processor and macosx 12+ with arm processor. My suggestion is to upgrade your OS, otherwise we can not support this configuration at this point.

But I have the very latest Mac OS, Sonoma:

Screenshot 2024-07-01 at 10 12 31 AM

dbickson commented 4 days ago

Apologies I misread the pip info statemnet. Did you try to pip3 install -U pip before installing.

dbickson commented 4 days ago

Maybe the issue is python3.12? try python3.11 or python3.10 or python3.9 and let us know if it works

kmandeville commented 4 days ago

So this is where I'm at currently. Forgot I got slightly further when I was able to load and run older Python. First off:

python3.9 -m pip install fastdup is working and returning a list of Requirement already satisfied

But when I open python console and type import fastdup I see this:

Python 3.9.19 (main, Mar 19 2024, 16:08:27) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin Type "help", "copyright", "credits" or "license" for more information.

import fastdup Traceback (most recent call last): File "", line 1, in File "/Users/kmandeville/PersonalProjects/fastdup/fastdup/init.py", line 53, in raise RuntimeError(f"fastdup detected your are running an old version {version} (10 versions or more vs. the latest) please upgrade fastdup") RuntimeError: fastdup detected your are running an old version 1.74 (10 versions or more vs. the latest) please upgrade fastdup`

I ran the following: python3.9 -m pip install --upgrade fastdup and it looked like it went from having 2.2 of fastdup to now having 2.5 of fastdup, but I still get the same error as above saying I'm running an old version 1.74

kmandeville commented 4 days ago

Thanks for the help BTW, I appreciate it!

dbickson commented 4 days ago

HI @kmandeville you surface an interesting problem we have recently seen but not able to reproduce. Can you try export FASTDUP_PRODUCTION=1 and then run python again and do

python3.9
import fastdup
fastdup.__version__

what is the output?

kmandeville commented 4 days ago

python3.9

Python 3.9.19 (main, Mar 19 2024, 16:08:27)
[Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import fastdup
Failed to find crashpad handler on  /Users/kmandeville/PersonalProjects/fastdup/fastdup/lib/crashpad_handler
Failed to find shared object /Users/kmandeville/PersonalProjects/fastdup/fastdup/libfastdup_shared.dylib
Current init file is on /Users/kmandeville/PersonalProjects/fastdup/fastdup/__init__.py

exits right after. Seems to be crashing (hence it looking for crashpad handler?)

dbickson commented 4 days ago

HI @kmandeville it seems there are a few colliding versions of fastdup (1.74, 2.5) on your system and a few python versions. The location of fastdup under 3.9.19 is not on the normal place under site-packages, something is wrong with the install, maybe you copied the folder manually or something? I suggest to uninstall fastdup from all python versions, and then clear the pip cache or use --no-cache-dir to try and install, when installing also do pip3 install fastdup -U --force-reinstall --no-cache-dir

kmandeville commented 4 days ago

I had initially checked out the project from github and was in that directory when trying to do these steps. I think I'm good now! I removed fastdup, got rid of the project cloned repo and tried it again and I think I'm in good shape now! Thanks so much for the hand holding!

dbickson commented 4 days ago

Our pleasure! please keep us posted for any further feedback, everything you shared is very useful.