nmslib / hnswlib

Header-only C++/python library for fast approximate nearest neighbors
https://github.com/nmslib/hnswlib
Apache License 2.0
4.12k stars 609 forks source link

Resolve wheel build failure on x86_64 architecture due to -march=native flag #455

Open XiChenn opened 1 year ago

XiChenn commented 1 year ago

The hnswlib wheel build failed on my MacOS (x86_64 architecture) when I run pip install hnswlib.

      clang: error: the clang compiler does not support '-march=native'
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for hnswlib
Failed to build hnswlib
ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects

Similar to the solution of the previous merged PR, by removing the building flag resolves the issue.

pip install .
Looking in indexes: https://pypi.python.org/simple/
...
Successfully built hnswlib
Installing collected packages: hnswlib
Successfully installed hnswlib-0.7.0
yurymalkov commented 1 year ago

Hm. I can imagine this would remove native optimizations from linux builds - I see it only uses Darwin Do I understand what is causing the problem? Is it due to clang?

yurymalkov commented 1 year ago

We can try to understand if the compiler is clang with something like


import sys
import platform
import subprocess
import re

def is_clang_compiler():
    try:
        # Run the compiler version command
        result = subprocess.run(['cc', '--version'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
        output = result.stdout.lower()

        # Check if the output contains the word 'clang'
        return 'clang' in output
    except FileNotFoundError:
        # If the 'cc' command is not found, it's not Clang
        return False
yurymalkov commented 1 year ago

Can you double check that your compiler is set up properly? I've added macos to the CI with x64 and the tests are passing. I am also confused that your error message has both gcc and clang mentioned, sounds fishy

XiChenn commented 1 year ago

My Mac is 2019 version. see the result below:

>>> result = subprocess.run(['cc', '--version'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
>>> output = result.stdout.lower()
>>> output
'apple clang version 14.0.3 (clang-1403.0.22.14.1)\ntarget: x86_64-apple-darwin22.4.0\nthread model: posix\ninstalleddir: /applications/xcode.app/contents/developer/toolchains/xcodedefault.xctoolchain/usr/bin\n'
dyashuni commented 1 year ago

@XiChenn Could you please try pip install . from PR https://github.com/nmslib/hnswlib/pull/461 ? it should fix your issue

yurymalkov commented 1 year ago

@XiChenn Can you check if the other change fixes your problem by running pip install git+https://github.com/nmslib/hnswlib.git@refs/pull/461/head?

cterrazas2 commented 1 year ago

fwiw, @yurymalkov I was having the same issue as @XiChenn (Mac x86_64 architecture) and installing via #461's head worked for me.