search

lyst / lightfm

A Python implementation of LightFM, a hybrid recommendation algorithm.
Apache License 2.0
4.78k stars 692 forks source link

Light FM throwing an installation error: AttributeError: 'dict' object has no attribute '__LIGHTFM_SETUP__' #687

Open aivansky-contractor opened 1 year ago

aivansky-contractor commented 1 year ago

When installing lightFM as a Github action I am getting the following error.

pip install -r requirements.txt
[4](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:4)
  shell: /usr/bin/bash -e {0}
[5](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:5)
  env:
[6](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:6)
    PREFECT_VERSION: 2.8.2
[7](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:7)
    K8S_NAMESPACE: prefect
[8](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:8)
    WORK_QUEUE: main
[9](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:9)
    STAGING_BUCKET: endvr-prefect-storage-staging
[10](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:10)
    STAGING_SNOWFLAKE_ACCOUNT: ama75210
[11](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:11)
    PROD_BUCKET: endvr-prefect-storage-production
[12](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:12)
    PROD_SNOWFLAKE_ACCOUNT: boa33536
[13](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:13)
    DEPLOYMENT_ENV: Production
[14](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:14)
    pythonLocation: /opt/hostedtoolcache/Python/3.10.11/x64
[15](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:15)
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.10.11/x64/lib/pkgconfig
[16](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:16)
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.11/x64
[17](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:17)
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.11/x64
[18](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:18)
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.11/x64
[19](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:19)
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.10.11/x64/lib
[20](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:20)
    FLOW: streaming-recommendation-model
[21](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:22)
Collecting git+https://github.com/Endeavor-Digital/lightfm.git (from -r requirements.txt (line 5))
[22](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:23)
  Cloning https://github.com/Endeavor-Digital/lightfm.git to /tmp/pip-req-build-rz1rrscb
[23](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:24)
  Running command git clone --filter=blob:none --quiet https://github.com/Endeavor-Digital/lightfm.git /tmp/pip-req-build-rz1rrscb
[24](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:25)
  Resolved https://github.com/Endeavor-Digital/lightfm.git to commit 2e2b2ac8841922ab4f593c6dc2e9addb5fc7eb6b
[25](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:26)
  Installing build dependencies: started
[26](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:27)
  Installing build dependencies: finished with status 'done'
[27](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:28)
  Getting requirements to build wheel: started
[28](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:29)
  Getting requirements to build wheel: finished with status 'error'
[29](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:30)
  error: subprocess-exited-with-error
[30](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:31)

[31](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:32)
  × Getting requirements to build wheel did not run successfully.
[32](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:33)
  │ exit code: 1
[33](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:34)
  ╰─> [17 lines of output]
[34](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:35)
      Traceback (most recent call last):
[35](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:36)
        File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
[36](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:37)
          main()
[37](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:38)
        File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
[38](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:39)
          json_out['return_val'] = hook(**hook_input['kwargs'])
[39](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:40)
        File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
[40](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:41)
          return hook(config_settings)
[41](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:42)
        File "/tmp/pip-build-env-haxk8_c9/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
[42](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:43)
          return self._get_build_requires(config_settings, requirements=['wheel'])
[43](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:44)
        File "/tmp/pip-build-env-haxk8_c9/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in _get_build_requires
[44](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:45)
          self.run_setup()
[45](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:46)
        File "/tmp/pip-build-env-haxk8_c9/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 484, in run_setup
[46](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:47)
          super(_BuildMetaLegacyBackend,
[47](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:48)
        File "/tmp/pip-build-env-haxk8_c9/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 335, in run_setup
[48](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:49)
          exec(code, locals())
[49](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:50)
        File "<string>", line 11, in <module>
[50](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:51)
      AttributeError: 'dict' object has no attribute '__LIGHTFM_SETUP__'
[51](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:52)
      [end of output]
[52](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:53)

[53](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:54)
  note: This error originates from a subprocess, and is likely not a problem with pip.
[54](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:55)
error: subprocess-exited-with-error
[55](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:56)

[56](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:57)
× Getting requirements to build wheel did not run successfully.
[57](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:58)
│ exit code: 1
[58](https://github.com/Endeavor-Digital/prefect-2-flows/actions/runs/4735271259/jobs/8405281638#step:6:59)
╰─> See above for output.

I think this is related to the line https://github.com/Endeavor-Digital/lightfm/blob/2e2b2ac8841922ab4f593c6dc2e9addb5fc7eb6b/setup.py#L11

I think we should use workaround suggested here https://github.com/RaRe-Technologies/gensim/issues/3362

ASH1998 commented 1 year ago

I am also facing the same issue with running lightfm install with docker.

Python image : public.ecr.aws/lambda/python:3.9
lightfm==1.17
#9 10.28   × Getting requirements to build wheel did not run successfully.
#9 10.28   │ exit code: 1
#9 10.28   ╰─> [17 lines of output]
#9 10.28       Traceback (most recent call last):
#9 10.28         File "/var/lang/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
#9 10.28           main()
#9 10.28         File "/var/lang/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
#9 10.28           json_out['return_val'] = hook(**hook_input['kwargs'])
#9 10.28         File "/var/lang/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
#9 10.28           return hook(config_settings)
#9 10.28         File "/tmp/pip-build-env-u1wccerz/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
#9 10.28           return self._get_build_requires(config_settings, requirements=['wheel'])
#9 10.28         File "/tmp/pip-build-env-u1wccerz/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
#9 10.28           self.run_setup()
#9 10.28         File "/tmp/pip-build-env-u1wccerz/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 487, in run_setup
#9 10.28           super(_BuildMetaLegacyBackend,
#9 10.28         File "/tmp/pip-build-env-u1wccerz/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in run_setup
#9 10.28           exec(code, locals())
#9 10.28         File "<string>", line 11, in <module>
#9 10.28       AttributeError: 'dict' object has no attribute '__LIGHTFM_SETUP__'
#9 10.28       [end of output]

May be its time to move away from lightfm.

filipeo2-mck commented 1 year ago

+1 installing it under docker (linux/amd64 arch)

xiaodong-xie commented 1 year ago

Came across this issue, and worked around it by using this hack: 

python -m pip install --upgrade pip setuptools wheel
python -m pip install --no-use-pep517 lightfm
pip install -r requirements.txt
ThijsHakkenbergLobster commented 5 months ago

Came across this issue, and worked around it by using this hack:

python -m pip install --upgrade pip setuptools wheel
python -m pip install --no-use-pep517 lightfm
pip install -r requirements.txt

Not working for me unfortunatly: `.... ^ /Library/Frameworks/Python.framework/Versions/3.12/include/python3.12/cpython/dictobject.h:22:5: note: 'ma_version_tag' has been explicitly marked deprecated here Py_DEPRECATED(3.12) uint64_t ma_version_tag; ^ /Library/Frameworks/Python.framework/Versions/3.12/include/python3.12/pyport.h:317:54: note: expanded from macro 'Py_DEPRECATED'

define Py_DEPRECATED(VERSION_UNUSED) attribute((deprecated))

                                                       ^
  lightfm/_lightfm_fast_no_openmp.c:30261:55: error: no member named 'ob_digit' in 'struct _longobject'
              const digit* digits = ((PyLongObject*)x)->ob_digit;
                                    ~~~~~~~~~~~~~~~~~~  ^
  lightfm/_lightfm_fast_no_openmp.c:30316:55: error: no member named 'ob_digit' in 'struct _longobject'
              const digit* digits = ((PyLongObject*)x)->ob_digit;
                                    ~~~~~~~~~~~~~~~~~~  ^
  lightfm/_lightfm_fast_no_openmp.c:30457:55: error: no member named 'ob_digit' in 'struct _longobject'
              const digit* digits = ((PyLongObject*)x)->ob_digit;
                                    ~~~~~~~~~~~~~~~~~~  ^
  lightfm/_lightfm_fast_no_openmp.c:30512:55: error: no member named 'ob_digit' in 'struct _longobject'
              const digit* digits = ((PyLongObject*)x)->ob_digit;
                                    ~~~~~~~~~~~~~~~~~~  ^
  lightfm/_lightfm_fast_no_openmp.c:30691:55: error: no member named 'ob_digit' in 'struct _longobject'
              const digit* digits = ((PyLongObject*)x)->ob_digit;
                                    ~~~~~~~~~~~~~~~~~~  ^
  lightfm/_lightfm_fast_no_openmp.c:30746:55: error: no member named 'ob_digit' in 'struct _longobject'
              const digit* digits = ((PyLongObject*)x)->ob_digit;
                                    ~~~~~~~~~~~~~~~~~~  ^
  lightfm/_lightfm_fast_no_openmp.c:31097:47: error: no member named 'ob_digit' in 'struct _longobject'
      const digit* digits = ((PyLongObject*)b)->ob_digit;
                            ~~~~~~~~~~~~~~~~~~  ^
  22 warnings and 10 errors generated.
  error: command '/usr/bin/clang' failed with exit code 1
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for lightfm Running setup.py clean for lightfm Failed to build lightfm ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (lightfm) `

Ezapataq07 commented 1 month ago

Came across this issue, and worked around it by using this hack:

python -m pip install --upgrade pip setuptools wheel
python -m pip install --no-use-pep517 lightfm
pip install -r requirements.txt

I found this helpful. First I run: python -m pip install --upgrade pip setuptools wheel

And then I had to install Microsoft Visual C++ via link

Finally run: pip install lightfm

nirakar-padhy-collinson commented 1 day ago

I always facing the same issue while installing LightFM in lambda python 3.11 runtime but using python 3.10 runtime with the below Dockerfile solved the error for me

Dockerfile:

FROM public.ecr.aws/lambda/python:3.10

RUN yum install -y gcc gcc-c++ python3-devel lapack-devel blas-devel

# Install Python packages
RUN pip3 install psycopg2-binary --target "${LAMBDA_TASK_ROOT}"
RUN pip3 install redis --target "${LAMBDA_TASK_ROOT}"
RUN pip3 install numpy --target "${LAMBDA_TASK_ROOT}"
RUN pip3 install pandas --target "${LAMBDA_TASK_ROOT}"
RUN pip3 install requests --target "${LAMBDA_TASK_ROOT}"
RUN pip3 install polars --target "${LAMBDA_TASK_ROOT}"
RUN pip3 install lightfm --target "${LAMBDA_TASK_ROOT}"

COPY app.py .

CMD ["app.handler"]