alibaba / EasyRec

A framework for large scale recommendation algorithms.
Apache License 2.0
1.76k stars 319 forks source link

Can't use EasyRec in docker or anaconda. #463

Closed EdwardSJ151 closed 5 months ago

EdwardSJ151 commented 5 months ago

I can't setup EasyRec. I have looked online for any possible solutions for my problems, but I am stuck. I would prefer to be able to run the docker image, but it's ok if it's only anaconda works. Please help me.

I tried to use the anaconda enviornment first, and I got an error with python not being above 3.9 for scikit-learn:

Searching for scikit-learn
Reading http://mirrors.aliyun.com/pypi/simple/scikit-learn/
Downloading http://mirrors.aliyun.com/pypi/packages/ef/e5/c09d20723bfd91315f6f4ddc77912b0dcc09588b4ca7ad2ffa204607ad7f/scikit-learn-1.4.2.tar.gz#sha256=daa1c471d95bad080c6e44b4946c9390a4842adc3082572c20e4f8884e39e959
Best match: scikit-learn 1.4.2
Processing scikit-learn-1.4.2.tar.gz
Writing /tmp/easy_install-byrarosl/scikit-learn-1.4.2/setup.cfg
Running scikit-learn-1.4.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-byrarosl/scikit-learn-1.4.2/egg-dist-tmp-s6nzhn75
Partial import of sklearn during the build process.
Traceback (most recent call last):
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 156, in save_modules
    yield saved
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 198, in setup_context
    yield
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 259, in run_setup
    _execfile(setup_script, ns)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 46, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-byrarosl/scikit-learn-1.4.2/setup.py", line 639, in <module>
  File "/tmp/easy_install-byrarosl/scikit-learn-1.4.2/setup.py", line 626, in setup_package
RuntimeError: Scikit-learn requires Python 3.9 or later. The current Python version is 3.6.8 installed in /home/pressprexx/anaconda3/envs/EasyRec/bin/python.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 70, in <module>
    'tests': parse_requirements('requirements/tests.txt'),
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/command/install.py", line 117, in do_egg_install
    cmd.run(show_deprecation=False)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 408, in run
    self.easy_install(spec, not self.no_deps)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 650, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 697, in install_item
    self.process_distribution(spec, dist, deps)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 745, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/pkg_resources/__init__.py", line 768, in resolve
    replace_conflicting=replace_conflicting
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1051, in best_match
    return self.obtain(req, installer)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1063, in obtain
    return installer(requirement)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 669, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 695, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 890, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1162, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 1146, in run_setup
    run_setup(setup_script, args)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 262, in run_setup
    raise
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 198, in setup_context
    yield
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 169, in save_modules
    saved_exc.resume()
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 143, in resume
    raise exc.with_traceback(self._tb)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 156, in save_modules
    yield saved
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 198, in setup_context
    yield
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 259, in run_setup
    _execfile(setup_script, ns)
  File "/home/pressprexx/anaconda3/envs/EasyRec/lib/python3.6/site-packages/setuptools/sandbox.py", line 46, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-byrarosl/scikit-learn-1.4.2/setup.py", line 639, in <module>
  File "/tmp/easy_install-byrarosl/scikit-learn-1.4.2/setup.py", line 626, in setup_package
RuntimeError: Scikit-learn requires Python 3.9 or later. The current Python version is 3.6.8 installed in /home/pressprexx/anaconda3/envs/EasyRec/bin/python.
´´´

Trying python versions 3.9 and above all don't work since I get dependency issues, but I didn't except them to work anyway since the documentation asks for python 3.6.8

Then I tried docker and I got the following issues:
Method 1 - Pull uploaded image

(EasyRec) pressprexx@titan:~/Code/Niceday/EasyRec$ docker pull mybigpai-registry.cn-beijing.cr.aliyuncs.com/easyrec/easyrec:py36-tf1.15-0.6.3 Error response from daemon: Get "https://mybigpai-registry.cn-beijing.cr.aliyuncs.com/v2/": context deadline exceeded ´´´

Method 2 - Build docker image


(EasyRec) pressprexx@titan:~/Code/Niceday/EasyRec$ bash scripts/build_docker.sh
EasyRec Version: 0.8.0
[sudo] password for pressprexx: 
[+] Building 30.3s (2/2) FINISHED                                                                                                                                                                               docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                                      0.0s
 => => transferring dockerfile: 2.17kB                                                                                                                                                                                    0.0s
 => ERROR [internal] load metadata for datascience-registry.cn-beijing.cr.aliyuncs.com/tensorflow/tensorflow:1.15.5                                                                                                      30.3s
------
 > [internal] load metadata for datascience-registry.cn-beijing.cr.aliyuncs.com/tensorflow/tensorflow:1.15.5:
------
Dockerfile:1
--------------------
   1 | >>> FROM datascience-registry.cn-beijing.cr.aliyuncs.com/tensorflow/tensorflow:1.15.5
   2 |     
   3 |     COPY docker/sources_18.04.list /etc/apt/sources.list
--------------------
ERROR: failed to solve: DeadlineExceeded: DeadlineExceeded: DeadlineExceeded: datascience-registry.cn-beijing.cr.aliyuncs.com/tensorflow/tensorflow:1.15.5: failed to resolve source metadata for datascience-registry.cn-beijing.cr.aliyuncs.com/tensorflow/tensorflow:1.15.5: failed to do request: Head "https://datascience-registry.cn-beijing.cr.aliyuncs.com/v2/tensorflow/tensorflow/manifests/1.15.5": dial tcp 47.94.99.112:443: i/o timeout
´´´