search

dso-org / deep-symbolic-optimization

A deep learning framework for symbolic optimization.
BSD 3-Clause "New" or "Revised" License
585 stars 125 forks source link

Kernel crashes when importing tensorflow version 1.14 #70

Closed unary-code closed 10 months ago

unary-code commented 1 year ago

My understanding is that the deep-symbolic-optimization repo requires that you have tensorflow version 1.14, and the code in the deep-symbolic-optimization repo does not work if you use any version of tensorflow 2. I know this because when I used tensorflow version 2.something (I forgot the exact version but I think it was the latest version of the tensorflow) on Google Colab and I executed "pip install -e ./dso", I got some errors like tensorflow's "set_random_seed" method does not exist. Which probably happened because tensorflow 1.14 has "set_random_seed" method but tensorflow 2.something (the tensorflow version I was using) does not have a "set_random_seed" method but tensorflow 1.14 does have a "set_random_seed" method.

By the way, which place in the directory are we supposed to execute "pip install -e ./dso" from? When the current working directory is inside the outer-most "deep-symbolic-optimization" folder, right (This is where I have executing "pip install -e ./dso" from). Not when the current working directory is inside the outer-most "dso" folder (the outer-most "dso" folder itself contains a "dso" folder, so "./dso" would be a valid path, which is why I am confused).

I tried Google Colab initially, but I got that issue about "set_random_seed" method of tensorflow not found, and Google Colab only allows tensorflow 2.something now, so I gave up on Google Colab. Then I tried out Jupyter Notebook, which led to the issue of my kernel crashing when I try to import tensorflow.

Steps that I did that led to my issue: First, on the terminal on my Macbook, I made a virtual environment (using Anaconda) and installed tensorflow (version 1.14) onto that virtual environment. Then, I opened up that virtual environment on the terminal and then from the terminal, I ran "jupyter notebook". Then, on the localhost website (Jupyter Notebook's interface is a localhost website) which shows the local files, I created a ipynb file. Then, in this ipynb file, I did "!pip install tensorflow==1.14", which worked. Then, I did "import tensorflow as tf" which caused the kernel to crash. I think because I am trying to import tensorflow version 1.14, and perhaps that version is somehow not compatible with some other software (like jupyter notebook).

brendenpetersen commented 1 year ago

Yes, DSO currently only works up to TF1.14, but we will add support for Pytorch soon. We know this is a current limitation as many systems don't work on TF1 anymore.

pip install -e ./dso should be run from above the outermost dso directory. Just inside deep-symbolic-optimization.

Lastly, I'd suggest installing in a terminal, not in a jupyter notebook. I have also only tested with pip, not Anaconda, following the README. After installing that way, notebooks on top should work fine.

unary-code commented 1 year ago

Hey, thanks for the suggestions and advice. When I was on the terminal on an Anaconda virtual environment and I ran "python -m dso.run ../BasicExampleConfigFile.json", I got this error: "/opt/anaconda3/envs/dso_env/bin/python: No module named dso.run"

unary-code commented 1 year ago

When I was on the terminal on an Anaconda virtual environment and I was inside the "deep-symbolic-optimization" folder, and I ran "pip install -e ./dso", I got this error message. Any idea how to fix this error message or if this error message is the reason why I do not have a module named "dso.run"? " Running setup.py develop for dso error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [70 lines of output]
    Compiling dso/cyfunc.pyx because it depends on /opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/Cython/Includes/libc/string.pxd.
    [1/1] Cythonizing dso/cyfunc.pyx
    running develop
    running egg_info
    creating dso.egg-info
    writing dso.egg-info/PKG-INFO
    writing dependency_links to dso.egg-info/dependency_links.txt
    writing requirements to dso.egg-info/requires.txt
    writing top-level names to dso.egg-info/top_level.txt
    writing manifest file 'dso.egg-info/SOURCES.txt'
    reading manifest file 'dso.egg-info/SOURCES.txt'
    writing manifest file 'dso.egg-info/SOURCES.txt'
    running build_ext
    building 'dso.cyfunc' extension
    creating build
    creating build/temp.macosx-10.9-x86_64-cpython-37
    creating build/temp.macosx-10.9-x86_64-cpython-37/dso
    gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/anaconda3/envs/dso_env/include -arch x86_64 -I/opt/anaconda3/envs/dso_env/include -arch x86_64 -I /opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/numpy/core/include -I /Users/davidkatz/Documents/RICE RESEARCH/venv3/lib/python3.8/site-packages/numpy/core/include -I -I -I /Users/davidkatz/venv3/lib/python3.8/site-packages/numpy/core/include -I/opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/numpy/core/include -I/opt/anaconda3/envs/dso_env/include/python3.7m -I/opt/anaconda3/envs/dso_env/include/python3.7m -c dso/cyfunc.c -o build/temp.macosx-10.9-x86_64-cpython-37/dso/cyfunc.o
    clang: error: no such file or directory: 'RESEARCH/venv3/lib/python3.8/site-packages/numpy/core/include'
    /Users/davidkatz/Documents/RICE RESEARCH/deep-symbolic-optimization/dso/setup.py:5: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
    !!

            ********************************************************************************
            Requirements should be satisfied by a PEP 517 installer.
            If you are using pip, you can try `pip install --use-pep517`.
            ********************************************************************************

    !!
      dist.Distribution().fetch_build_eggs(['Cython', 'numpy'])
    /opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/davidkatz/Documents/RICE RESEARCH/deep-symbolic-optimization/dso/dso/cyfunc.pyx
      tree = Parsing.p_module(s, pxd, full_module_name)
    /opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
    !!

            ********************************************************************************
            Requirements should be satisfied by a PEP 517 installer.
            If you are using pip, you can try `pip install --use-pep517`.
            ********************************************************************************

    !!
      dist.fetch_build_eggs(dist.setup_requires)
    /opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/setuptools/dist.py:520: InformationOnly: Normalizing '1.0dev' to '1.0.dev0'
      self._validate_version(self.metadata.version)
    /opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` and ``easy_install``.
            Instead, use pypa/build, pypa/installer, pypa/build or
            other standards-based tools.

            See https://github.com/pypa/setuptools/issues/917 for details.
            ********************************************************************************

    !!
      easy_install.initialize_options(self)
    /opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
    !!

            ********************************************************************************
            Please avoid running ``setup.py`` directly.
            Instead, use pypa/build, pypa/installer, pypa/build or
            other standards-based tools.

            See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
            ********************************************************************************

    !!
      self.initialize_options()
    error: command '/usr/bin/gcc' failed with exit code 1
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.

error: subprocess-exited-with-error

× python setup.py develop did not run successfully. │ exit code: 1 ╰─> [70 lines of output] Compiling dso/cyfunc.pyx because it depends on /opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/Cython/Includes/libc/string.pxd. [1/1] Cythonizing dso/cyfunc.pyx running develop running egg_info creating dso.egg-info writing dso.egg-info/PKG-INFO writing dependency_links to dso.egg-info/dependency_links.txt writing requirements to dso.egg-info/requires.txt writing top-level names to dso.egg-info/top_level.txt writing manifest file 'dso.egg-info/SOURCES.txt' reading manifest file 'dso.egg-info/SOURCES.txt' writing manifest file 'dso.egg-info/SOURCES.txt' running build_ext building 'dso.cyfunc' extension creating build creating build/temp.macosx-10.9-x86_64-cpython-37 creating build/temp.macosx-10.9-x86_64-cpython-37/dso gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/anaconda3/envs/dso_env/include -arch x86_64 -I/opt/anaconda3/envs/dso_env/include -arch x86_64 -I /opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/numpy/core/include -I /Users/davidkatz/Documents/RICE RESEARCH/venv3/lib/python3.8/site-packages/numpy/core/include -I -I -I /Users/davidkatz/venv3/lib/python3.8/site-packages/numpy/core/include -I/opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/numpy/core/include -I/opt/anaconda3/envs/dso_env/include/python3.7m -I/opt/anaconda3/envs/dso_env/include/python3.7m -c dso/cyfunc.c -o build/temp.macosx-10.9-x86_64-cpython-37/dso/cyfunc.o clang: error: no such file or directory: 'RESEARCH/venv3/lib/python3.8/site-packages/numpy/core/include' /Users/davidkatz/Documents/RICE RESEARCH/deep-symbolic-optimization/dso/setup.py:5: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated. !!

        ********************************************************************************
        Requirements should be satisfied by a PEP 517 installer.
        If you are using pip, you can try `pip install --use-pep517`.
        ********************************************************************************

!!
  dist.Distribution().fetch_build_eggs(['Cython', 'numpy'])
/opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /Users/davidkatz/Documents/RICE RESEARCH/deep-symbolic-optimization/dso/dso/cyfunc.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)
/opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!

        ********************************************************************************
        Requirements should be satisfied by a PEP 517 installer.
        If you are using pip, you can try `pip install --use-pep517`.
        ********************************************************************************

!!
  dist.fetch_build_eggs(dist.setup_requires)
/opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/setuptools/dist.py:520: InformationOnly: Normalizing '1.0dev' to '1.0.dev0'
  self._validate_version(self.metadata.version)
/opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/setuptools/command/develop.py:40: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer, pypa/build or
        other standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  easy_install.initialize_options(self)
/opt/anaconda3/envs/dso_env/lib/python3.7/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer, pypa/build or
        other standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. "

unary-code commented 1 year ago

Meanwhile for when I did the exact commands as the Github README, I got the same error as when I used Anaconda to create a virtual environment (see below):

When I was on the terminal on a Python virtual environment ("python3 -m venv venv3 " followed by "source venv3/bin/activate ") and I was inside the "deep-symbolic-optimization" folder, and I ran "pip install -e ./dso", I got this error message. Any idea how to fix this error message or if this error message is the reason why I do not have a module named "dso.run"?

"distutils.errors.DistutilsError: Command '['/Users/[USERNAME]/Documents/[Folder Name]/deep-symbolic-optimization/venv3/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/var/folders/hk/7vbk1d0x74dcjw_wpf8mp38h0000gn/T/tmp6dg6__h1', '--quiet', 'numpy']' returned non-zero exit status 1. [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. "

brendenpetersen commented 1 year ago

Were you able to get this to work? I haven't used DSO with Anaconda, just regular Pip.