Error with python-rocksdb while trying to build

[Maxino22]

Hello trying to setup indy-node first time I followed steps cloned and tried to use .DevContainer however I'm getting errors with at the end python-rocks db setup is not building and crushes everything. even running pip install python-rocksdb

  rocksdb/_rocksdb.pyx:2413:37: Constructing Python tuple not allowed without gil

    Error compiling Cython file:
    ------------------------------------------------------------
    ...

        def get_backup_info(self):
            cdef vector[backup.BackupInfo] backup_info

            with nogil:
                self.engine.GetBackupInfo(cython.address(backup_info))
                                                ^
    ------------------------------------------------------------

    rocksdb/_rocksdb.pyx:2413:44: Converting to Python object not allowed without gil
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-wzq_8yme/python-rocksdb/setup.py", line 21, in <module>
        setup(
      File "/usr/local/lib/python3.8/dist-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.8/distutils/command/install.py", line 589, in run
        self.run_command('build')
      File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/local/lib/python3.8/dist-packages/setuptools/command/build_ext.py", line 79, in run
        _build_ext.run(self)
      File "/usr/lib/python3.8/distutils/command/build_ext.py", line 340, in run
        self.build_extensions()
      File "/usr/lib/python3.8/distutils/command/build_ext.py", line 449, in build_extensions
        self._build_extensions_serial()
      File "/usr/lib/python3.8/distutils/command/build_ext.py", line 474, in _build_extensions_serial
        self.build_extension(ext)
      File "/usr/local/lib/python3.8/dist-packages/setuptools/command/build_ext.py", line 196, in build_extension
        _build_ext.build_extension(self, ext)
      File "/home/vscode/.local/lib/python3.8/site-packages/Cython/Distutils/build_ext.py", line 130, in build_extension
        new_ext = cythonize(
      File "/home/vscode/.local/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
        cythonize_one(*args)
      File "/home/vscode/.local/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
        raise CompileError(None, pyx_file)
    Cython.Compiler.Errors.CompileError: rocksdb/_rocksdb.pyx
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-wzq_8yme/python-rocksdb/setup.py'"'"'; __file__='"'"'/tmp/pip-install-wzq_8yme/python-rocksdb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-asiu2aol/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/vscode/.local/include/python3.8/python-rocksdb Check the logs for full command output.

Kindly assist

[smqik]

Hi, even I am getting this error when building indy-node in the dev-container. I tried building the node both in Windows (using WSL), and Ubuntu 16.04, 20.04 LTS, and I am getting the same error. The python-rocksdb package is throwing this error. I even tried building rocksdb from the source, but still faced the same issue.

[WadeBarnes]

@Maxino22, @smqik, I've tried this on Windows, and it's working for me. I selected the "Rebuild Container Without Cache" option and the devcontainer builds and launches fine. I did need to update the devcontainer.json file to the latest format. ~PR incoming~. See https://github.com/hyperledger/indy-node/pull/1824.

Would either of you be able to provide me with a complete build log?

[WadeBarnes]

Ok, I see. The issue is with the post install when it's installing and processing the pip install .[tests] command.

[Maxino22]

hello yes that was the issue and is related with python-rocksdb ?

[smqik]

I tried building the node again after updating devcontainer.json. Still getting the same error.

[WadeBarnes]

I think I found the issue. It seems to be an update to Cython causing the issue. If I pip install Cython==0.29.36 I can then pip install python-rocksdb.

I'm testing out this change to the .devcontainer\Dockerfile:

RUN pip3 install -U \
    # Required by setup.py
    setuptools==50.3.2 \
    'pyzmq==22.3.0'

to

RUN pip3 install -U \
    # Required by setup.py
    setuptools==50.3.2 \
    'pyzmq==22.3.0' \
    Cython==0.29.36

[WadeBarnes]

Ok, That is working for me. I'll add that to the PR. This is just a workaround as the Cython - python-rocksdb python package dependency is actually at the indy-plenum level. I'll enter a ticket for that.

[WadeBarnes]

The PR has been updated with the workaround.

[Maxino22]

nice I tried with the Cython version provided it works