Building issues `pycapnp v1.3.0` vs `capnproto v0.10.3`

[sfavazza]

Hello there,

I am evaluating pycapnp in a docker container but I cannot successfully build the generated code for the calculator example.

Steps to reproduce:

• build a test image with the following Dockerfile:

  
  FROM python:3.8
  ARG USER=cooluser
  ARG HOME=/home/${USER}
  RUN useradd -ms /bin/bash ${USER}

requirements for cap'n proto

RUN apt-get update && apt-get install -y \ autoconf \ automake \ libtool \ pkg-config \ cmake

install "cap'n proto" from source (on master specific commit)

ARG TRGT_TAG="v0.10.3" ARG BUILD_DIR=/opt/capnproto RUN git clone https://github.com/capnproto/capnproto.git ${BUILD_DIR} \ && cd ${BUILD_DIR} \ && git checkout ${TRGT_TAG} \ && cd c++ \ && autoreconf -i \ && ./configure \ && make -j6 \ && make install

RUN mkdir /data && chown ${USER}:${USER} /data

USER ${USER}

ENV PATH=${HOME}/.local/bin:$PATH

RUN cd ${HOME} \ && pip install virtualenv \ && virtualenv .pyvenv \ && pip install \ jinja2 \ 'cython<3'

ENV PATH=${HOME}/.pyvenv/bin:$PATH

WORKDIR /data ENTRYPOINT [ "bash" ]

- mkdir ~/pycapnp-build-test
- docker run --rm -ti -v  ~/pycapnp-build-test:/data 
- from within the docker shell:
```shell
git clone https://github.com/capnproto/pycapnp.git .
pip install .
cd examples
capnpc -oc++ calculator.capnp
capnpc -ocython calculator.capnp
python setup_capnp.py build_ext --inplace

I got the following error:

Warning: passing language='c++' to cythonize() is deprecated. Instead, put "# distutils: language=c++" in your .pyx or .pxd file(s)
Compiling calculator_capnp_cython.pyx because it changed.
[1/1] Cythonizing calculator_capnp_cython.pyx
/home/cooluser/.pyvenv/lib/python3.8/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /data/examples/calculator_capnp_cython.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)

Error compiling Cython file:
------------------------------------------------------------
...
        def __set__(self, value):
            self._set_literal(value)
    cpdef _get_previousResult(self):
        return self.thisptr_child.getPreviousResult()

    cpdef _set_previousResult(self, interface_t value):
                                   ^
------------------------------------------------------------

calculator_capnp_cython.pyx:174:36: 'interface_t' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        self.thisptr_child = (<C_DynamicStruct_Builder>struct.thisptr).asCalculator_Expression_call()

    cpdef _get_function(self):
        return self.thisptr_child.getFunction()

    cpdef _set_function(self, interface_t value):
                             ^
------------------------------------------------------------

calculator_capnp_cython.pyx:324:30: 'interface_t' is not a type identifier
Traceback (most recent call last):
  File "setup_capnp.py", line 37, in <module>
    ext_modules=cythonize('*_capnp_cython.pyx', language="c++")
  File "/home/cooluser/.pyvenv/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1115, in cythonize
    cythonize_one(*args)
  File "/home/cooluser/.pyvenv/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1238, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: calculator_capnp_cython.pyx

I tried with the cap'n proto library version v0.10.3 because of the v1.3.0 (2023-01-26) changelog, though trying with more recent cap'n proto versions (namely v1.0.0 and v1.0.1) gives the same result.

Any idea where the interface_t type is defined?

[tobiasah]

Have you tried the latest beta release (2.0.0.b1)?

[LasseBlaauwbroek]

You are trying to use an incomplete attempt at implementing a capnp->cython compiler. This code was never completed and has never worked.

[sfavazza]

Have you tried the latest beta release (2.0.0.b1)?

I get 1648 lines on the stderr, for brevity I report the first chunk and the last one:

/pyvenv/lib/python3.8/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /data/modules/pycapnp/examples/calculator_capnp_cython.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)

Error compiling Cython file:
------------------------------------------------------------
...

import capnp
import calculator_capnp

from capnp.includes.types cimport *
^
------------------------------------------------------------

calculator_capnp_cython.pyx:22:0: 'capnp/includes/types.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...

import capnp
import calculator_capnp

from capnp.includes.types cimport *
from capnp cimport helpers
^
------------------------------------------------------------

calculator_capnp_cython.pyx:23:0: 'capnp.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...

import capnp
import calculator_capnp

from capnp.includes.types cimport *
from capnp cimport helpers
^
------------------------------------------------------------

calculator_capnp_cython.pyx:23:0: 'capnp/helpers.pxd' not found

Error compiling Cython file:
------------------------------------------------------------
...
import capnp
import calculator_capnp

from capnp.includes.types cimport *
from capnp cimport helpers
from capnp.includes.capnp_cpp cimport DynamicValue, Schema, VOID, StringPtr, ArrayPtr, Data
^
------------------------------------------------------------

calculator_capnp_cython.pyx:24:0: 'capnp/includes/capnp_cpp.pxd' not found

Last chunk:

Error compiling Cython file:
------------------------------------------------------------
...
            return self._get_function()
        def __set__(self, value):
            self._set_function(value)
    cpdef _get_params(self):
        cdef DynamicValue.Builder temp = self.thisptr_child.getParams()
        return to_python_builder(temp, self._parent)
              ^
------------------------------------------------------------

calculator_capnp_cython.pyx:335:15: 'to_python_builder' is not a constant, variable or function identifier

Error compiling Cython file:
------------------------------------------------------------
...

    def to_dict(self, verbose=False, ordered=False):
        ret = {

        'function': _to_dict(self.function, verbose, ordered),
                   ^
------------------------------------------------------------

calculator_capnp_cython.pyx:356:20: '_to_dict' is not a constant, variable or function identifier

Error compiling Cython file:
------------------------------------------------------------
...

        'function': _to_dict(self.function, verbose, ordered),

        'params': _to_dict(self.params, verbose, ordered),
                 ^
------------------------------------------------------------

calculator_capnp_cython.pyx:359:18: '_to_dict' is not a constant, variable or function identifier
Traceback (most recent call last):
  File "setup_capnp.py", line 37, in <module>
    ext_modules=cythonize('*_capnp_cython.pyx', language="c++")
  File "/pyvenv/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1115, in cythonize
    cythonize_one(*args)
  File "/pyvenv/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1238, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: calculator_capnp_cython.pyx

Though I just realized that the steps I performed are automatically performed in background (namely the code generation steps with capnpc), so I think the approach I reported here needs more configuration (it would explain why the compilation fails).

To run the example was enough to simply perform:

python3 async_calculator_server.py 0.0.0.0:1234  &
python3 async_calculator_client.py 0.0.0.0:1234

Feel free to close this issue!