search

trueagi-io / hyperon-experimental

MeTTa programming language implementation
https://metta-lang.dev
MIT License
133 stars 44 forks source link

Rust type error when building docker image using Mac ARM #599

Closed Rune23 closed 6 months ago

Rune23 commented 6 months ago

I'm getting rust type errors when trying to build the docker file currently.

This kind of stuff:

22.20 error[E0308]: mismatched types 22.20 --> c/src/metta.rs:1892:13 22.20 | 22.20 1888 | let bytes_written = (api.path_for_name)( 22.20 | ------------------- arguments to this function are incorrect 22.20 ... 22.20 1892 | buffer.as_mut_ptr(), 22.20 | ^^^^^^^^^^^^^^^^^^^ expected *mut u8, found *mut i8 22.20 | 22.20 = note: expected raw pointer *mut u8 22.20 found raw pointer *mut i8 22.20 22.20 error[E0308]: mismatched types 22.20 --> c/src/metta.rs:1896:44 22.20 | 22.20 1896 | vec![PathBuf::from(cstr_as_str(buffer[0..=bytes_written].as_ptr()))] 22.20 | ----------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected *const u8, found *const i8 22.20 | | 22.20 | arguments to this function are incorrect 22.20 | 22.20 = note: expected raw pointer *const u8 22.20 found raw pointer *const i8 22.20 note: function defined here 22.20 --> c/src/util.rs:8:8 22.20 | 22.20 8 | pub fn cstr_as_str<'a>(s: *const c_char) -> &'a str { 22.20 | ^^^^^^^^^^^ ---------------- 22.20 22.20 For more information about this error, try rustc --explain E0308. 22.20 error: could not compile hyperonc (lib) due to 2 previous errors 22.20 make[5]: [CMakeFiles/build-hyperonc.dir/build.make:70: CMakeFiles/build-hyperonc] Error 101 22.20 make[4]: [CMakeFiles/Makefile2:164: CMakeFiles/build-hyperonc.dir/all] Error 2 22.20 make[3]: [Makefile:146: all] Error 2 22.20 make[2]: [CMakeFiles/hyperonc.dir/build.make:86: hyperonc-prefix/src/hyperonc-stamp/hyperonc-build] Error 2 22.20 make[1]: [CMakeFiles/Makefile2:86: CMakeFiles/hyperonc.dir/all] Error 2 22.21 make: [Makefile:101: all] Error 2

context:35

33 | WORKDIR ${HOME}/hyperon-experimental/build 34 | RUN cmake .. 35 | >>> RUN make 36 | RUN make check 37 |

ERROR: failed to solve: process "/bin/sh -c make" did not complete successfully: exit code: 2

vsbogd commented 6 months ago

Could you please try building using docker build --no-cache ... probably old Rust toolchain prevents docker to build correctly.

Rune23 commented 6 months ago

Unfortunately it didn't work, I still get the same error. I have a relatively fresh MacBook with OSX 13.5, I have installed python, conda and docker but pretty much nothing else, and haven't installed anything related to rust.

vsbogd commented 6 months ago

Could you please share the full log using command docker build --no-cache -t trueagi/hyperon https://raw.githubusercontent.com/trueagi-io/hyperon-experimental/main/Dockerfile 2>&1 |tee -a /tmp/log

Rune23 commented 6 months ago

docker-crash-log.txt

here it is

vsbogd commented 6 months ago

Thanks @Rune23 , looks like root cause is that C compiler for aarch64 has c_char equal to unsigned byte and on amd64 c_char equals to signed byte.

vsbogd commented 6 months ago

Could you please check that https://github.com/trueagi-io/hyperon-experimental/pull/602 fixes your issue.

Rune23 commented 6 months ago

I checked it and I think I applied the PR correctly, and unfortunately got the same error as before, or one that's very similar - here's the log: docker-crash-log-2.txt

vsbogd commented 6 months ago

Sorry @Rune23 , the issue repeated is probably because you used docker build ... command above and this command builds Docker image by cloning main branch of the repo each time. I have fixed this in #604 .

You can check the fix by running docker build -t trueagi/hyperon http://github.com/vsbogd/hyperon-experimental.git#fix-char locally. This command builds docker using branch from the #602.

Rune23 commented 6 months ago

It fixed my error but.... Now there's a new issue, a lookup error with the setuptools-scm file unable to find version

docker-crash-log-3.txt

vsbogd commented 6 months ago

Thanks @Rune23 , will look at it.

Rune23 commented 6 months ago

I have rented a remote server with ubuntu and am trying to install from there instead, to avoid the issues with MacBooks. Maybe it is better to wait with troubleshooting Mac until a later release, but I will be happy to help with Mac troubleshooting

Rune23 commented 6 months ago

ah... I get the same error on my remote server as well

root@ubuntu-s-2vcpu-4gb-120gb-intel-fra1-01:~# docker build -t trueagi/hyperon http://github.com/trueagi-io/hyperon-experimental.git#main
[+] Building 376.7s (25/25) FINISHED                                                                   docker:default
 => [internal] load git source http://github.com/trueagi-io/hyperon-experimental.git#main                        1.1s
 => [internal] load metadata for docker.io/library/ubuntu:22.04                                                  1.2s
 => [ 1/23] FROM docker.io/library/ubuntu:22.04@sha256:f9d633ff6640178c2d0525017174a688e2c1aef28f0a0130b26bd555  1.9s
 => => resolve docker.io/library/ubuntu:22.04@sha256:f9d633ff6640178c2d0525017174a688e2c1aef28f0a0130b26bd55544  0.0s
 => => sha256:3db8720ecbf5f5927d409cc61f9b4f7ffe23283917caaa992f847c4d83338cc1 2.30kB / 2.30kB                   0.0s
 => => sha256:01007420e9b005dc14a8c8b0f996a2ad8e0d4af6c3d01e62f123be14fe48eec7 29.54MB / 29.54MB                 0.5s
 => => sha256:f9d633ff6640178c2d0525017174a688e2c1aef28f0a0130b26bd5554491f0da 1.13kB / 1.13kB                   0.0s
 => => sha256:81bba8d1dde7fc1883b6e95cd46d6c9f4874374f2b360c8db82620b33f6b5ca1 424B / 424B                       0.0s
 => => extracting sha256:01007420e9b005dc14a8c8b0f996a2ad8e0d4af6c3d01e62f123be14fe48eec7                        1.3s
 => [ 2/23] RUN apt-get update &&     DEBIAN_FRONTEND=noninteractive     TZ=UTC     apt-get install -y sudo gi  35.3s
 => [ 3/23] RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers                                              0.2s
 => [ 4/23] RUN useradd -m -g users user                                                                         0.4s
 => [ 5/23] RUN usermod -aG sudo user                                                                            0.3s
 => [ 6/23] WORKDIR /home/user                                                                                   0.1s
 => [ 7/23] RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs > /tmp/rustup.sh                       0.3s
 => [ 8/23] RUN sh /tmp/rustup.sh -y && rm /tmp/rustup.sh                                                       16.6s
 => [ 9/23] RUN cargo install cbindgen                                                                         113.4s
 => [10/23] RUN python3 -m pip install conan==1.62 pip==23.1.2                                                   9.2s
 => [11/23] RUN conan profile new --detect default                                                               0.7s
 => [12/23] ADD --chown=user:users . /home/user/hyperon-experimental                                             0.1s
 => [13/23] WORKDIR /home/user/hyperon-experimental                                                              0.0s
 => [14/23] RUN mkdir build                                                                                      0.2s
 => [15/23] WORKDIR /home/user/hyperon-experimental/lib                                                          0.1s
 => [16/23] RUN cargo build                                                                                     20.9s
 => [17/23] RUN cargo test                                                                                      45.7s
 => [18/23] WORKDIR /home/user/hyperon-experimental/build                                                        0.0s
 => [19/23] RUN cmake ..                                                                                         0.7s
 => [20/23] RUN make                                                                                           108.0s
 => [21/23] RUN make check                                                                                       9.6s
 => [22/23] WORKDIR /home/user/hyperon-experimental                                                              0.1s
 => ERROR [23/23] RUN python3 -m pip install -e ./python[dev]                                                   10.5s
------
 > [23/23] RUN python3 -m pip install -e ./python[dev]:
0.589 Defaulting to user installation because normal site-packages is not writeable
0.718 Obtaining file:///home/user/hyperon-experimental/python
0.727   Installing build dependencies: started
9.452   Installing build dependencies: finished with status 'done'
9.455   Checking if build backend supports build_editable: started
9.777   Checking if build backend supports build_editable: finished with status 'done'
9.779   Getting requirements to build editable: started
10.12   Getting requirements to build editable: finished with status 'error'
10.13   error: subprocess-exited-with-error
10.13   
10.13   × Getting requirements to build editable did not run successfully.
10.13   │ exit code: 1
10.13   ╰─> [37 lines of output]
10.13       Traceback (most recent call last):
10.13         File "/home/user/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
10.13           main()
10.13         File "/home/user/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
10.13           json_out['return_val'] = hook(**hook_input['kwargs'])
10.13         File "/home/user/.local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 132, in get_requires_for_build_editable
10.13           return hook(config_settings)
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 447, in get_requires_for_build_editable
10.13           return self.get_requires_for_build_wheel(config_settings)
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 338, in get_requires_for_build_wheel
10.13           return self._get_build_requires(config_settings, requirements=['wheel'])
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 320, in _get_build_requires
10.13           self.run_setup()
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 335, in run_setup
10.13           exec(code, locals())
10.13         File "<string>", line 74, in <module>
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools/__init__.py", line 87, in setup
10.13           return distutils.core.setup(**attrs)
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 147, in setup
10.13           _setup_distribution = dist = klass(attrs)
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools/dist.py", line 475, in __init__
10.13           _Distribution.__init__(
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 283, in __init__
10.13           self.finalize_options()
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools/dist.py", line 899, in finalize_options
10.13           ep(self)
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools_scm/_integration/setuptools.py", line 121, in infer_version
10.13           _assign_version(dist, config)
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools_scm/_integration/setuptools.py", line 56, in _assign_version
10.13           _version_missing(config)
10.13         File "/tmp/pip-build-env-pxob8x1c/overlay/local/lib/python3.10/dist-packages/setuptools_scm/_get_version_impl.py", line 112, in _version_missing
10.13           raise LookupError(
10.13       LookupError: setuptools-scm was unable to detect version for /home/user/hyperon-experimental.
10.13       
10.13       Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.
10.13       
10.13       For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
10.13       [end of output]
10.13   
10.13   note: This error originates from a subprocess, and is likely not a problem with pip.
10.13 error: subprocess-exited-with-error
10.13 
10.13 × Getting requirements to build editable did not run successfully.
10.13 │ exit code: 1
10.13 ╰─> See above for output.
10.13 
10.13 note: This error originates from a subprocess, and is likely not a problem with pip.
10.19 
10.19 [notice] A new release of pip is available: 23.1.2 -> 24.0
10.19 [notice] To update, run: pip install --upgrade pip
------
Dockerfile:39
--------------------
  37 |     
  38 |     WORKDIR ${HOME}/hyperon-experimental
  39 | >>> RUN python3 -m pip install -e ./python[dev]
  40 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c python3 -m pip install -e ./python[dev]" did not complete successfully: exit code: 1
root@ubuntu-s-2vcpu-4gb-120gb-intel-fra1-01:~#
vsbogd commented 6 months ago

Yes, it is really strange may be the reason is that I have old Ubuntu image locally. I am trying to reproduce.

vsbogd commented 6 months ago

I pulled the latest ubuntu:22.04 docker image and still cannot reproduce it. I added additional debug log in a separate branch to investigate it. @Rune23 could you please run docker build -t trueagi/hyperon http://github.com/vsbogd/hyperon-experimental.git#fix-599 2>&1 | tee /tmp/log and attach the resulting /tmp/log file.

Rune23 commented 6 months ago

log1.txt

I just tried it on my MacBook now, and it looks like it's giving a different error this time but still related to a similar file. Will try to run on the ubuntu server tomorrow

vsbogd commented 6 months ago

Thanks @Rune23 , error is the same. setuptools_scm cannot infer package version because it depends on Git information. Fresh versions of the docker doesn't include .git directory into build context. This is breaking change in Docker BuildKit. One can override this behavior by using BUILDKIT_CONTEXT_KEEP_GIT_DIR build arg withint docker build command line.

Please use the following command to build docker:

docker build \
    --build-arg BUILDKIT_CONTEXT_KEEP_GIT_DIR=1 \
    -t trueagi/hyperon \
    http://github.com/trueagi-io/hyperon-experimental.git#main

606 is raised to fix README.md file.

Rune23 commented 6 months ago

it works!