rust-lang / rustc_codegen_gcc

libgccjit AOT codegen for rustc
Apache License 2.0
905 stars 60 forks source link

SuperH: `cannot add block to an imported function` #419

Closed darcagn closed 6 months ago

darcagn commented 6 months ago

Hi, I'm attempting to get rustc_codegen_gcc to generate code for SuperH (specifically SH4) for Sega Dreamcast. I'm very interested in Rust on Dreamcast -- I am a maintainer for the KallistiOS Dreamcast toolchain and if I can get this working I'd add it to my regular testing. We've already done some interesting stuff with gccrs including a 3D cube demo calling C and Fortran functions but because it is in such early stages I'd like to see if we can use rustc_codegen_gcc instead.

Unfortunately the SuperH dockerfile that was used to generate a rustc_codegen_gcc toolchain for Casio calculators (referenced in the wiki) no longer works, its author hasn't been able to update it, and trying to alter it to get it to work ended up resulting in this same error as I'm receiving now attempting the process manually from scratch.

When I run the y.sh build step, I receive the following error. Any ideas?

libgccjit.so: error: gcc_jit_function_new_block: cannot add block to an imported function
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_param: NULL function
gcc_jit_param_as_rvalue: NULL param
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_param: NULL function
gcc_jit_param_as_rvalue: NULL param
gcc_jit_block_get_function: NULL block
gcc_jit_function_new_local: NULL function
gcc_jit_lvalue_get_address: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_get_pointer: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_unqualified: NULL type
gcc_jit_type_get_aligned: NULL type
gcc_jit_type_get_pointer: NULL type
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_block_add_assignment: NULL block
gcc_jit_block_get_function: NULL block
gcc_jit_function_new_local: NULL function
gcc_jit_lvalue_get_address: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_get_pointer: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_unqualified: NULL type
gcc_jit_type_get_aligned: NULL type
gcc_jit_type_get_pointer: NULL type
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_block_add_assignment: NULL block
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_is_integral: NULL type
gcc_jit_type_is_pointer: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_is_integral: NULL type
gcc_jit_type_is_pointer: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_block_get_function: NULL block
libgccjit.so: error: gcc_jit_context_new_call_through_ptr: NULL argument 1 to fn_ptr: _RNvNtCsfWEm02ePFrQ_17compiler_builtins4math4fmin (type: double)
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_function_new_local: NULL function
gcc_jit_block_add_assignment: NULL block
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_return_type: NULL function_type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_compatible_types: NULL ltype
libgccjit.so: error: gcc_jit_context_new_cast: NULL rvalue
gcc_jit_block_end_with_return: NULL block
libgccjit.so: error: gcc_jit_function_new_block: cannot add block to an imported function
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_param: NULL function
gcc_jit_param_as_rvalue: NULL param
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_param: NULL function
gcc_jit_param_as_rvalue: NULL param
gcc_jit_block_get_function: NULL block
gcc_jit_function_new_local: NULL function
gcc_jit_lvalue_get_address: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_get_pointer: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_unqualified: NULL type
gcc_jit_type_get_aligned: NULL type
gcc_jit_type_get_pointer: NULL type
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_block_add_assignment: NULL block
gcc_jit_block_get_function: NULL block
gcc_jit_function_new_local: NULL function
gcc_jit_lvalue_get_address: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_get_pointer: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_unqualified: NULL type
gcc_jit_type_get_aligned: NULL type
gcc_jit_type_get_pointer: NULL type
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_block_add_assignment: NULL block
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_is_integral: NULL type
gcc_jit_type_is_pointer: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_is_integral: NULL type
gcc_jit_type_is_pointer: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_block_get_function: NULL block
libgccjit.so: error: gcc_jit_context_new_call_through_ptr: NULL argument 1 to fn_ptr: _RNvNtCsfWEm02ePFrQ_17compiler_builtins4math5fminf (type: float)
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_function_new_local: NULL function
gcc_jit_block_add_assignment: NULL block
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_return_type: NULL function_type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_compatible_types: NULL ltype
libgccjit.so: error: gcc_jit_context_new_cast: NULL rvalue
gcc_jit_block_end_with_return: NULL block
libgccjit.so: error: gcc_jit_function_new_block: cannot add block to an imported function
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_param: NULL function
gcc_jit_param_as_rvalue: NULL param
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_param: NULL function
gcc_jit_param_as_rvalue: NULL param
gcc_jit_block_get_function: NULL block
gcc_jit_function_new_local: NULL function
gcc_jit_lvalue_get_address: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_get_pointer: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_unqualified: NULL type
gcc_jit_type_get_aligned: NULL type
gcc_jit_type_get_pointer: NULL type
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_block_add_assignment: NULL block
gcc_jit_block_get_function: NULL block
gcc_jit_function_new_local: NULL function
gcc_jit_lvalue_get_address: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_get_pointer: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_unqualified: NULL type
gcc_jit_type_get_aligned: NULL type
gcc_jit_type_get_pointer: NULL type
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_block_add_assignment: NULL block
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_is_integral: NULL type
gcc_jit_type_is_pointer: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_is_integral: NULL type
gcc_jit_type_is_pointer: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_block_get_function: NULL block
libgccjit.so: error: gcc_jit_context_new_call_through_ptr: NULL argument 1 to fn_ptr: _RNvNtCsfWEm02ePFrQ_17compiler_builtins4math4fmax (type: double)
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_function_new_local: NULL function
gcc_jit_block_add_assignment: NULL block
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_return_type: NULL function_type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_compatible_types: NULL ltype
libgccjit.so: error: gcc_jit_context_new_cast: NULL rvalue
gcc_jit_block_end_with_return: NULL block
libgccjit.so: error: gcc_jit_function_new_block: cannot add block to an imported function
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_param: NULL function
gcc_jit_param_as_rvalue: NULL param
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_param: NULL function
gcc_jit_param_as_rvalue: NULL param
gcc_jit_block_get_function: NULL block
gcc_jit_function_new_local: NULL function
gcc_jit_lvalue_get_address: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_get_pointer: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_unqualified: NULL type
gcc_jit_type_get_aligned: NULL type
gcc_jit_type_get_pointer: NULL type
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_block_add_assignment: NULL block
gcc_jit_block_get_function: NULL block
gcc_jit_function_new_local: NULL function
gcc_jit_lvalue_get_address: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_get_pointer: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_unqualified: NULL type
gcc_jit_type_get_aligned: NULL type
gcc_jit_type_get_pointer: NULL type
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_dereference: NULL rvalue
gcc_jit_block_add_assignment: NULL block
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_is_integral: NULL type
gcc_jit_type_is_pointer: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_is_integral: NULL type
gcc_jit_type_is_pointer: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_type_dyncast_vector: NULL type
gcc_jit_type_unqualified: NULL type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_rvalue_get_type: NULL rvalue
libgccjit.so: error: gcc_jit_context_new_bitcast: NULL rvalue
gcc_jit_block_get_function: NULL block
libgccjit.so: error: gcc_jit_context_new_call_through_ptr: NULL argument 1 to fn_ptr: _RNvNtCsfWEm02ePFrQ_17compiler_builtins4math5fmaxf (type: float)
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_function_new_local: NULL function
gcc_jit_block_add_assignment: NULL block
gcc_jit_lvalue_as_rvalue: NULL lvalue
gcc_jit_block_get_function: NULL block
gcc_jit_function_get_return_type: NULL function_type
gcc_jit_rvalue_get_type: NULL rvalue
gcc_jit_compatible_types: NULL ltype
libgccjit.so: error: gcc_jit_context_new_cast: NULL rvalue
gcc_jit_block_end_with_return: NULL block
antoyo commented 6 months ago

Ok, I'll attempt to do this tomorrow. In the meantime, please follow the instructions here as the instructions linked from the wiki are outdated.

I opened a PR to build a libgccjit cross-compiler for SuperH. In case you don't want to build a cross-compiling libgccjit yourself (although it's easy with crosstool-ng as is used here), you can wait that this PR is merged and you'll be able to just download a Debian package with the binaries.

darcagn commented 6 months ago

I very much appreciate you looking into this. To clarify, the error above happens in all 3 scenarios: 1) following along with the old Dockerfile, 2) when I build an SH4 toolchain with our Sega Dreamcast toolchain building scripts but from your git repo's GCC sources, and 3) when using a toolchain built with crosstool-ng.

antoyo commented 6 months ago

Could you please share your crosstool-ng defconfig?

Edit: A JSON target spec would also be useful, if you have one.

Edit 2: Ok, I'm able to reproduce the issue locally. I'd still appreciate those files just to make sure.

Edit 3: I'm not done investigating, but here are my results so far. This seems to be caused by the fact that some builtin functions are defined in this case. We can go further in the sysroot build by removing "fminf", "fmin", "fmaxf", "fmax" from here.

But then, I ran into this error which I'm not sure what to do about:

error[E0658]: use of unstable library feature 'restricted_std'
   --> sysroot_src/library/test/src/lib.rs:129:25
    |
129 |                         std::mem::forget(std::io::stderr().lock());
    |                         ^^^^^^^^^^^^^^^^
    |
    = help: add `#![feature(restricted_std)]` to the crate attributes to enable
darcagn commented 6 months ago

Here's what I'm using for defconfig:

CT_CONFIG_VERSION="4"
CT_EXPERIMENTAL=y
CT_PREFIX_DIR="${CT_PREFIX:-${HOME}/x-tools}/${CT_HOST:+HOST-${CT_HOST}/}${CT_TARGET}/usr"
CT_DOWNLOAD_AGENT_CURL=y
CT_ARCH_CPU="sh4"
CT_ARCH_SH=y
CT_ARCH_LE=y
CT_KERNEL_BARE_METAL=y
CT_BINUTILS_V_2_40=y
CT_BINUTILS_PLUGINS=y
CT_GCC_SRC_DEVEL=y
CT_GCC_DEVEL_URL="https://github.com/antoyo/gcc.git"
CT_CC_GCC_EXTRA_CONFIG_ARRAY="--enable-host-shared --disable-bootstrap"
CT_CC_GCC_MULTILIB_LIST="m4,m4-single,m4-single-only"
CT_CC_GCC_USE_LTO=n
CT_CC_LANG_JIT=y
CT_OMIT_TARGET_VENDOR=y
CT_PATCH_BUNDLED_LOCAL=y
CT_LOCAL_PATCH_DIR="/home/darc/rustgcc/patches"

(in that patch dir I have this patch required for default-jit.o to compile)

and my JSON target:

{
  "arch": "mips",
  "data-layout": "e-m:m-p:32:32-i8:8:32-i16:16:32-i64:64-n32-S64",
  "env": "gnu",
  "has-rpath": true,
  "llvm-target": "mips-unknown-linux-gnu",
  "max-atomic-width": 32,
  "position-independent-executables": false,
  "panic-strategy": "abort",
  "linker-flavor": "gcc",
  "linker": "-Clinker=sh-elf-gcc",
  "split-debuginfo": "off",
  "target-endian": "little",
  "late-link-args": {
    "gcc": ["-ffreestanding", "-nostdlib"]
  },
  "target-pointer-width": "32"
}

I can confirm on my end that removing "fminf", "fmin", "fmaxf", "fmax" from context.rs also leads me to the same restricted_std errors.

bjorn3 commented 6 months ago

But then, I ran into this error which I'm not sure what to do about:

The build.rs of libstd has an allow list of supported targets. You need to modify the source of libstd to add this target to the allow list.

darcagn commented 6 months ago

By commenting out "fminf", "fmin", "fmaxf", "fmax" and combining that with hints from the previous SH-related issue (#265), I was able to successfully build a no_std test crate with a test function, link the library in a C-based project, and call that test function, operational on real Sega Dreamcast hardware (sh4le).

When I get some time in the next few days I'll re-trace my steps here and put together instructions so others can follow along. Thank you both for your help, it is much appreciated!

darcagn commented 6 months ago

I'll go ahead and close this issue -- it seems that "fminf", "fmin", "fmaxf", "fmax" doesn't need commenting out anymore after changes in the project.

I've been using rustc_codegen_gcc to target the Sega Dreamcast for the past 3 or so weeks and it's been working very well! It's definitely a huge upgrade from the limited functionality of the gccrs frontend.

I put together a tutorial on how to target Dreamcast here: https://dreamcast.wiki/Rust_on_Dreamcast Along with a repo with supporting patches, scripts, and examples for the setup: https://github.com/darcagn/rust-for-dreamcast

Feel free to put any of that in the wiki under SuperH if you'd like. I'm super excited for the possibilities here and I'll definitely keep building this regularly and test for any regressions or issues. I hope perhaps I can contribute someday as well.

antoyo commented 6 months ago

Nice!

Btw, I saw this patch and that makes me wondered if you knew that arguments sent to -Cllvm-args will directly be sent to GCC when using rustc_codegen_gcc. That could be a way to not need any patches to it.

darcagn commented 6 months ago

Nice!

Btw, I saw this patch and that makes me wondered if you knew that arguments sent to -Cllvm-args will directly be sent to GCC when using rustc_codegen_gcc. That could be a way to not need any patches to it.

You are right -- I had not realized this! I now set up my scripts with a CG_GCCFLAGS variable that will have -Cllvm-args= tacked on to each argument and it will be passed through to cargo or rustc as necessary. And now that patch is removed. Thank you very much for pointing this out!