search

thepowersgang / mrustc

Alternative rust compiler (re-implementation)
MIT License
2.15k stars 105 forks source link

Fails to build under x86 (32bit): Excessive memory usage #78

Open aclemons opened 6 years ago

aclemons commented 6 years ago

Opening this so we have a place to track remaining issues for x86 support.

Building 42bdc18 with GCC 5.5 fails as below:

[...]
BUILDING core from core v0.0.0 with features []
output/libcore.hir.o.c: In function ‘_ZRI$aLusize$aR9max_value’:
output/libcore.hir.o.c:54780:7: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  rv = 0xffffffffffffffffull; // Return = Constant(18446744073709551615 usize)
       ^
output/libcore.hir.o.c: In function ‘_ZRI$aLusize$aR14saturating_add’:
output/libcore.hir.o.c:55084:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var1 = 0xffffffffffffffffull; // Local(1) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRI$aLusize$aR14saturating_mul’:
output/libcore.hir.o.c:55105:68: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  rv = _ZRI$aL_ZN4core6option6Option$aLusize$aR$aR9unwrap_or( var0, 0xffffffffffffffffull );
                                                                    ^
output/libcore.hir.o.c: In function ‘_ZRI$aL_ZN4core4cell12BorrowRefMut$aR3new’:
output/libcore.hir.o.c:67942:60: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var2 = _ZRI$aL_ZN4core4cell4Cell$aLusize$aR$aR3set( arg0, 0xffffffffffffffffull );
                                                            ^
output/libcore.hir.o.c: In function ‘_ZRI$aL_ZN4core3str7pattern14TwoWaySearcher$aR3new’:
output/libcore.hir.o.c:79087:13: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var14._6 = 0xffffffffffffffffull;
             ^
output/libcore.hir.o.c:79088:13: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var14._7 = 0xffffffffffffffffull; // Local(14) = Struct(::"core"::str::pattern::TwoWaySearcher, {Local(18), Local(21), Local(22), Lo
             ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLusize$aR$aR8try_from’:
output/libcore.hir.o.c:84805:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var2 = 0xffffffffffffffffull; // Local(2) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLisize$aR$aR8try_from’:
output/libcore.hir.o.c:84849:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var1 = 0xffffffffffffffffull; // Local(1) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLu8$aR$aR8try_from’:
output/libcore.hir.o.c:84896:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var2 = 0xffffffffffffffffull; // Local(2) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLi8$aR$aR8try_from’:
output/libcore.hir.o.c:84941:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var1 = 0xffffffffffffffffull; // Local(1) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLu16$aR$aR8try_from’:
output/libcore.hir.o.c:84988:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var2 = 0xffffffffffffffffull; // Local(2) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLi16$aR$aR8try_from’:
output/libcore.hir.o.c:85033:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var1 = 0xffffffffffffffffull; // Local(1) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLu32$aR$aR8try_from’:
output/libcore.hir.o.c:85080:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var2 = 0xffffffffffffffffull; // Local(2) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLi32$aR$aR8try_from’:
output/libcore.hir.o.c:85125:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var1 = 0xffffffffffffffffull; // Local(1) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLu64$aR$aR8try_from’:
output/libcore.hir.o.c:85172:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var2 = 0xffffffffffffffffull; // Local(2) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLi64$aR$aR8try_from’:
output/libcore.hir.o.c:85217:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var1 = 0xffffffffffffffffull; // Local(1) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLu128$aR$aR8try_from’:
output/libcore.hir.o.c:85263:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var2 = 0xffffffffffffffffull; // Local(2) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core7convert7TryFrom$aLi128$aR$aR8try_from’:
output/libcore.hir.o.c:85306:9: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  var1 = 0xffffffffffffffffull; // Local(1) = Constant(18446744073709551615 usize)
         ^
output/libcore.hir.o.c: In function ‘_ZRK$aLusize_as__ZN4core3num18FromStrRadixHelper$aR9max_value’:
output/libcore.hir.o.c:85938:7: warning: large integer implicitly truncated to unsigned type [-Woverflow]
  rv = 0xffffffffffffffffull; // Return = Constant(18446744073709551615 usize)
       ^
BUILDING rand from rand v0.0.0 with features []
BUILDING compiler_builtins from compiler_builtins v0.0.0 with features []
Macro:why_are_abi_strings_checked_by_parser:0: BUG:src/hir/from_ast.cpp:901: TODO: LowerHIR_Struct - Handle struct repr 'simd'
rustc-1.19.0-src/src/libcompiler_builtins/lib.rs:719: note: From here
BUILD FAILED
minicargo.mk:63: recipe for target 'output/libstd.hir' failed
make: *** [output/libstd.hir] Error 1
thepowersgang commented 6 years ago

The failure above is unimplemented support for #[repr(simd)]

thepowersgang commented 6 years ago

Tested with both hello_world and local tests, didn't run the full run-pass suite

aclemons commented 6 years ago

Thanks, core built with c1d4b60 successfully now. It falls over on a subsequent crate though:

[....]
BUILDING rand from rand v0.0.0 with features []
BUILDING term from term v0.0.0 with features []
BUILDING getopts from getopts v0.0.0 with features []
BUILDING test from test v0.0.0 with features []
mkdir -p output/rustc-build
CFG_COMPILER_HOST_TRIPLE=i686-unknown-linux-gnu LLVM_CONFIG=/tmp/SBo/mrustc-c1d4b60b462814d064e36c2e8fe833ece823da75/rustc-1.19.0-src/build/bin/llvm-config CFG_RELEASE= CFG_RELEASE_CHANNEL=stable CFG_VERSION=1.19.0-stable-mrustc CFG_PREFIX=mrustc CFG_LIBDIR_RELATIVE=lib tools/bin/minicargo rustc-1.19.0-src/src/rustc --vendor-dir rustc-1.19.0-src/src/vendor --output-dir output/rustc-build -L output/
BUILDING rustc_platform_intrinsics from rustc_platform_intrinsics v0.0.0 with features []
BUILDING rustc_serialize from rustc-serialize v0.3.24 with features []
MIR TODO: ::"core"::num::dec2flt::algorithm::fpu_precision::set_precision<f64,> BB2/4: Handle asm! output leader 'm'
BUILD FAILED
minicargo.mk:89: recipe for target 'output/rustc' failed
make: *** [output/rustc] Error 1
Build time 21 min, peak load 13.51, peak memory 12687M, peak tmp 9745M
thepowersgang commented 6 years ago

Above failure fixed in 2315092151c309f2d1a37b41d2ec2b6787dcc21c, current blocker is casting i128/u128 to f32

aclemons commented 6 years ago

Above failure fixed in 2315092

Confirmed. Thanks

current blocker is casting i128/u128 to f32

Also confirmed thanks.

MIR ERROR: <::"rustc_const_math"::int::ConstInt/E/ /- />::to_f32 BB5/0: Bad i128/u128 cast

thepowersgang commented 6 years ago

The i128 support should now be complete enough to build rustc, but cargo build is crashing due to an unknown codegen error.

thepowersgang commented 6 years ago

Codegen issue isolated to a mismatch between mrustc's calculated object size and gcc's calculation.

aclemons commented 6 years ago

I've just tested with f4483c0. It gets further along, but fails with

make[1]: Leaving directory '/tmp/build_mrustc/mrustc-f4483c076291f6c13b44d67a7a8f824ea2fcc682'
[RUSTC] -o prefix/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore.rlib
error: int literal is too large
  --> ../rustc-1.19.0-src/src/libcore/num/f32.rs:25:24
   |
25 | pub const RADIX: u32 = 2;
   |                        ^

error: aborting due to previous error(s)

Command terminated by signal 6
0.03user 0.00system 0:00.04elapsed 97%CPU (0avgtext+0avgdata 79824maxresident)k
0inputs+0outputs (0major+663minor)pagefaults 0swaps
Makefile:37: recipe for target 'prefix/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore.rlib' failed
make: *** [prefix/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore.rlib] Error 6
make: Leaving directory '/tmp/build_mrustc/mrustc-f4483c076291f6c13b44d67a7a8f824ea2fcc682/run_rustc'
thepowersgang commented 6 years ago

Bad i128 code fixed, but rustc isn't running correctly due to (probably) bad codegen on gcc's part (leading to a x87 FPU stack overflow). This may just be a local gcc bug (I'm testing with 6.3.0-18+deb9u1), but it's hard to determined exactly what

aclemons commented 6 years ago

Thanks. I've checked here with GCC 5.5 and it failed too.

Makefile:37: recipe for target 'prefix/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore.rlib' failed

Valgrind says:

==12316== Command: ../output/rustc -L prefix/lib/rustlib/x86_64-unknown-linux-gnu/lib/ -L ../output/libs ../rustc-1.19.0-src/src/libcore/lib.rs -o prefix/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore.rlib
==12316==
==12316== warning: Can't handle line info entry with line number 1241189 greater than 1048575
==12316== (Nb: this message is only shown once)
==12316== warning: Can't handle inlined call info entry with line number 1427365 greater than 1048575
==12316== (Nb: this message is only shown once)
==12316== Thread 2 rustc:
==12316== Invalid write of size 4
==12316==    at 0x851BE21: _ZRK$aL_ZN11collections3vec3Vec$aL_ZN5rustc3mir10BasicBlock$aR_as__ZN11collections3vec10SpecExtend$aL_ZN5rustc3mir10BasicBlock$C_ZN4core4iter5Chain$aL_ZN4core4iter3Map$aL_ZN4core5slice4Iter$aL_ZN5rustc6mid
dle9const_val8ConstVal$aR$C_ZN9rustc_mir12closure_I_32$aL_ZN9rustc_mir5build7Builder$aR$aR$C_ZN4core6option8IntoIter$aL_ZN5rustc3mir10BasicBlock$aR$aR$aR$aR11spec_extend (librustc_mir.hir.o.c:306800)
==12316==    by 0x851BE21: _ZRK$aL_ZN11collections3vec3Vec$aL_ZN5rustc3mir10BasicBlock$aR_as__ZN11collections3vec10SpecExtend$aL_ZN5rustc3mir10BasicBlock$C_ZN4core4iter5Chain$aL_ZN4core4iter3Map$aL_ZN4core5slice4Iter$aL_ZN5rustc6mid
dle9const_val8ConstVal$aR$C_ZN9rustc_mir12closure_I_32$aL_ZN9rustc_mir5build7Builder$aR$aR$C_ZN4core6option8IntoIter$aL_ZN5rustc3mir10BasicBlock$aR$aR$aR$aR9from_iter (librustc_mir.hir.o.c:306612)
==12316==    by 0x851BE21: _ZRI$aL_ZN9rustc_mir5build7Builder$aR12perform_test (librustc_mir.hir.o.c:217483)
==12316==    by 0x855E098: _ZRI$aL_ZN9rustc_mir5build7Builder$aR15test_candidates (librustc_mir.hir.o.c:220841)
==12316==    by 0x855DB97: _ZRI$aL_ZN9rustc_mir5build7Builder$aR16match_candidates (librustc_mir.hir.o.c:215587)
==12316==    by 0x8560E28: _ZRI$aL_ZN9rustc_mir5build7Builder$aR10match_expr (librustc_mir.hir.o.c:215815)
==12316==    by 0x85587B4: _ZRI$aL_ZN9rustc_mir5build7Builder$aR9into_expr (librustc_mir.hir.o.c:214528)
==12316==    by 0x85597D5: _ZRK$aL_ZN9rustc_mir4hair7ExprRef_as__ZN9rustc_mir5build4into8EvalInto$aR9eval_into (librustc_mir.hir.o.c:377772)
==12316==    by 0x85598B0: _ZRI$aL_ZN9rustc_mir5build7Builder$aR8in_scope$aL_ZN9rustc_mir12closure_I_16$aL_ZN9rustc_mir5build7Builder$aR$C$T0$aR (librustc_mir.hir.o.c:213078)
==12316==    by 0x85583E7: _ZRI$aL_ZN9rustc_mir5build7Builder$aR9into_expr (librustc_mir.hir.o.c:214387)
==12316==    by 0x85597D5: _ZRK$aL_ZN9rustc_mir4hair7ExprRef_as__ZN9rustc_mir5build4into8EvalInto$aR9eval_into (librustc_mir.hir.o.c:377772)
==12316==    by 0x855735E: _ZRI$aL_ZN9rustc_mir5build7Builder$aR15ast_block_stmts (librustc_mir.hir.o.c:208071)
==12316==    by 0x8557874: _ZRK$aL_ZN9rustc_mir11closure_I_1$aL_ZN9rustc_mir5build7Builder$aR_as__ZN4core3ops6FnOnce$aL$T1_$Ru__ZN9rustc_mir5build7Builder$aR$aR9call_once (librustc_mir.hir.o.c:370802)
==12316==    by 0x8557A48: _ZRI$aL_ZN9rustc_mir5build7Builder$aR8in_scope$aL_ZN9rustc_mir11closure_I_1$aL_ZN9rustc_mir5build7Builder$aR$C$T0$aR (librustc_mir.hir.o.c:212607)
==12316==  Address 0x1f42fcbc is 0 bytes after a block of size 4 alloc'd
==12316==    at 0x402F26C: malloc (vg_replace_malloc.c:299)
==12316==    by 0x8CDC448: _ZN12alloc_system3imp8allocate (liballoc_system.hir.o.c:361)
==12316==    by 0x84EA2D5: _ZN5alloc4heap8allocate (librustc_mir.hir.o.c:107766)
==12316==    by 0x84EA2D5: _ZRI$aL_ZN5alloc7raw_vec6RawVec$aL_ZN5rustc3mir10BasicBlock$aR$aR7reserve.part.740 (librustc_mir.hir.o.c:149496)
==12316==    by 0x851C4AB: _ZRI$aL_ZN5alloc7raw_vec6RawVec$aL_ZN5rustc3mir10BasicBlock$aR$aR7reserve (librustc_mir.hir.o.c:195437)
==12316==    by 0x851C4AB: _ZRK$aL_ZN11collections3vec3Vec$aL_ZN5rustc3mir10BasicBlock$aR_as__ZN11collections3vec10SpecExtend$aL_ZN5rustc3mir10BasicBlock$C_ZN4core4iter5Chain$aL_ZN4core4iter3Map$aL_ZN4core5slice4Iter$aL_ZN5rustc6mid
dle9const_val8ConstVal$aR$C_ZN9rustc_mir12closure_I_32$aL_ZN9rustc_mir5build7Builder$aR$aR$C_ZN4core6option8IntoIter$aL_ZN5rustc3mir10BasicBlock$aR$aR$aR$aR11spec_extend (librustc_mir.hir.o.c:306757)
==12316==    by 0x851C4AB: _ZRK$aL_ZN11collections3vec3Vec$aL_ZN5rustc3mir10BasicBlock$aR_as__ZN11collections3vec10SpecExtend$aL_ZN5rustc3mir10BasicBlock$C_ZN4core4iter5Chain$aL_ZN4core4iter3Map$aL_ZN4core5slice4Iter$aL_ZN5rustc6mid
dle9const_val8ConstVal$aR$C_ZN9rustc_mir12closure_I_32$aL_ZN9rustc_mir5build7Builder$aR$aR$C_ZN4core6option8IntoIter$aL_ZN5rustc3mir10BasicBlock$aR$aR$aR$aR9from_iter (librustc_mir.hir.o.c:306612)
==12316==    by 0x851C4AB: _ZRI$aL_ZN9rustc_mir5build7Builder$aR12perform_test (librustc_mir.hir.o.c:217483)
==12316==    by 0x855E098: _ZRI$aL_ZN9rustc_mir5build7Builder$aR15test_candidates (librustc_mir.hir.o.c:220841)
==12316==    by 0x855DB97: _ZRI$aL_ZN9rustc_mir5build7Builder$aR16match_candidates (librustc_mir.hir.o.c:215587)
==12316==    by 0x8560E28: _ZRI$aL_ZN9rustc_mir5build7Builder$aR10match_expr (librustc_mir.hir.o.c:215815)
==12316==    by 0x85587B4: _ZRI$aL_ZN9rustc_mir5build7Builder$aR9into_expr (librustc_mir.hir.o.c:214528)
==12316==    by 0x85597D5: _ZRK$aL_ZN9rustc_mir4hair7ExprRef_as__ZN9rustc_mir5build4into8EvalInto$aR9eval_into (librustc_mir.hir.o.c:377772)
==12316==    by 0x85598B0: _ZRI$aL_ZN9rustc_mir5build7Builder$aR8in_scope$aL_ZN9rustc_mir12closure_I_16$aL_ZN9rustc_mir5build7Builder$aR$C$T0$aR (librustc_mir.hir.o.c:213078)
==12316==    by 0x85583E7: _ZRI$aL_ZN9rustc_mir5build7Builder$aR9into_expr (librustc_mir.hir.o.c:214387)
==12316==    by 0x85597D5: _ZRK$aL_ZN9rustc_mir4hair7ExprRef_as__ZN9rustc_mir5build4into8EvalInto$aR9eval_into (librustc_mir.hir.o.c:377772)
==12316==
==12316== Invalid write of size 4
==12316==    at 0x851C283: _ZRK$aL_ZN11collections3vec3Vec$aL_ZN16rustc_const_math3int8ConstInt$aR_as__ZN11collections3vec10SpecExtend$aL_ZN16rustc_const_math3int8ConstInt$C_ZN4core4iter3Map$aL_ZN4core5slice4Iter$aL_ZN5rustc6middle9
const_val8ConstVal$aR$C_ZN9rustc_mir12closure_I_33$aL_ZN9rustc_mir5build7Builder$aR$aR$aR$aR11spec_extend (librustc_mir.hir.o.c:318653)
==12316==    by 0x851C283: _ZRK$aL_ZN11collections3vec3Vec$aL_ZN16rustc_const_math3int8ConstInt$aR_as__ZN11collections3vec10SpecExtend$aL_ZN16rustc_const_math3int8ConstInt$C_ZN4core4iter3Map$aL_ZN4core5slice4Iter$aL_ZN5rustc6middle9
const_val8ConstVal$aR$C_ZN9rustc_mir12closure_I_33$aL_ZN9rustc_mir5build7Builder$aR$aR$aR$aR9from_iter (librustc_mir.hir.o.c:318455)
==12316==    by 0x851C283: _ZRI$aL_ZN9rustc_mir5build7Builder$aR12perform_test (librustc_mir.hir.o.c:217493)
==12316==    by 0x855E098: _ZRI$aL_ZN9rustc_mir5build7Builder$aR15test_candidates (librustc_mir.hir.o.c:220841)
==12316==    by 0x855DB97: _ZRI$aL_ZN9rustc_mir5build7Builder$aR16match_candidates (librustc_mir.hir.o.c:215587)
==12316==    by 0x8560E28: _ZRI$aL_ZN9rustc_mir5build7Builder$aR10match_expr (librustc_mir.hir.o.c:215815)
==12316==    by 0x85587B4: _ZRI$aL_ZN9rustc_mir5build7Builder$aR9into_expr (librustc_mir.hir.o.c:214528)
==12316==    by 0x85597D5: _ZRK$aL_ZN9rustc_mir4hair7ExprRef_as__ZN9rustc_mir5build4into8EvalInto$aR9eval_into (librustc_mir.hir.o.c:377772)
==12316==    by 0x85598B0: _ZRI$aL_ZN9rustc_mir5build7Builder$aR8in_scope$aL_ZN9rustc_mir12closure_I_16$aL_ZN9rustc_mir5build7Builder$aR$C$T0$aR (librustc_mir.hir.o.c:213078)
==12316==    by 0x85583E7: _ZRI$aL_ZN9rustc_mir5build7Builder$aR9into_expr (librustc_mir.hir.o.c:214387)
==12316==    by 0x85597D5: _ZRK$aL_ZN9rustc_mir4hair7ExprRef_as__ZN9rustc_mir5build4into8EvalInto$aR9eval_into (librustc_mir.hir.o.c:377772)
==12316==    by 0x855735E: _ZRI$aL_ZN9rustc_mir5build7Builder$aR15ast_block_stmts (librustc_mir.hir.o.c:208071)
==12316==    by 0x8557874: _ZRK$aL_ZN9rustc_mir11closure_I_1$aL_ZN9rustc_mir5build7Builder$aR_as__ZN4core3ops6FnOnce$aL$T1_$Ru__ZN9rustc_mir5build7Builder$aR$aR9call_once (librustc_mir.hir.o.c:370802)
==12316==    by 0x8557A48: _ZRI$aL_ZN9rustc_mir5build7Builder$aR8in_scope$aL_ZN9rustc_mir11closure_I_1$aL_ZN9rustc_mir5build7Builder$aR$C$T0$aR (librustc_mir.hir.o.c:212607)
==12316==  Address 0x4 is not stack'd, malloc'd or (recently) free'd

I can upload librustc_mir.hir.o.c again if you need it.

thepowersgang commented 6 years ago

The valgrind output is caused by bad alignment/size calculation, but even with that fixed (see 74c2ec9d) rustc is crashing with a SIGFPU.

It appears to be caused by gcc doing incorrect codegen and overflowing the x87 stack, but I haven't been able to confirm that.

aclemons commented 6 years ago

Right, I was building 346da186. I've now built 74c2ec9d with GCC 5.5.0.

It fails with

[...]
> /home/aclemons/slack-rust-bootstrap/mrustc/tmp/mrustc-74c2ec9d3c7eb09540a2a5de3127390fa8252179/bin/mrustc rustc-1.19.0-src/src/tools/cargo/src/crates-io/lib.rs --crate-name crates_io --crate-type rlib --crate-tag 0_9_0 -g --cfg debug_assertions -O -o output/cargo-build/libcrates_io-0_9_0.hir -L output/cargo-build --extern curl=output/cargo-build/libcurl-0_4_6.hir --extern error_chain=output/cargo-build/liberror_chain-0_10_0.hir --extern serde=output/cargo-build/libserde-1_0_6.hir --extern serde_derive=output/cargo-build/libserde_derive-1_0_6.hir --extern serde_json=output/cargo-build/libserde_json-1_0_2.hir --extern url=output/cargo-build/liburl-1_4_0.hir -L output
output/cargo-build/libcrates_io-0_9_0.hir.o.c: In function ‘_ZN4core6option6Option4Some$aL_ZN12serde$$1_0_62de11ignored_any10IgnoredAny$aR’:
output/cargo-build/libcrates_io-0_9_0.hir.o.c:9054:9: error: unknown field ‘var_1’ specified in initializer
  struct e__ZN4core6option6Option$aL_ZN12serde$$1_0_62de11ignored_any10IgnoredAny$aR rv = { .TAG = 1, .DATA = { .var_1 = {
         ^
output/cargo-build/libcrates_io-0_9_0.hir.o.c:9054:121: error: extra brace group at end of initializer
  struct e__ZN4core6option6Option$aL_ZN12serde$$1_0_62de11ignored_any10IgnoredAny$aR rv = { .TAG = 1, .DATA = { .var_1 = {
                                                                                                                         ^
output/cargo-build/libcrates_io-0_9_0.hir.o.c:9054:121: note: (near initialization for ‘rv.DATA’)
output/cargo-build/libcrates_io-0_9_0.hir.o.c:9054:121: warning: excess elements in union initializer
output/cargo-build/libcrates_io-0_9_0.hir.o.c:9054:121: note: (near initialization for ‘rv.DATA’)
C Compiler failed to execute - error code 256
BUILD FAILED
minicargo.mk:93: recipe for target 'output/cargo' failed

Is this what you are referring to?

thepowersgang commented 6 years ago

No... (notes to test cargo with new code) - rustc should compile, but is crashing with SIGFPE

aclemons commented 6 years ago

Ok, dd4e3c88 builds again. Thanks.

aclemons commented 6 years ago

Also fyi, dd4e3c8 does not currently finish on x86_64 (GCC 5.5) either. I'm getting a SIGABRT here:

[...]
[RUSTC] -o prefix/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore.rlib
Command terminated by signal 6
Apteryks commented 2 years ago

Here's a fresh log when attempting to build for i686-linux, using mrustc at commit 474bec9cfd7862a20e7288cecd7fcf5e18648b9a to bootstrap from 1.39 (GNU Guix with GCC 10.3.0).


[AR] -o bin/mrustc.a
[CXX] -o bin/mrustc
objcopy --only-keep-debug bin/mrustc bin/mrustc.debug
objcopy --add-gnu-debuglink=bin/mrustc.debug bin/mrustc
strip bin/mrustc
make[1]: Leaving directory '/tmp/guix-build-rust-1.39.0.drv-0/mrustc'
test -e bin/mrustc
bin/minicargo rustc-1.39.0-src/src/libstd --vendor-dir rustc-1.39.0-src/vendor --script-overrides script-overrides/stable-1.39.0-linux/ --output-dir output/ -j16 --manifest-overrides rustc-1.39.0-overrides.toml
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc -Z print-cfgs
(0/17) BUILDING core v0.0.0
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/src/libcore/lib.rs -o output/libcore.rlib --crate-name core --crate-type rlib -C emit-depfile=output/libcore.rlib.d --crate-tag 0_0_0 -g --cfg debug_assertions -O -L output --edition 2018
rustc-1.39.0-src/src/libcore/slice/mod.rs:67: warn:0:Unexpected attribute allow_internal_unstable on associated item
rustc-1.39.0-src/src/libcore/str/mod.rs:2173: warn:0:Unexpected attribute allow_internal_unstable on associated item
output/libcore.rlib.c: In function ‘ZRG4ch4core50_0_09core_arch3x863adx17llvm_addcarry_u320g’:
output/libcore.rlib.c:60184:10: warning: implicit declaration of function ‘__builtin_add_overflow_u32’; did you mean ‘__builtin_add_overflow_p’? [-Wimplicit-function-declaration]
60184 |  rv._0 = __builtin_add_overflow_u32(arg1, arg2, &rv._1);
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      |          __builtin_add_overflow_p
output/libcore.rlib.c: In function ‘ZRG4ch4core50_0_09core_arch3x863adx18llvm_subborrow_u320g’:
output/libcore.rlib.c:60205:10: warning: implicit declaration of function ‘__builtin_sub_overflow_u32’; did you mean ‘__builtin_sub_overflow_p’? [-Wimplicit-function-declaration]
60205 |  rv._0 = __builtin_sub_overflow_u32(arg1, arg2, &rv._1);
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      |          __builtin_sub_overflow_p
In file included from output/libcore.rlib.c:7:
output/libcore.rlib.c: In function ‘ZRQG2ch4core50_0_03ffi10VaListImpl0g2cb05clone5Clone0gb30g’:
output/libcore.rlib.c:393911:10: warning: passing argument 1 of ‘__builtin_va_copy’ from incompatible pointer type [-Wincompatible-pointer-types]
393911 |  va_copy(var3, arg0);
       |          ^~~~
       |          |
       |          struct s_ZRG2ch4core50_0_03ffi10VaListImpl0g **
output/libcore.rlib.c:393911:10: note: expected ‘char **’ but argument is of type ‘struct s_ZRG2ch4core50_0_03ffi10VaListImpl0g **’
output/libcore.rlib.c:393911:16: warning: passing argument 2 of ‘__builtin_va_copy’ from incompatible pointer type [-Wincompatible-pointer-types]
393911 |  va_copy(var3, arg0);
       |                ^~~~
       |                |
       |                struct s_ZRG2ch4core50_0_03ffi10VaListImpl0g *
output/libcore.rlib.c:393911:16: note: expected ‘__builtin_va_list’ but argument is of type ‘struct s_ZRG2ch4core50_0_03ffi10VaListImpl0g *’
output/libcore.rlib.c: Assembler messages:
output/libcore.rlib.c:192155: Error: unknown pseudo-op: `.syntax'
output/libcore.rlib.c:192157: Error: no such instruction: `pushfd'
output/libcore.rlib.c:192166: Error: no such instruction: `popfd '
output/libcore.rlib.c:192168: Error: no such instruction: `pushfd '
output/libcore.rlib.c:192173: Error: unknown pseudo-op: `.syntax'
output/libcore.rlib.c:192166: Error: no such instruction: `pushfd'
output/libcore.rlib.c:192177: Error: no such instruction: `popfd'
C Compiler failed to execute - error code 256
Process exited with non-zero exit status 1
BUILD FAILED
make: *** [minicargo.mk:82: output/libstd.rlib] Error 1
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("-f" "minicargo.mk" "output/rustc" "-j16" "RUSTC_TARGET=i686-unknown-linux-gnu" "RUSTC_VERSION=1.39.0" "MRUSTC_TARGET_VER=1.39" "OUTDIR_SUF=") exit-status: 2 term-signal: #f stop-signal: #f>
phase `build' failed after 200.0 seconds
command "make" "-f" "minicargo.mk" "output/rustc" "-j16" "RUSTC_TARGET=i686-unknown-linux-gnu" "RUSTC_VERSION=1.39.0" "MRUSTC_TARGET_VER=1.39" "OUTDIR_SUF=" failed with status 2
builder for `/gnu/store/2hy27x5ws74x2hx9v8rnfdks9439dmj9-rust-1.39.0.drv' failed with exit code 1
@ build-failed /gnu/store/2hy27x5ws74x2hx9v8rnfdks9439dmj9-rust-1.39.0.drv - 1 builder for `/gnu/store/2hy27x5ws74x2hx9v8rnfdks9439dmj9-rust-1.39.0.drv' failed with exit code 1
derivation '/gnu/store/2hy27x5ws74x2hx9v8rnfdks9439dmj9-rust-1.39.0.drv' offloaded to '141.80.167.171' failed: build of `/gnu/store/2hy27x5ws74x2hx9v8rnfdks9439dmj9-rust-1.39.0.drv' failed
build of /gnu/store/2hy27x5ws74x2hx9v8rnfdks9439dmj9-rust-1.39.0.drv failed
thepowersgang commented 2 years ago

Confirmed, part of it is basic issue with .syntax intel instead of .intel_syntax. However, two of those errors are not caused by that.

thepowersgang commented 2 years ago

The above commits fixed the asm issues on my testing machine, and allows it to build a basic "hello, world"

Apteryks commented 2 years ago

Hello! I tried to build with the latest commit on x86_64 to get started, and it failed like so while building cargo via minicargo:

(98/109) BUILDING openssl_sys from openssl-sys v0.9.43
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/vendor/openssl-sys/src/lib.rs -o output/cargo-build/libopenssl_sys-0_9_43.rlib --crate-name openssl_sys --crate-type rlib -C emit-depfile=output/cargo-build/libopenssl_sys-0_9_43.rlib.d --crate-tag 0_9_43 -g --cfg debug_assertions -O -L output -L output/cargo-build -L /gnu/store/cs1kihs34ccqhc69yx0c4kaf3rkiwyyy-openssl-1.1.1l/lib -l ssl -l crypto --cfg const_fn --cfg osslconf="OPENSSL_NO_SSL3_METHOD" --cfg ossl101 --cfg ossl102 --cfg ossl102f --cfg ossl102h --cfg ossl110 --cfg ossl110f --cfg ossl110g --cfg ossl111 --cfg ossl111b --extern libc=output/cargo-build/liblibc-0_2_62_H10.rlib
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/vendor/openssl/build.rs --crate-name build --crate-type bin -o output/cargo-build/build_openssl-0_10_16_run -L output/cargo-build -g -L output
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/output/cargo-build/build_openssl-0_10_16_run
(99/109) BUILDING openssl v0.10.16
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/vendor/openssl/src/lib.rs -o output/cargo-build/libopenssl-0_10_16.rlib --crate-name openssl --crate-type rlib -C emit-depfile=output/cargo-build/libopenssl-0_10_16.rlib.d --crate-tag 0_10_16 -g --cfg debug_assertions -O -L output -L output/cargo-build --cfg osslconf="OPENSSL_NO_SSL3_METHOD" --cfg ossl101 --cfg ossl102 --cfg ossl110 --cfg ossl110g --cfg ossl111 --extern bitflags=output/cargo-build/libbitflags-1_1_0.rlib --extern cfg_if=output/cargo-build/libcfg_if-0_1_8.rlib --extern foreign_types=output/cargo-build/libforeign_types-0_3_2.rlib --extern lazy_static=output/cargo-build/liblazy_static-1_3_0.rlib --extern libc=output/cargo-build/liblibc-0_2_62_H10.rlib --extern openssl_sys=output/cargo-build/libopenssl_sys-0_9_43.rlib
rustc-1.39.0-src/vendor/openssl/src/macros.rs:97: warn:0:Spare Rule - *const ::"openssl_sys-0_9_43"::ossl_typ::DH := *const <::"openssl-0_10_16"::dh::DhRef<T/*I:0*/,> as ::"foreign_types_shared-0_1_1"::ForeignTypeRef>::CType
Macro:to_der:0: note: From here
rustc-1.39.0-src/vendor/openssl/src/dh.rs:36: note: From here
:0: BUG:src/hir_typeck/expr_cs.cpp:7222: Spare rules left after typecheck stabilised
Process was terminated with signal 6
FAILING COMMAND:  /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/vendor/openssl/src/lib.rs -o output/cargo-build/libopenssl-0_10_16.rlib --crate-name openssl --crate-type rlib -C emit-depfile=output/cargo-build/libopenssl-0_10_16.rlib.d --crate-tag 0_10_16 -g --cfg debug_assertions -O -L output -L output/cargo-build --cfg osslconf="OPENSSL_NO_SSL3_METHOD" --cfg ossl101 --cfg ossl102 --cfg ossl110 --cfg ossl110g --cfg ossl111 --extern bitflags=output/cargo-build/libbitflags-1_1_0.rlib --extern cfg_if=output/cargo-build/libcfg_if-0_1_8.rlib --extern foreign_types=output/cargo-build/libforeign_types-0_3_2.rlib --extern lazy_static=output/cargo-build/liblazy_static-1_3_0.rlib --extern libc=output/cargo-build/liblibc-0_2_62_H10.rlib --extern openssl_sys=output/cargo-build/libopenssl_sys-0_9_43.rlib make: *** [minicargo.mk:140: output/cargo] Error 1
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("-f" "minicargo.mk" "output/cargo" "-j24" "RUSTC_TARGET=x86_64-unknown-linux-gnu" "RUSTC_VERSION=1.39.0" "MRUSTC_TARGET_VER=1.39" "OUTDIR_SUF=") exit-status: 2 term-signal: #f stop-signal: #f> 
phase `build' failed after 3632.7 seconds
command "make" "-f" "minicargo.mk" "output/cargo" "-j24" "RUSTC_TARGET=x86_64-unknown-linux-gnu" "RUSTC_VERSION=1.39.0" "MRUSTC_TARGET_VER=1.39" "OUTDIR_SUF=" failed with status 2
builder for `/gnu/store/cgqxy64ab1h8lsymvxwvmisj27ldk8rb-rust-1.39.0.drv' failed with exit code 1
build of /gnu/store/cgqxy64ab1h8lsymvxwvmisj27ldk8rb-rust-1.39.0.drv failed
View build log at '/var/log/guix/drvs/cg/qxy64ab1h8lsymvxwvmisj27ldk8rb-rust-1.39.0.drv'

Any ideas? If I can get passed this bump, I'll test with i686-linux!

Apteryks commented 2 years ago

On current master, attempting to build for i686-linux fails early like:

> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/output/rustc-build/build_bitflags-1_1_0_run
(16/112) BUILDING bitflags v1.1.0
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/vendor/bitflags/src/lib.rs -o output/rustc-build/libbitflags-1_1_0.rlib --crate-name bitflags --crate-type rlib -C emit-depfile=output/rustc-build/libbitflags-1_1_0.rlib.d --crate-tag 1_1_0 -g --cfg debug_assertions -O -L output -L output/rustc-build --cfg bitflags_const_fn
(17/112) BUILDING cc v1.0.35
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/vendor/cc/src/lib.rs -o output/rustc-build/libcc-1_0_35.rlib --crate-name cc --crate-type rlib -C emit-depfile=output/rustc-build/libcc-1_0_35.rlib.d --crate-tag 1_0_35 -g --cfg debug_assertions -O -L output -L output/rustc-build
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/src/librustc_llvm/build.rs --crate-name build --crate-type bin -o output/rustc-build/build_rustc_llvm_run -L output/rustc-build -g -L output --extern build_helper=output/rustc-build/libbuild_helper-0_1_0.rlib --extern cc=output/rustc-build/libcc-1_0_35.rlib --edition 2018
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/output/rustc-build/build_rustc_llvm_run
thread 'main' panicked at 'assertion failed: `(left != right)`
  left: `0`,
 right: `0`', rustc-1.39.0-src/vendor/hashbrown/src/raw/mod.rs:446:47
Process was terminated with signal 6
FAILING COMMAND:  /tmp/guix-build-rust-1.39.0.drv-0/mrustc/output/rustc-build/build_rustc_llvm_run make: *** [minicargo.mk:133: output/rustc] Error 1
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("-f" "minicargo.mk" "output/rustc" "-j24" "RUSTC_TARGET=i686-unknown-linux-gnu" "RUSTC_VERSION=1.39.0" "MRUSTC_TARGET_VER=1.39" "OUTDIR_SUF=") exit-status: 2 term-signal: #f stop-signal: #f> 
phase `build' failed after 230.6 seconds
command "make" "-f" "minicargo.mk" "output/rustc" "-j24" "RUSTC_TARGET=i686-unknown-linux-gnu" "RUSTC_VERSION=1.39.0" "MRUSTC_TARGET_VER=1.39" "OUTDIR_SUF=" failed with status 2
Apteryks commented 2 years ago

@thepowersgang OK, x86_64 can now build cargo again, but on i686-linux it fails early still with:

(15/112) BUILDING annotate_snippets from annotate-snippets v0.6.1
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/vendor/annotate-snippets/src/lib.rs -o output/rustc-build/libannotate_snippets-0_6_1.rlib --crate-name annotate_snippets --crate-type rlib -C emit-depfile=output/rustc-build/libannotate_snippets-0_6_1.rlib.d --crate-tag 0_6_1 -g --cfg debug_assertions -O -L output -L output/rustc-build --edition 2018
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/vendor/bitflags/build.rs --crate-name build --crate-type bin -o output/rustc-build/build_bitflags-1_1_0_run -L output/rustc-build -g -L output
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/output/rustc-build/build_bitflags-1_1_0_run
(16/112) BUILDING bitflags v1.1.0
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/vendor/bitflags/src/lib.rs -o output/rustc-build/libbitflags-1_1_0.rlib --crate-name bitflags --crate-type rlib -C emit-depfile=output/rustc-build/libbitflags-1_1_0.rlib.d --crate-tag 1_1_0 -g --cfg debug_assertions -O -L output -L output/rustc-build --cfg bitflags_const_fn
(17/112) BUILDING cc v1.0.35
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/vendor/cc/src/lib.rs -o output/rustc-build/libcc-1_0_35.rlib --crate-name cc --crate-type rlib -C emit-depfile=output/rustc-build/libcc-1_0_35.rlib.d --crate-tag 1_0_35 -g --cfg debug_assertions -O -L output -L output/rustc-build
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/src/librustc_llvm/build.rs --crate-name build --crate-type bin -o output/rustc-build/build_rustc_llvm_run -L output/rustc-build -g -L output --extern build_helper=output/rustc-build/libbuild_helper-0_1_0.rlib --extern cc=output/rustc-build/libcc-1_0_35.rlib --edition 2018
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/output/rustc-build/build_rustc_llvm_run
thread 'main' panicked at 'assertion failed: `(left != right)`
  left: `0`,
 right: `0`', rustc-1.39.0-src/vendor/hashbrown/src/raw/mod.rs:446:47
Process was terminated with signal 6
FAILING COMMAND:  /tmp/guix-build-rust-1.39.0.drv-0/mrustc/output/rustc-build/build_rustc_llvm_run make: *** [minicargo.mk:133: output/rustc] Error 1
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("-f" "minicargo.mk" "output/rustc" "-j16" "RUSTC_TARGET=i686-unknown-linux-gnu" "RUSTC_VERSION=1.39.0" "MRUSTC_TARGET_VER=1.39" "OUTDIR_SUF=") exit-status: 2 term-signal: #f stop-signal: #f>
phase `build' failed after 463.0 seconds
command "make" "-f" "minicargo.mk" "output/rustc" "-j16" "RUSTC_TARGET=i686-unknown-linux-gnu" "RUSTC_VERSION=1.39.0" "MRUSTC_TARGET_VER=1.39" "OUTDIR_SUF=" failed with status 2
builder for `/gnu/store/kwj602j88g5jfp46mwxghyssw0i3yc7v-rust-1.39.0.drv' failed with exit code 1

thanks for the support!

thepowersgang commented 2 years ago

Cause: https://github.com/thepowersgang/mrustc/blob/832b512bd112d3e7cd4291a6bace62a50310173f/src/trans/codegen_c.cpp#L6175 It was assuming that usize is always 64-bit

Apteryks commented 2 years ago

Nice! How did you find out about it?

thepowersgang commented 2 years ago

Reading the generated code, adding printf statements, and checking that each value was sane - until finding the source of the bad value.

Apteryks commented 2 years ago

Eh, it got really far this time, before failing on this:

(93/112) BUILDING rustc v0.0.0
> /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/src/librustc/lib.rs -o output/rustc-build/librustc.rlib --crate-name rustc --crate-type rlib -C emit-depfile=output/rustc-build/librustc.rlib.d --crate-tag 0_0_0 -g --cfg debug_assertions -O -L output -L output/rustc-build --edition 2018 --extern arena=output/rustc-build/libarena.rlib --extern bitflags=output/rustc-build/libbitflags-1_1_0.rlib --extern fmt_macros=output/rustc-build/libfmt_macros.rlib --extern graphviz=output/rustc-build/libgraphviz.rlib --extern jobserver=output/rustc-build/libjobserver-0_1_16.rlib --extern num_cpus=output/rustc-build/libnum_cpus-1_8_0.rlib --extern scoped_tls=output/rustc-build/libscoped_tls-1_0_0.rlib --extern log=output/rustc-build/liblog-0_4_8_H8400.rlib --extern rustc_rayon=output/rustc-build/librustc_rayon-0_2_0.rlib --extern rustc_rayon_core=output/rustc-build/librustc_rayon_core-0_2_0.rlib --extern polonius_engine=output/rustc-build/libpolonius_engine-0_10_0.rlib --extern rustc_apfloat=output/rustc-build/librustc_apfloat.rlib --extern rustc_target=output/rustc-build/librustc_target.rlib --extern rustc_macros=output/rustc-build/librustc_macros-0_1_0-plugin --extern rustc_data_structures=output/rustc-build/librustc_data_structures.rlib --extern errors=output/rustc-build/librustc_errors.rlib --extern rustc_serialize=output/rustc-build/libserialize.rlib --extern syntax=output/rustc-build/libsyntax.rlib --extern syntax_pos=output/rustc-build/libsyntax_pos.rlib --extern backtrace=output/rustc-build/libbacktrace-0_3_37_H101862.rlib --extern parking_lot=output/rustc-build/libparking_lot-0_9_0_H2.rlib --extern byteorder=output/rustc-build/libbyteorder-1_3_2_H2.rlib --extern chalk_engine=output/rustc-build/libchalk_engine-0_9_0.rlib --extern rustc_fs_util=output/rustc-build/librustc_fs_util.rlib --extern smallvec=output/rustc-build/libsmallvec-0_6_10_H19.rlib --extern measureme=output/rustc-build/libmeasureme-0_3_0.rlib

cc1: out of memory allocating 438707880 bytes after a total of 95367168 bytes
C Compiler failed to execute - error code 256
Process exited with non-zero exit status 1
FAILING COMMAND:  /tmp/guix-build-rust-1.39.0.drv-0/mrustc/bin/mrustc rustc-1.39.0-src/src/librustc/lib.rs -o output/rustc-build/librustc.rlib --crate-name rustc --crate-type rlib -C emit-depfile=output/rustc-build/librustc.rlib.d --crate-tag 0_0_0 -g --cfg debug_assertions -O -L output -L output/rustc-build --edition 2018 --extern arena=output/rustc-build/libarena.rlib --extern bitflags=output/rustc-build/libbitflags-1_1_0.rlib --extern fmt_macros=output/rustc-build/libfmt_macros.rlib --extern graphviz=output/rustc-build/libgraphviz.rlib --extern jobserver=output/rustc-build/libjobserver-0_1_16.rlib --extern num_cpus=output/rustc-build/libnum_cpus-1_8_0.rlib --extern scoped_tls=output/rustc-build/libscoped_tls-1_0_0.rlib --extern log=output/rustc-build/liblog-0_4_8_H8400.rlib --extern rustc_rayon=output/rustc-build/librustc_rayon-0_2_0.rlib --extern rustc_rayon_core=output/rustc-build/librustc_rayon_core-0_2_0.rlib --extern polonius_engine=output/rustc-build/libpolonius_engine-0_10_0.rlib --extern rustc_apfloat=output/rustc-build/librustc_apfloat.rlib --extern rustc_target=output/rustc-build/librustc_target.rlib --extern rustc_macros=output/rustc-build/librustc_macros-0_1_0-plugin --extern rustc_data_structures=output/rustc-build/librustc_data_structures.rlib --extern errors=output/rustc-build/librustc_errors.rlib --extern rustc_serialize=output/rustc-build/libserialize.rlib --extern syntax=output/rustc-build/libsyntax.rlib --extern syntax_pos=output/rustc-build/libsyntax_pos.rlib --extern backtrace=output/rustc-build/libbacktrace-0_3_37_H101862.rlib --extern parking_lot=output/rustc-build/libparking_lot-0_9_0_H2.rlib --extern byteorder=output/rustc-build/libbyteorder-1_3_2_H2.rlib --extern chalk_engine=output/rustc-build/libchalk_engine-0_9_0.rlib --extern rustc_fs_util=output/rustc-build/librustc_fs_util.rlib --extern smallvec=output/rustc-build/libsmallvec-0_6_10_H19.rlib --extern measureme=output/rustc-build/libmeasureme-0_3_0.rlib make: *** [minicargo.mk:133: output/rustc] Error 1
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("-f" "minicargo.mk" "output/rustc" "-j24" "RUSTC_TARGET=i686-unknown-linux-gnu" "RUSTC_VERSION=1.39.0" "MRUSTC_TARGET_VER=1.39" "OUTDIR_SUF=") exit-status: 2 term-signal: #f stop-signal: #f>
phase `build' failed after 4035.4 seconds
command "make" "-f" "minicargo.mk" "output/rustc" "-j24" "RUSTC_TARGET=i686-unknown-linux-gnu" "RUSTC_VERSION=1.39.0" "MRUSTC_TARGET_VER=1.39" "OUTDIR_SUF=" failed with status 2
builder for `/gnu/store/z65896b3wj21gc1731py14mzlpd95dd6-rust-1.39.0.drv' failed with exit code 1
build of /gnu/store/z65896b3wj21gc1731py14mzlpd95dd6-rust-1.39.0.drv failed
View build log at '/var/log/guix/drvs/z6/5896b3wj21gc1731py14mzlpd95dd6-rust-1.39.0.drv'.
guix build: error: build of `/gnu/store/z65896b3wj21gc1731py14mzlpd95dd6-rust-1.39.0.drv' failed

real    67m59.376s
thepowersgang commented 2 years ago

Aah, good old librustc chewing memory. Will need to look into a way of reducing memory usage while calling gcc.

civodul commented 2 years ago

@thepowersgang Do you think there's a way the generated code could cause GCC to be less memory-hungry?

Or perhaps getting mrustc to invoke gcc with -O0 or similar would help (whether the resulting code is efficient is a secondary concern)?

thepowersgang commented 2 years ago

I was thinking of doing a lot of memory cleanup before invoking gcc, although - just adding more swap would help :)

stikonas commented 2 years ago

I was thinking of doing a lot of memory cleanup before invoking gcc, although - just adding more swap would help :)

Would it really help if we are bound by 32-bit addressing limit of 4 GiB?

thepowersgang commented 2 years ago

438707880 = 0x1A26_26A8 95367168 = 0x5AF_3000

Seems like it's just out of RAM (not out of address space)?

Apteryks commented 2 years ago

I'd rather think we're hitting a 32-bit limit because building fails on a machine with 32 GiB of RAM and 16 GiB of swap:

$ free -h
              total        used        free      shared  buff/cache   available
Mem:           31Gi       1.3Gi        29Gi       0.0Ki       952Mi        29Gi
Swap:          15Gi       820Mi        15Gi
thepowersgang commented 2 years ago

Ooh, and yeah - 2GB limit: 0x1A26_26A8 + 0x5AF_3000 > 0x8000_0000 Maybe turning optimistion off would help (@Apteryks you can try that by editing the _cmd.txt file and manually running gcc), or disabling debug information, or improvements to mrustc's optimisation passes (reducing the size of the C code)

Apteryks commented 2 years ago

Sharing the experiment of another fellow Guix user:

With ./build-1.39.0.sh in mrustc, some help from #mrustc, and some local patches, I was able to get it build under 4GiB of RAM per process with an x86_64 kernel (by removing -O2 and -g), but it fails for 3GiB of RAM per process (i686 kernel) no mater what options are passed to it.

This suggests playing with GCC flags alone won't be enough to have it built on i686.

thepowersgang commented 2 years ago

Maybe some sort of splitting up of the source file would work - would be rather complex (and likely will cause longer compiles for non-32bit platforms). If you're brave, see if there's any low-hanging optimisations that could be added?

GNUtoo commented 1 year ago

I somehow managed to compile it under 3GiB under x86_64 with a 5.19.12 kernel.

I used the following commit:

597593ab Windows build - fix compilation errors on VS2022

I then created a limited memory cgroup with cgroup v2 and this script:

#!/bin/sh
# License: MIT (the same than mrustc)
if [ $# -ne 1 ] ; then
    echo "Usage: $0 <group-name>"
    exit 64 # EX_USAGE
fi
group="$1"
cgcreate -g memory:/${group}
echo 3220176896 > /sys/fs/cgroup/${group}/memory.max
echo cgexec -g memory:/${group} bash

I then build like that:

$ sudo su
# git clean -dfX                     # only needed if building a second time 
# cgdelete memory:/mrustc # only needed if building a second time
# ./setup.sh mrustc
# cgexec -g memory:/${group} bash
# su -l <my-user>
$ ./build-1.39.0.sh

It then fails with the following error:

In file included from [...]/rustc-1.39.0-src/src/llvm-project/llvm/utils/benchmark/src/benchmark_register.cc:15:
[...]/rustc-1.39.0-src/src/llvm-project/llvm/utils/benchmark/src/benchmark_register.h: In function ‘void AddRange(std::vector<T>*, T, T, int)’:
[...]/rustc-1.39.0-src/src/llvm-project/llvm/utils/benchmark/src/benchmark_register.h:17:30: error: ‘numeric_limits’ is not a member of ‘std’
   17 |   static const T kmax = std::numeric_limits<T>::max();
      |                              ^~~~~~~~~~~~~~
[...]/rustc-1.39.0-src/src/llvm-project/llvm/utils/benchmark/src/benchmark_register.h:17:46: error: expected primary-expression before ‘>’ token
   17 |   static const T kmax = std::numeric_limits<T>::max();
      |                                              ^
[...]/rustc-1.39.0-src/src/llvm-project/llvm/utils/benchmark/src/benchmark_register.h:17:49: error: ‘::max’ has not been declared; did you mean ‘std::max’?
   17 |   static const T kmax = std::numeric_limits<T>::max();
      |                                                 ^~~
      |                                                 std::max
In file included from /usr/include/c++/12.2.0/algorithm:61,
                 from [...]/rustc-1.39.0-src/src/llvm-project/llvm/utils/benchmark/include/benchmark/benchmark.h:175,
                 from [...]/rustc-1.39.0-src/src/llvm-project/llvm/utils/benchmark/src/internal_macros.h:4,
                 from [...]/rustc-1.39.0-src/src/llvm-project/llvm/utils/benchmark/src/check.h:8,
                 from [...]/rustc-1.39.0-src/src/llvm-project/llvm/utils/benchmark/src/benchmark_register.h:6:
/usr/include/c++/12.2.0/bits/stl_algo.h:5756:5: note: ‘std::max’ declared here
 5756 |     max(initializer_list<_Tp> __l, _Compare __comp)
      |     ^~~
make[3]: *** [utils/benchmark/src/CMakeFiles/benchmark.dir/build.make:90: utils/benchmark/src/CMakeFiles/benchmark.dir/benchmark_register.cc.o] Error 1
make[3]: Leaving directory '[...]/rustc-1.39.0-src/build'
make[2]: *** [CMakeFiles/Makefile2:17516: utils/benchmark/src/CMakeFiles/benchmark.dir/all] Error 2
make[2]: Leaving directory '[...]/rustc-1.39.0-src/build'
make[1]: *** [Makefile:156: all] Error 2
make[1]: Leaving directory '[...]/rustc-1.39.0-src/build'
make: *** [minicargo.mk:251: rustc-1.39.0-src/build/bin/llvm-config] Error 2

In the same shell I then re-launch the build again:

$ ./build-1.39.0.sh

And at the end it finishes.

Now, cgroups v2 doesn't impose any hard limit on the memory usage, so we need to validate that it was under 3GiB and for that we need a recent kernel to get the memory.peak file in sysfs which records the peak memory used.

I've written a naive script to check that:

#!/bin/env python3
# License: MIT: the same than mrustc
import os
import sys

KiB = 1024
MiB = 1024 * KiB
GiB = 1024 * MiB

def read_int(group, filename):
    f = open('/sys/fs/cgroup/{}/{}'.format(group, filename), 'r')
    content = int(f.readline().replace('\n', ''))
    f.close()
    return content

def print_memory(string, memory):
    sign = ''
    negative = memory < 0

    if negative:
        memory = -memory
        sign = '-'

    if memory > KiB:
        print('{}: {}{} B:'.format(string, sign, memory), end='')
    else:
        print('{}: {}{} B'.format(string, sign, memory))
        return

    nr_gib = memory // GiB
    if nr_gib > 0:
        print (' {}{} GiB'.format(sign, nr_gib), end='')

    remaining = memory % GiB
    if remaining == 0:
        print()
        return
    else:
        nr_mib = remaining // MiB
        if nr_mib > 0:
            print (' {}{} MiB'.format(sign, nr_mib), end='')
            remaining = remaining % MiB

    if remaining == 0:
        print()
        return
    else:
        nr_kib = remaining // KiB
        if nr_kib > 0:
            print (' {}{} KiB'.format(sign, nr_kib), end='')
            remaining = remaining % KiB

    if remaining == 0:
        print()
        return
    else:
        print (' {}{} B'.format(sign, remaining), end='')
        print()

if len(sys.argv) < 2:
    print('Usage: {} <group>'.format(sys.argv[0]))
else:
    group = sys.argv[1]
    print("group: {}".format(group))
    max_memory = read_int(group, 'memory.max')
    print_memory('max memory', max_memory)

    peak_memory = read_int(group, 'memory.peak')
    print_memory('peak memory', peak_memory)

    print_memory('max_memory - peak_memory', max_memory - peak_memory)

And with the settings above at the end of the build we have:

# ./status.py mrustc
group: mrustc
max memory: 3220176896 B: 2 GiB 1023 MiB
peak memory: 3220332544 B: 2 GiB 1023 MiB 152 KiB
max_memory - peak_memory: -155648 B: -152 KiB

So that means that somehow with cgroup and with restarting the build we can build it under 3GiB.

I'm not sure why it works better with cgroups than without. On i686 without cgroup it just fails to build, but I should probably try again and restart the build again when it fails.

The bonus here is that (given a recent kernel) we now have a way to track the maximum memory usage. If I give it 4GiB, it somehow goes over the 4GiB but again I need to try again to restart the build to understand if we can just workaround by assuming that the build will fail and restart it a given number of times until it works.

edit1: fixed some (markdown?) formatting