Cannot build it with lots of cores.

[onefang]

I've been trying and failing to build cosmopolitan under Devuan Chimaera, which is a Debian based Linux distribution without systemd. I do notice that you keep mentioning GNU/systemd instead of Linux, I was thinking that was some sort of joke, but maybe my problem is the lack of systemd?

I have tried git head, and the source tarball, same result.

make -j1 -O
♥cosmo
error: clone(0) failed EAGAIN/11/Try again

`make MODE= -j128 o//depend` exited with 1:
build/bootstrap/mkdeps.com -o o//depend -r o// @o//srcs.txt @o//hdrs.txt @o//incs.txt
consumed 561µs wall time
ballooned to 160kb in size
needed 523us cpu (0% kernel)
caused 18 page faults (100% memcpy)

x86_64-linux-musl-gcc: fatal error: cannot execute '/media/DATA/home/dvs1/src/APE_Cosmo_STUFFS/cosmopolitan_ORIGINAL_BUILD/o/third_party/gcc/bin/../libexec/gcc/x86_64-linux-musl/9.2.0/cc1': vfork: Resource temporarily unavailable
compilation terminated.

`make MODE= -j128 o/ape/config.h.ok` exited with 1:
o/third_party/gcc/bin/x86_64-linux-musl-gcc -S -Wall -Werror -fdebug-prefix-map=/media/DATA/home/dvs1/src/APE_Cosmo_STUFFS/cosmopolitan_ORIGINAL_BUILD= -frecord-gcc-switches -fno-schedule-insns2 -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer -DSYSDEBUG -Og -DCOSMO -DMODE="" -DIMAGE_BASE_VIRTUAL=0x400000 -nostdinc -iquote. -includelibc/integral/normalize.inc -msse3 -fno-math-errno -fno-trapping-math -fno-fp-int-builtin-inexact -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -std=gnu2x -xc -g0 -o o/ape/config.h.ok ape/config.h -fdiagnostics-color=always -pg -D__PG__ -mno-red-zone -D__MNO_RED_ZONE__ -fno-omit-frame-pointer -D__FNO_OMIT_FRAME_POINTER__
consumed 15,001,251µs wall time
ballooned to 1,148kb in size
needed 884us cpu (0% kernel)
caused 49 page faults (100% memcpy)
5 context switches (100% consensual)

make: *** [build/rules.mk:32: o/ape/config.h.ok] Error 1

The number of "vfork: Resource temporarily unavailable" sections I get depends on how many -jx I use. Despite giving specific -jx, there's always the "make MODE= -j128". Yes I have a Ryzen Threadripper with 64 cores, so CPU counting ends up at 128. Do I have too many cores, and something is tripping up on that? It seems to be running out of cores. lol

I notice there's no way of building completely from source, otherwise I would hack away at the source code until it worked, but the build system tries to use it's own pre compiled build tools that come with the source code, and it's this build system binary blob that is failing. So I can't fix this myself with out lots of work essentially rebuilding things from the ground up one step at a time.

Redbean worked fine, which is the thing that attracted me to cosmo in the first place. I want to try redbean out as the basis for my OpenSim rewrite (3D virtual world metaverse that has been around for over a decade).

[onefang]

BTW, I did spend hours reading through the other issues and pull requests to see if there was any clues. binfmt_misc was mentioned often, which might have been an issue since OpenSim is written in C#, which means I have Mono installed, which adds cli in binfmt_misc.

This was always mentioned -

sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"

My system has the binfmt_misc kernel module running, but there's no /proc/sys. My kernel is 5.16.0-0.bpo.4-amd64 #1 SMP PREEMPT Debian 5.16.12-1~bpo11+1.

I used "update-binfmts --disable" to disable all the binfmt entries, no difference.

[jart]

System Five Hundred is an abomination and therefore not a requirement. The issue you're likely encountering is the build system imposes quotas in order to prevent something like a fork() bomb in the Python unit tests from nuking your system.

https://github.com/jart/cosmopolitan/blob/206f0731815280387a2001e09550aee48a1cba5c/tool/build/compile.c#L745-L749

The unfortunate thing about RLIMIT_NPROC is it applies user-wide so if you've already got a lot of processes running and you choose make -j128 then it's likely to exceed quota.

Could you please try something for me?

diff --git a/build/definitions.mk b/build/definitions.mk
index 6274df0b3..5965d88aa 100644
--- a/build/definitions.mk
+++ b/build/definitions.mk
@@ -63,7 +63,7 @@ PKG = build/bootstrap/package.com
 MKDEPS = build/bootstrap/mkdeps.com
 ZIPOBJ = build/bootstrap/zipobj.com
 MKDIR = build/bootstrap/mkdir.com -p
-COMPILE = build/bootstrap/compile.com -V9 $(QUOTA)
+COMPILE = build/bootstrap/compile.com -V9 -P2048 $(QUOTA)

 COMMA := ,
 PWD := $(shell build/bootstrap/pwd.com)

If that solves the issue for you, then I'll commit it.

[onefang]

I'm not choosing -j128, you can see in my original message that I'm doing -j1, it's your build system that's saying "make MODE= -j128 o//depend". I had tried other -j numbers, including the -j8 your instructions say to use, same problem. From what I saw in the source code, it's counting my CPUs to get that 128.

I'm just about to start cooking dinner, I'll try that patch after I've eaten.

[jart]

It's possible this is related to the mkdeps tool spawning 128 threads? It's programmed to spawn as many threads as there are CPUs. That's the only build tool I know of that uses threads.

[jart]

It might also help if you post your ulimit -a

[onefang]

With your patch on the release tarball version, didn't make any difference.

ulimit -a
real-time non-blocking time  (microseconds, -R) unlimited
core file size              (blocks, -c) 0
data seg size               (kbytes, -d) unlimited
scheduling priority                 (-e) 0
file size                   (blocks, -f) unlimited
pending signals                     (-i) 1030282
max locked memory           (kbytes, -l) 8192
max memory size             (kbytes, -m) unlimited
open files                          (-n) 1024
pipe size                (512 bytes, -p) 8
POSIX message queues         (bytes, -q) 819200
real-time priority                  (-r) 0
stack size                  (kbytes, -s) 8192
cpu time                   (seconds, -t) unlimited
max user processes                  (-u) 1030282
virtual memory              (kbytes, -v) unlimited
file locks                          (-x) unlimited

The number of times I see the ".... cc1': vfork: Resource temporarily unavailable" message is always the number I put after -j. So -j1 shows the error once, -j2 shows it twice. I'm guessing it's using up all cores THEN trying to use as many as specified in the -j. With no -jX I see the error once. Makes me wish I could do -j0, but "make: the '-j' option requires a positive integer argument". DOH!

If I could compile it from source without using your pre-built tools, I could change the threads = GetCpuCount(); line in mkdeps.c to halve the result. That would be a quick test of your new theory.

[onefang]

Thought I could be clever with that first error in the output -

make MODE= -j12 o//depend
♥cosmo
error: clone(0) failed EAGAIN/11/Try again

`make MODE= -j128 o//depend` exited with 1:
build/bootstrap/mkdeps.com -o o//depend -r o// @o//srcs.txt @o//hdrs.txt @o//incs.txt
consumed 426µs wall time
ballooned to 160kb in size
needed 384us cpu (0% kernel)
caused 19 page faults (100% memcpy)

make: *** No rule to make target 'o//depend'.  Stop.

Nope, still tries to do -j128.

[onefang]

Soooo, maybe change mkdep to "the user said -j8 for a reason, let's not ignore that, and just do what they said".

[onefang]

I'm reminded of the MONO build tool. You can tell it to use more cores, but it takes longer coz it pauses a long time while it does .. something ... with 128 cores. I'm guessing one core at a time, since the more cores you tell it to use, the longer that initial pause is. lol

[jart]

Quick update. This issue may be blocked on https://github.com/jart/cosmopolitan/pull/430 over the next few days, due to a lack of configurability in the RLIMIT_NPROC quota.

If I could compile it from source without using your pre-built tools, I could change the threads = GetCpuCount(); line in mkdeps.c to halve the result. That would be a quick test of your new theory

Here's how we can test the theory. I've made the following change locally:

diff --git a/tool/build/mkdeps.c b/tool/build/mkdeps.c
index eefb4dfa3..df798b04a 100644
--- a/tool/build/mkdeps.c
+++ b/tool/build/mkdeps.c
@@ -442,7 +442,7 @@ int main(int argc, char *argv[]) {
   char path[PATH_MAX];
   if (argc == 2 && !strcmp(argv[1], "-n")) exit(0);
   GetOpts(argc, argv);
-  threads = GetCpuCount();
+  threads = 1;
   tls = calloc(threads, sizeof(*tls));
   stack = calloc(threads, sizeof(*stack));
   bouts = calloc(threads, sizeof(*bouts));

There's a prebuilt binary available with the above change at https://justine.lol/cosmopolitan/mkdeps-zero-threads.com which you can try. This will only spawn a single thread. Let me know if that fixes things for you.

[tgbugs]

For the record the threads = 1 in mkdeps.c probably won't work. I tried that.

edit: I also always see make -j8 in my error logs and could never track down how 8 was getting passed to make again, but it was a red herring.

[tgbugs]

@onefang you should be able to build corrected binaries if you do it as another user on the system that doesn't have more than 1024 processes running (given the 128 you would probably want to to be below 800 just in case). Alternately, if you can get the number of processes running as your user well under 1024 (e.g. by closing firefox as I did) then you can build binaries with the fix for the quota from #430.

On a related note: it would be nice if there was an alternate way to bootstrap the build/bootstrap/*.com files in a way that avoided the circularity we run into here. I can open a separate issue related to that. Hopefully there won't be similar issues in the future, but if there are, then it would be nice to have an escape hatch when the system has e.g. a whole bunch of fully functional toolchains that could help.

[jart]

On a related note: it would be nice if there was an alternate way to bootstrap the build/bootstrap/*.com files in a way that avoided the circularity we run into here.

The recommended approach would be to spin up a VM and build them there, or as a different user as you suggested. The build bootstrap folder is intended only for commands that have a "chicken and egg problem" because the tools are required to compile the tools themselves.

The only tool where that isn't the case is build/bootstrap/ape.elf which completely freestanding and written in a portable dialect that can be compiled independently of the Makefile regime. You may read its source code in https://github.com/jart/cosmopolitan/blob/master/ape/loader.c and note the fact that this is a 4 kilobyte program. The other tools, such as package.com and zipobj.com which weigh in at a whopping 100+kb are obviously much too complicated to be written in this manner, and as such, need to be be bootstrapped.

[jart]

I've just doubled the default process quota to 2048 and rebuilt compile.com. I'm hoping that should be sufficient for your threadrippers. I'd bump it up to 4096 except people build this repo on low power laptops too, so I'm trying to walk a fine line here. The good news, is that thanks to the bug fix in https://github.com/jart/cosmopolitan/pull/430 your build process quota is now configurable using the -P option as described above. I believe that fixes the issue. If there's anything else, let me know and I'll re-open.

[tgbugs]

I suspect that there is the remaining issue of QUOTA not inheriting via QUOTA := $(QUOTA) ... in the various *.mk files that will pop up. I don't know if there is a simpler way to fix that than to do it manually?

The affected files are:

third_party/python/python.mk
third_party/sqlite3/sqlite3.mk
third_party/chibicc/test/test.mk
third_party/quickjs/quickjs.mk
test/tool/plinko/test.mk

[jart]

Did you try locally editing build/definitions.mk to change the COMPILE = .... line? If that works, we can add an additional variable for offering an iron-clad guarantee the quota will be applied to all build rules.

[tgbugs]

Yes, putting -P4096 on the COMPILE = line before $(QUOTA) works, so it should be possible to expand it there I think?

[onefang]

I did a git pull. Then make -j8 -O which took 1 minute 1 seconds. Then make -j128 -O, 26 seconds. Then make -O, which 6 minutes 43 seconds later spit out the "make MODE= -j128 o//third_party/python/Lib/test/test_os.py.runs` exited with 1" message after three errors from 'File "/zip/.python/test/test_os.py", line 1140, in setUp' and one from 'File "/zip/.python/test/test_os.py", line 2870, in test_path_t_converter'. I started from fresh copies of the git repo each time.

Looks like it worked. Thousands of lines of output, but no red error messages. I'll read through the test output to double check none failed, and report back later. I was very amused by -

test_concat (__main__.BytesTest) ... skipped 'not enough memory: 8589934592.0G minimum needed'

Damn, I only have 256.0G of memory. What did you say about low power laptops? 256.0G is all this motherboard will hold, and I'm on a pension, couldn't afford that much RAM even if I could find a box to put it all in. lol

Thanks for helping out. Now I get to play with it, see if I can bend it to my will. Muahahaha!

BTW, I'm very interested in getting LuaJIT working with this, so I'm keeping an eye on the effort to do that.

[ahgamut]

@onefang refer to #272, LuaJIT can be built as a standalone executable. Adding to redbean can also be done, I think.

[onefang]

@ahgamut #272 was already open in a tab and read before I opened this ticket, that's what I meant by "keeping an eye on the effort". So now that I can actually compile cosmo and redbean myself, I can try the things mentioned in #272. B-)

I already build LuaJIT into an existing project of mine that I'm looking at using redbean for. Currently it uses FCGI to work with a variety of web servers.

Thanks for your efforts to get that working.

[onefang]

So, reviewing the output of make -j128 -O. I hope I provided enough context.

TL,DR: no errors, but lots of skips that are questionable, including lots of "not enough memory", some expecting me to have billions of GB of RAM, some thinking 256 GB isn't enough for a test that needs 32GB or less. Some saying I don't have POSIX or Linux. This is a Linux desktop, it should be POSIX compliant.

Near the end of the build section, it's only a warning, so probably not important.

     1,107⏰      1,068⏳   1,220k      24iop o//test/tool/net/sqlite_test.com                                                                                                                                                                                                                                                                                                                                                                                                                                                 
o/third_party/gcc/bin/x86_64-linux-musl-ld.bfd -static -nostdlib -melf_x86_64 --gc-sections --build-id=none --no-dynamic-linker -zmax-page-size=0x1000    o//ape/ape-no-modify-self.o -T o//ape/ape.lds o//libc/crt/crt.o o//examples/pylife/pylife.o o//third_party/python/python-stage2.a o//third_party/sqlite3/libsqlite3.a o//third_party/python/python-stage1.a o//tool/args/args.a o//third_party/getopt/getopt.a o//tool/build/lib/buildlib.a o//dsp/scale/scale.a o//third_party/musl/musl.a o//third_party/linenoise/
linenoise.a o//third_party/bzip2/bzip2.a o//net/https/https.a o//third_party/mbedtls/mbedtls.a o//net/http/http.a o//libc/x/x.a o//libc/log/log.a o//libc/dns/dns.a o//libc/sock/sock.a o//libc/unicode/unicode.a o//libc/time/time.a o//libc/zipos/zipos.a o//libc/stdio/stdio.a o//third_party/gdtoa/gdtoa.a o//libc/nt/MsWSock.a o//libc/alg/alg.a o//dsp/core/core.a o//libc/mem/mem.a o//third_party/dlmalloc/dlmalloc.a o//libc/runtime/runtime.a o//third_party/xed/xed.a o//third_party/zlib/zlib.a o//libc/elf/elf.a o
//libc/bits/bits.a o//libc/rand/rand.a o//libc/calls/syscalls.a o//libc/nt/ws2_32.a o//libc/nt/powrprof.a o//libc/nt/psapi.a o//libc/nt/pdh.a o//libc/nt/ntdll.a o//libc/nt/iphlpapi.a o//libc/nt/advapi32.a o//libc/fmt/fmt.a o//libc/str/str.a o//third_party/compiler_rt/compiler_rt.a o//libc/tinymath/tinymath.a o//libc/intrin/intrin.a o//libc/nt/kernel32.a o//libc/sysv/calls.a o//libc/sysv/sysv.a o//libc/nexgen32e/nexgen32e.a o//libc/stubs/stubs.a -o o//examples/pylife/pylife.com.dbg
o/third_party/gcc/bin/x86_64-linux-musl-ld.bfd: o//examples/pylife/pylife.com.dbg: warning: allocated section `.tdata' not in segment

The bit that amused me at the very end of the build section -

test_concat (__main__.BytesTest) ... skipped 'not enough memory: 8589934592.0G minimum needed'
test_optimized_concat (__main__.BytesTest) ... skipped 'not enough memory: 8589934592.0G minimum needed'
test_repeat (__main__.BytesTest) ... skipped 'not enough memory: 8589934592.0G minimum needed'
test_concat (__main__.StrTest) ... skipped 'not enough memory: 8589934592.0G minimum needed'
test_optimized_concat (__main__.StrTest) ... skipped 'not enough memory: 8589934592.0G minimum needed'
test_repeat (__main__.StrTest) ... skipped 'not enough memory: 8589934592.0G minimum needed'

    89,506⏰     89,450⏳  18,724k      16iop o//third_party/python/pythontester.com -m test.test_bool
    95,845⏰     95,157⏳  18,816k       0iop o//third_party/python/pythontester.com -m test.test_bisect
    58,947⏰     58,834⏳  14,888k      24iop o//third_party/python/pythontester.com -m test.test_defaultdict
    69,187⏰     69,116⏳  14,380k       0iop o//third_party/python/pythontester.com -m test.test_copy
    94,597⏰     94,514⏳  18,788k       0iop o//third_party/python/pythontester.com -m test.test_call
    65,991⏰     65,918⏳  14,328k       0iop o//third_party/python/pythontester.com -m test.test_decorators
   112,441⏰    111,795⏳  19,984k       0iop o//third_party/python/pythontester.com -m test.test_base64
    77,750⏰     77,964⏳  14,788k      56iop o//third_party/python/pythontester.com -m test.test_cosmo
    65,858⏰     65,773⏳  14,256k       0iop o//third_party/python/pythontester.com -m test.test_dictcomps
    65,898⏰     65,767⏳  14,460k       0iop o//third_party/python/pythontester.com -m test.test_dictviews
    65,812⏰     65,692⏳  14,276k       0iop o//third_party/python/pythontester.com -m test.test_dynamicclassattribute
test_from_2G_generator (__main__.TupleTest) ... skipped 'test needs much more memory than advertised, see issue5438'
test_from_almost_2G_generator (__main__.TupleTest) ... skipped 'test needs much more memory than advertised, see issue5438'

   162,878⏰    163,292⏳  20,028k      32iop o//third_party/python/pythontester.com -m test.test_cprofile
test_fcntl_64_bit (__main__.TestFcntl) ... skipped 'F_NOTIFY or DN_MULTISHOT unavailable'

A couple of "... skipped 'TODO: find out why this fails'" which I'll ignore.

"this test sucks" lol

OK (skipped=2)
doctest (test.test_generators) ... 283 tests with zero failures
   232,182⏰    232,050⏳  22,804k       8iop o//third_party/python/pythontester.com -m test.test_generators
    76,509⏰     76,230⏳  14,464k       0iop o//third_party/python/pythontester.com -m test.test_secrets
test_poll2 (__main__.PollTests) ... skipped '[jart] this test sucks'
test_poll_blocks_with_negative_ms (__main__.PollTests) ... skipped 'Threading required for this test.'
test_threaded_poll (__main__.PollTests) ... skipped 'Threading required for this test.'

These surprised me, a bunch of POSIX things skipped? I didn't dig deeper, but some of those look like things an ordinary Debian Linux should have, just off the top of my head. The skipped coz not 32 bit things are no surprise, this is a 64 bit Ryzen Threadripper.

   186,259⏰    176,656⏳  19,712k      24iop o//third_party/python/pythontester.com -m test.test_optparse
test_chdir (__main__.PosixTester) ... skipped 'test needs os.getgroups()'
test_chflags (__main__.PosixTester) ... skipped 'test needs os.chflags()'
test_chown (__main__.PosixTester) ... skipped 'test needs os.getgroups()'
test_confstr (__main__.PosixTester) ... skipped 'test needs posix.confstr()'
test_fchown (__main__.PosixTester) ... skipped 'test needs os.getgroups()'
test_fexecve (__main__.PosixTester) ... skipped 'test needs execve() to support the fd parameter'
test_fs_holes (__main__.PosixTester) ... skipped 'test needs an OS that reports file holes'
test_fstatvfs (__main__.PosixTester) ... skipped 'test needs posix.fstatvfs()'
test_get_and_set_scheduler_and_param (__main__.PosixTester) ... skipped 'requires POSIX scheduler API'
test_getgrouplist (__main__.PosixTester) ... skipped 'test needs posix.getgrouplist()'
test_getgroups (__main__.PosixTester) ... skipped 'test needs os.getgroups()'
test_initgroups (__main__.PosixTester) ... skipped 'test needs os.initgroups()'
test_lchflags_regular_file (__main__.PosixTester) ... skipped 'test needs os.lchflags()'
test_lchflags_symlink (__main__.PosixTester) ... skipped 'test needs os.lchflags()'
test_lchown (__main__.PosixTester) ... skipped 'test needs os.getgroups()'
test_listdir (__main__.PosixTester) ... skipped ''
test_listdir_bytes (__main__.PosixTester) ... skipped ''
test_listdir_bytes_like (__main__.PosixTester) ... skipped ''
test_listdir_default (__main__.PosixTester) ... skipped ''
test_listdir_fd (__main__.PosixTester) ... skipped ''
test_lockf (__main__.PosixTester) ... skipped 'test needs posix.lockf()'
test_mkfifo_dir_fd (__main__.PosixTester) ... skipped ''                                                                       
test_osexlock (__main__.PosixTester) ... skipped 'test needs posix.O_EXLOCK'
test_osshlock (__main__.PosixTester) ... skipped 'test needs posix.O_SHLOCK'
test_posix_fadvise_errno (__main__.PosixTester) ... skipped 'TODO: why does this fail'
test_posix_fallocate (__main__.PosixTester) ... skipped 'test needs posix.posix_fallocate()'
test_posix_fallocate_errno (__main__.PosixTester) ... skipped 'test needs posix.posix_fallocate()'
test_sched_getaffinity (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_sched_priority (__main__.PosixTester) ... skipped 'requires sched_get_priority_max()'
test_sched_rr_get_interval (__main__.PosixTester) ... skipped 'no function'
test_sched_setaffinity (__main__.PosixTester) ... skipped "don't have sched affinity support"
test_statvfs (__main__.PosixTester) ... skipped 'test needs posix.statvfs()'
test_utime_with_fd (__main__.PosixTester) ... skipped 'TODO: why does this fail'
test_waitid (__main__.PosixTester) ... skipped 'test needs posix.waitid()'

Some more surprises. Skipped coz my Linux system isn't Linux?

   220,774⏰    216,623⏳  22,564k     640iop o//third_party/python/pythontester.com -m test.test_sax
   280,432⏰    275,538⏳  19,916k       0iop o//third_party/python/pythontester.com -m test.test_ordered_dict
test_fsize_ismax (__main__.ResourceTest) ... skipped 'RLIM_INFINITY is -1, expected positive value'
test_linux_constants (__main__.ResourceTest) ... skipped 'test requires Linux'
test_prlimit (__main__.ResourceTest) ... skipped 'no prlimit'
test_prlimit_refcount (__main__.ResourceTest) ... skipped 'no prlimit'

A bunch of "skipped 'requires APE debug build'" which is fine, this isn't a debug build. Happy to do that if you want.

Are these expected?

    416,533⏰    372,841⏳  21,960k     416iop o//third_party/python/pythontester.com -m test.test_gzip
   507,789⏰    507,564⏳  22,496k       0iop o//third_party/python/pythontester.com -m test.test_codecmaps_jp
   281,442⏰    281,920⏳  20,536k       0iop o//third_party/python/pythontester.com -m test.test_script_helper
   147,172⏰    147,061⏳  14,656k       0iop o//third_party/python/pythontester.com -m test.test_userdict
   143,256⏰    141,154⏳  19,236k     736iop o//third_party/python/pythontester.com -m test.test_wave
   134,225⏰    131,282⏳  16,472k     200iop o//third_party/python/pythontester.com -m test.test_zipapp
   455,042⏰    451,197⏳  34,124k       0iop o//third_party/python/pythontester.com -m test.test_fstring
   128,922⏰    128,557⏳  20,528k   5,080iop ld.bfd ape-no-modify-self.o -T o//ape/ape.lds crt.o int128_test.chibicc2.o test2.a chibicc.a buildlib.a https.a mbedtls.a http.a dns.a sock.a MsWSock.a log.a time.a zipos.a x.a unicode.a stdio.a gdtoa.a alg...
   156,537⏰    155,683⏳  20,588k       0iop o//third_party/python/pythontester.com -m test.test_uuid
     2,595⏰      2,556⏳   2,280k   2,136iop o/third_party/gcc/bin/x86_64-linux-musl-objcopy -S -O binary o//third_party/chibicc/test/int128_test2.com.dbg o//third_party/chibicc/test/int128_test2.com
   207,293⏰    207,034⏳  22,448k       8iop o//third_party/python/pythontester.com -m test.test_tokenize
       304⏰        269⏳     976k       0iop o//third_party/chibicc/test/int128_test2.com
got b'HTTP/1.0 500 Internal Server Error\r\nDate: Fri, 24 Jun 2022 18:10:09 GMT\r\nServer: WSGIServer/0.2 CPython/3.6.14+\r\nContent-Type: text/plain\r\nContent-Length: 59\r\n\r\nA server error occurred.  Please contact the administrator.'
got b'HTTP/1.0 500 Internal Server Error\r\nDate: Fri, 24 Jun 2022 18:10:09 GMT\r\nServer: WSGIServer/0.2 CPython/3.6.14+\r\nContent-Type: text/plain\r\nContent-Length: 59\r\n\r\nA server error occurred.  Please contact the administrator.'
got b'HTTP/1.0 500 Internal Server Error\r\nDate: Fri, 24 Jun 2022 18:10:09 GMT\r\nServer: WSGIServer/0.2 CPython/3.6.14+\r\nContent-Type: text/plain\r\nContent-Length: 59\r\n\r\nA server error occurred.  Please contact the administrator.'

For these I'm not familar with the things it says it needs but don't have.

doctest (collections) ... 66 tests with zero failures
   529,800⏰    529,580⏳  23,552k      16iop o//third_party/python/pythontester.com -m test.test_collections
   159,087⏰    158,646⏳  20,300k       0iop o//third_party/python/pythontester.com -m test.test_yield_from
   305,311⏰    305,036⏳  21,760k      40iop o//third_party/python/pythontester.com -m test.test_site
   377,612⏰    367,925⏳  21,160k   3,232iop o//third_party/python/pythontester.com -m test.test_pathlib
   575,150⏰    574,738⏳  20,200k       0iop o//third_party/python/pythontester.com -m test.test_codecencodings_jp
   402,778⏰    368,700⏳  49,464k      24iop o//third_party/python/pythontester.com -m test.test_plistlib
   613,768⏰    569,758⏳  20,384k     208iop o//third_party/python/pythontester.com -m test.test_array
test_select_interrupt_noraise (__main__.DefaultSelectorTestCase) ... skipped '[jart] unacceptable test'
test_timeout (__main__.DefaultSelectorTestCase) ... skipped '[jart] unacceptable test'
test_select_interrupt_noraise (__main__.SelectSelectorTestCase) ... skipped '[jart] unacceptable test'
test_timeout (__main__.SelectSelectorTestCase) ... skipped '[jart] unacceptable test'
test_select_interrupt_noraise (__main__.PollSelectorTestCase) ... skipped '[jart] unacceptable test'
test_timeout (__main__.PollSelectorTestCase) ... skipped '[jart] unacceptable test'
test_select_interrupt_noraise (__main__.EpollSelectorTestCase) ... skipped '[jart] unacceptable test'
test_above_fd_setsize (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_close (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_context_manager (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_empty_select (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_fileno (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_get_key (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_get_map (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_modify (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_register (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_register_bad_fd (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_select (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_select_interrupt_exc (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_select_interrupt_noraise (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_selector (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_timeout (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_unregister (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_unregister_after_fd_close (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_unregister_after_fd_close_and_reuse (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_unregister_after_socket_close (__main__.KqueueSelectorTestCase) ... skipped 'Test needs selectors.KqueueSelector)'
test_above_fd_setsize (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_close (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_context_manager (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_empty_select (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_fileno (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_get_key (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_get_map (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_modify (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_register (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_select (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_select_interrupt_exc (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_select_interrupt_noraise (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_selector (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_timeout (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_unregister (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_unregister_after_fd_close (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_unregister_after_fd_close_and_reuse (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'
test_unregister_after_socket_close (__main__.DevpollSelectorTestCase) ... skipped 'Test needs selectors.DevpollSelector'

This one has a few skipped coz not Windoze, so fair enough, but the rest I should have with ext4 file system.

OK (skipped=1)
restoring automatic collection
   523,202⏰    518,173⏳  19,760k      40iop o//third_party/python/pythontester.com -m test.test_gc
test_resize_past_pos (__main__.MmapTests) ... skipped 'resizing not supported'
test_around_2GB (__main__.LargeMmapTests) ... skipped 'filesystem does not have largefile support'
test_around_4GB (__main__.LargeMmapTests) ... skipped 'filesystem does not have largefile support'
test_large_filesize (__main__.LargeMmapTests) ... skipped 'filesystem does not have largefile support'
test_large_offset (__main__.LargeMmapTests) ... skipped 'filesystem does not have largefile support'

OK, this memory test is even more confused than the others. lol

   615,357⏰    615,123⏳  24,004k      56iop o//third_party/python/pythontester.com -m test.test_http_cookiejar
   630,048⏰    632,625⏳  19,284k       0iop o//third_party/python/pythontester.com -m test.test_hash
   446,361⏰    444,397⏳  19,080k       0iop o//third_party/python/pythontester.com -m test.test_struct
test_length_overflow (__main__.MiscTests) ... skipped 'not enough memory: 2.0G minimum needed'

   789,601⏰    789,096⏳  23,288k       8iop o//third_party/python/pythontester.com -m test.test_difflib
   764,831⏰    770,723⏳  26,632k      48iop o//third_party/python/pythontester.com -m test.test_faulthandler
   565,413⏰    565,147⏳  17,848k       0iop o//third_party/python/pythontester.com -m test.test_statistics
   605,930⏰     64,930⏳  14,156k       0iop o//third_party/python/pythontester.com -m test.test_sched
   463,475⏰    461,972⏳  52,644k       0iop o//third_party/python/pythontester.com -m test.test_zlib
   527,059⏰    526,885⏳  56,404k       0iop o//third_party/python/pythontester.com -m test.test_tuple
   567,737⏰    520,373⏳  20,792k     520iop o//third_party/python/pythontester.com -m test.test_tempfile
   693,163⏰    689,124⏳  32,064k       0iop o//third_party/python/pythontester.com -m test.test_re
   912,565⏰    904,988⏳  27,328k     248iop o//third_party/python/pythontester.com -m test.test_codecs
test_getallocatedblocks (__main__.SysModuleTest) ... skipped 'sys.getallocatedblocks unavailable on this build'
test_ioencoding_nonascii (__main__.SysModuleTest) ... skipped 'requires FS encoding to match locale'
test_switchinterval (__main__.SysModuleTest) ... skipped 'Threading required for this test.'
test_thread_info (__main__.SysModuleTest) ... skipped 'Threading required for this test.'
test_objecttypes (__main__.SizeofTest) ... skipped 'alignment of C struct?'

OK (skipped=5)                                                                                                                 
   631,870⏰    638,399⏳  23,664k       8iop o//third_party/python/pythontester.com -m test.test_sys
test_free_after_iterating (__main__.TestSequence) ... skipped "Exhausted deque iterator doesn't free a deque"

Includes "only Windos" and "only Mac OS X" and "only Android". Wait, this runs on Android? Though you didn't have ARM support? I'd love to run this on Android.

doctest (test.test_deque) ... 32 tests with zero failures                                                                      
   924,297⏰    924,002⏳  22,840k      32iop o//third_party/python/pythontester.com -m test.test_deque
test_no_stderr (__main__.CmdLineTest) ... skipped 'test needs POSIX semantics'
test_no_stdin (__main__.CmdLineTest) ... skipped 'test needs POSIX semantics'
test_no_stdout (__main__.CmdLineTest) ... skipped 'test needs POSIX semantics'
test_xoptions (__main__.CmdLineTest) ... skipped 'Cannot run -E tests when PYTHON env vars are required.'

 1,047,329⏰  1,051,295⏳  19,308k      64iop o//third_party/python/pythontester.com -m test.test_cmd_line
 1,124,645⏰    125,267⏳  14,352k       0iop o//third_party/python/pythontester.com -m test.signalinterproctester
   738,815⏰    738,672⏳  19,016k       0iop o//third_party/python/pythontester.com -m test.test_userstring
  1,131,085⏰  1,130,478⏳  20,064k      24iop o//third_party/python/pythontester.com -m test.test_bytes
    819,249⏰    819,061⏳   165,812k       0iop o//third_party/python/pythontester.com -m test.test_unicode
test_bad_getattr (__main__.OptimizedPickleTests) ... skipped 'disabled recursion checking + slow in asan, dbg'

This worried me, I'm in Australia/Brisbane. My system includes Australia/Lord_Howe, but from memory you have your own timezone stuff included in Cosmo's zip file.

`doctest (pickletools) ... 129 tests with zero failures
   995,352⏰    985,013⏳  39,160k      48iop o//third_party/python/pythontester.com -m test.test_pickletools
    815,804⏰    815,629⏳  14,596k       0iop o//third_party/python/pythontester.com -m test.test_urlparse
   966,929⏰    866,616⏳  26,820k       0iop o//third_party/python/pythontester.com -m test.test_random
   977,916⏰    977,011⏳  29,804k      96iop o//third_party/python/pythontester.com -m test.test_set
est_fromtimestamp_lord_howe (test.datetimetester.TestLocalTimeDisambiguation_Fast) ... skipped 'Australia/Lord_Howe timezone is not supported on this platform'
test_timestamp_lord_howe (test.datetimetester.TestLocalTimeDisambiguation_Fast) ... skipped 'Australia/Lord_Howe timezone is not supported on this platform'
test_roundtrip (test.datetimetester.TestSubclassDateTime_Fast) ... skipped 'not appropriate for subclasses'

More CPU and memory confusion.

  1,125,529⏰  1,124,596⏳  19,692k      16iop o//third_party/python/pythontester.com -m test.test_multibytecodec
 1,109,077⏰    107,662⏳  18,688k       0iop o//third_party/python/pythontester.com -m test.test_select
 1,392,034⏰  1,401,133⏳  19,936k       0iop o//third_party/python/pythontester.com -m test.test_calendar
 1,374,989⏰  1,374,264⏳   197,960k       8iop o//third_party/python/pythontester.com -m test.test_email.test_email
test_combinations (__main__.TestBasicOps) ... skipped 'exceeds cpu quota'
test_combinations_overflow (__main__.TestBasicOps) ... skipped 'not enough memory: 8589934592.0G minimum needed'
test_combinations_with_replacement (__main__.TestBasicOps) ... skipped 'exceeds cpu quota'
test_combinations_with_replacement_overflow (__main__.TestBasicOps) ... skipped 'not enough memory: 8589934592.0G minimum needed'
test_combinations_with_replacement_tuple_reuse (__main__.TestBasicOps) ... skipped 'exceeds cpu quota'
test_combinatorics (__main__.TestBasicOps) ... skipped 'exceeds cpu quota'
test_permutations (__main__.TestBasicOps) ... skipped 'exceeds cpu quota'
test_permutations_overflow (__main__.TestBasicOps) ... skipped 'exceeds cpu quota'
test_permutations_tuple_reuse (__main__.TestBasicOps) ... skipped 'exceeds cpu quota'
test_product_overflow (__main__.TestBasicOps) ... skipped 'not enough memory: 8589934592.0G minimum needed'
test_long_chain_of_empty_iterables (__main__.RegressionTests) ... skipped 'exceeds cpu quota'
test_combinations_with_replacement (__main__.TestExamples) ... skipped 'exceeds cpu quota'
test_combinations_sizeof (__main__.SizeofTest) ... skipped 'exceeds cpu quota'
test_combinations_with_replacement_sizeof (__main__.SizeofTest) ... skipped 'exceeds cpu quota'
test_permutations_sizeof (__main__.SizeofTest) ... skipped 'exceeds cpu quota'

One of those looks like a Windoze only test, and I do indeed lack IPv6 on my desktop.

OK (skipped=15)
doctest (__main__) ... 70 tests with zero failures
 1,320,226⏰  1,319,970⏳   200,788k      16iop o//third_party/python/pythontester.com -m test.test_itertools
test_race (__main__.HandlerTest) ... skipped 'Threading required for this test.'
test_noserver (__main__.SocketHandlerTest) ... skipped 'Threading required for this test.'
test_output (__main__.SocketHandlerTest) ... skipped 'Threading required for this test.'
test_output (__main__.DatagramHandlerTest) ... skipped 'Threading required for this test.'
test_listen_config_10_ok (__main__.ConfigDictTest) ... skipped 'Threading required for this test.'
test_listen_config_1_ok (__main__.ConfigDictTest) ... skipped 'Threading required for this test.'
test_listen_verify (__main__.ConfigDictTest) ... skipped 'Threading required for this test.'
test_queue_listener (__main__.QueueHandlerTest) ... skipped 'logging.handlers.QueueListener required for this test'
test_basic (__main__.SMTPHandlerTest) ... skipped 'Threading required for this test.'
test_output (__main__.SysLogHandlerTest) ... skipped 'Threading required for this test.'
test_output (__main__.IPv6SysLogHandlerTest) ... skipped 'IPv6 support required for this test.'
test_output (__main__.HTTPHandlerTest) ... skipped 'Threading required for this test.'
test_basic (__main__.NTEventLogHandlerTest) ... skipped 'win32evtlog/win32evtlogutil/pywintypes required for this test.'
test_noserver (__main__.UnixSocketHandlerTest) ... skipped 'Threading required for this test.'
test_output (__main__.UnixSocketHandlerTest) ... skipped 'Threading required for this test.'
test_output (__main__.UnixDatagramHandlerTest) ... skipped 'Threading required for this test.'
test_output (__main__.UnixSysLogHandlerTest) ... skipped 'Threading required for this test.'

OK (skipped=6)
 1,641,859⏰  1,620,562⏳  19,636k     560iop o//third_party/python/pythontester.com -m test.test_cmd_line_script
  1,569,758⏰  1,569,454⏳  19,660k       0iop o//third_party/python/pythontester.com -m test.test_unicodedata
  1,969,608⏰  1,969,248⏳  35,876k       0iop o//third_party/python/pythontester.com -m test.test_long
 2,154,733⏰     52,498⏳  14,284k       0iop o//third_party/python/pythontester.com -m test.test_epoll
testThreading (__main__.BZ2FileTest) ... skipped 'Threading required for this test.'
test_refleaks_in___init__ (__main__.BZ2DecompressorTest) ... skipped "object <module 'sys' (built-in)> has no attribute 'gettotalrefcount'"

A bunch of 32 bit build skips, an yet more memory skips that shouldn't.

OK (skipped=2)
  2,328,690⏰  2,324,741⏳  72,900k     488iop o//third_party/python/pythontester.com -m test.test_bz2
 2,015,564⏰    215,083⏳  19,908k       0iop o//third_party/python/pythontester.com -m test.test_time
test_bad_getattr (__main__.CPicklerTests) ... skipped 'disabled recursion checking + slow in asan, dbg'
test_bad_getattr (__main__.CDumpPickle_LoadPickle) ... skipped 'disabled recursion checking + slow in asan, dbg'
test_bad_getattr (__main__.DumpPickle_CLoadPickle) ... skipped 'disabled recursion checking + slow in asan, dbg'
test_bad_getattr (__main__.InMemoryPickleTests) ... skipped 'disabled recursion checking + slow in asan, dbg'
test_huge_bytes_32b (__main__.InMemoryPickleTests) ... skipped 'not enough memory: 5.0G minimum needed'
test_huge_bytes_64b (__main__.InMemoryPickleTests) ... skipped 'not enough memory: 10.0G minimum needed'
test_huge_long_32b (__main__.InMemoryPickleTests) ... skipped 'not enough memory: 7.2G minimum needed'
test_huge_str_32b (__main__.InMemoryPickleTests) ... skipped 'not enough memory: 16.0G minimum needed'
test_huge_str_64b (__main__.InMemoryPickleTests) ... skipped 'not enough memory: 32.0G minimum needed'

Some requires build with -DEXTRA_FUNCTIONALITY which I guess the default build doesn't have.

doctest (pickle) ... 14 tests with zero failures
 2,163,908⏰  2,163,322⏳  34,084k      64iop o//third_party/python/pythontester.com -m test.test_pickle
test_wide_char_separator_decimal_point (__main__.CFormatTest) ... skipped "inappropriate decimal point separator ('.' not '\\u066b')"
test_threading (__main__.CThreadingTest) ... skipped 'threading required'

[onefang]

The real test - o/tool/net/redbean-demo.com works and all the pages are OK. B-)

[jart]

including lots of "not enough memory", some expecting me to have billions of GB of RAM

There were several Python unit tests we disabled which required 4GB of RAM which seemed largely intended to test the transition away from the 32-bit era. Once I confirmed they work, I disabled them, because doing things like creating a bunch of 4gb files with gzip and bz2 every time I edit a header in libc/str/ and run make wasn't rocking my world. I don't know why it's reporting that high a number. It's possible it was disabled in a lazy way, by simply having it request an amount of RAM we know certain won't be available.

These surprised me, a bunch of POSIX things skipped?

I love POSIX. The support of Austin Group in helping us change the shell requirements w.r.t. binary is one of the things that made this project possible. We're working on getting there when it comes to functions like setgroups and waitid. Just keep in mind that these kinds of functions haven't been the primary focus, since it's non-obvious how we'll ensure they work on all platforms, and in particular, Windows. But that doesn't mean we shouldn't have them! For example, we have a long tradition of including features that might only for instance be available on FreeBSD (e.g. O_VERIFY) since FreeBSD users should have the ability to fully harness the capabilities of their platform. In other words, it's just a question of time and resources.

Are these expected?

I assume you're running make in one of the verbose modes. By default it captures test output and doesn't show things like that. But when it's shown, the Python tests are configured to show output verbosely. There's a bunch of Python stuff behind if verbose: statements that looks like it's not intended. But it may actually just be the case that it's being verbose and intended to fail.

One of those looks like a Windoze only test, and I do indeed lack IPv6 on my desktop.

No IPv6 yet. Even though IPv6 is something I personally disagree with (note: I am actually a Hurricane Electric certified IPv6 "Sage" funnily enough) I believe IPv6 is something we should support eventually. One of the soft blockers is that it gets hairy on Windows, where, for example, poll() cannot be used in a way that mixes sockets from different address families.

skipped 'filesystem does not have largefile support

That looks wrong. But could by a red herring, since as I mentioned, we don't want a bunch of tests with four gigabytes of disk i/o churn running every time we run make. Unit tests shouldn't have any i/o. What we should do instead, is have a build flag of some kind for running the large tests on an as-needed basis.

Wait, this runs on Android

We've got someone looking into that on the Discord channel. https://discord.gg/mvkhxRaW

test needs POSIX semantics

That probably needs to be fixed. Command line should have POSIX semantics. But it depends on what they mean by that. For example, we currently have _PATH_BSHELL set to CMD.EXE sadly on Windows, until we can find a way to make cocmd.com mostly feature complete w.r.t. posix command spawning. Then we may consider embedding it into _PATH_BSHELL for system().

skipped 'Australia/Lord_Howe timezone is not supported on this platform'

The timezone database is 3.3mb in size. We only embed a subset of it into each binary that calls localtime. See https://redbean.dev/#unix.localtime As far as I can tell all those /usr/share/zoneinfo files are generated from one small file which can be viewed in Eggbert's TZ repo: https://github.com/eggert/tz/blob/main/zone1970.tab It'd be great if we could just embed something like that instead and compile it on the fly.

test_permutations_overflow (main.TestBasicOps) ... skipped 'exceeds cpu quota'

Yeah test.test_email.test_email is another example of poorly written Python unit tests which consume an outrageous amount of resources for a feature very few people are worried about. The disabling of those tests was most likely intentional.

Please keep in mind that while this whole repository builds/tests on your machine in 26 seconds, if you were to run the official Python unit test suite, it would likely take 15 minutes on your machine. The Python devs haven't figured out how to run their own builds/tests in parallel yet.

All we had to do to turn 15 minutes into 26 seconds was disable a small subset of sloppy tests that a huge project like Python accumulated from third party contributors over the years.

The real test - o/tool/net/redbean-demo.com works and all the pages are OK. B-)

Woot.

[onefang]

Messages in syslog during the build / test is worrying though, or are they to be expected?

kernel: [3773914.130227] audit: type=1326 audit(1656094137.040:8): auid=1000 uid=1000 gid=1000 ses=2 subj==unconfined pid=11355 comm="seccomp_test.co" exe="/usr/bin/ape" sig=9 arch=c000003e syscall=257 compat=0 ip=0x42cd65 code=0x0
kernel: [3773983.850788] audit: type=1326 audit(1656094206.763:9): auid=1000 uid=1000 gid=1000 ses=2 subj==unconfined pid=83484 comm="seccomp_test.co" exe="/usr/bin/ape" sig=9 arch=c000003e syscall=257 compat=0 ip=0x42cd65 code=0x0
/test_syslog.pyc: test message from python test_syslog
python: test message from python test_syslog
python: test error from python test_syslog
kernel: [3773986.930246] traps: pythontester.co[91032] trap divide error ip:6ef284 sp:10008021faa0 error:0 in pythontester.com[400000+471000]

[onefang]

I've been a professional computer programmer and sysadmin since the late '70s, I have used 100 programming languages in my career (used to call myself Digital Polyglot), so I think I have earned the right to be a language bigot. I hate any programming language whose name begins with the letter P. Python is one of those, so I don't care about Python support, so wont be running any 15 minute Python test suite.

[jart]

Messages in syslog during the build / test is worrying though, or are they to be expected?

In order to test that our security features work, we need to violate them. That's why you're seeing things like seccomp warnings in your log. We're simply trying to make sure our sandboxing will work for you as advertised. The same applies to things like trapping division errors.

[jart]

I've been a professional computer programmer and sysadmin since the late '70s, I have used 100 programming languages in my career (used to call myself Digital Polyglot), so I think I have earned the right to be a language bigot.

How do you feel about X3J11? One of the things on my bucket list is to convert //third_party/chibicc to an ILP64 data model and adding K&R C support. I'm reasonably certain that there's no reason we shouldn't use classic C syntax (which felt much more like a scripting language without the formalities that exist today) when pointers, ints, and longs all have the same width. Modern CPUs make 64-bit types fast. Depending on the quality of the LP64 code that's written, perhaps even faster. If you share my passion on this topic, then it might be something worth working on.

[jart]

Also I forgot to mention, the most compelling use case for K&R C is it polyglots with JavaScript https://justine.lol/sectorlisp2/#evaluation

[onefang]

X3J11? Standards are good, makes things like cross platform support easy.

I have no particular passions in that area. I do have passions in reducing bloat and increasing speed. Which is why my favourite languages are C and assembler, and why I prefer LuaJIT over plain Lua. LuaJIT tends to beat all other scripting languages in benchmarks.

For web stuff I'm more a "use the old standards, with minimal JavaScript only when needed" kinda guy. But wasm has my interest, I've not actually played with it yet though.

[jart]

I agree. Standards are nice. It's also nice to restore the original visions of clarity that necessitated standardization, now that we no longer need to attract the consensus of companies like UNIVAC.

We recently ported luajit thanks to @ahgamut and we'll likely consider merging it in here so it can be used by redbean. LuaJIT is something that's frequently requested.

[onefang]

Cool, definitely want to see LuaJIT officially in redbean. I'll help test that. I'm in weekend mode now though, I should get some non coding stuff done. lol

[jart]

I've bumped up the default limit to 4096 processes. Enjoy!

[onefang]

Cool, definitely want to see LuaJIT officially in redbean. I'll help test that. I'm in weekend mode now though, I should get some non coding stuff done. lol

Edit: Oops, forgot to actually post this. lol No longer in weekend mode, starting to poke at redbean now.

[onefang]

Sooo, just git pulled and 'make -j128 -O' got errors -

14,306⏰     14,286⏳  18,272k     152iop gcc -Og -c -o o//third_party/python/python.o third_party/python/python.c
   914⏰        882⏳     528k      24iop build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/test/hello.com.zip.o third_party/python/Lib/test/hello.com
   760⏰        726⏳     492k      16iop build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/email/architecture.rst.zip.o third_party/python/Lib/email/architecture.rst
   706⏰        532⏳     484k       8iop build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/venv/scripts/common/activate.zip.o third_party/python/Lib/venv/scripts/common/activate

error:tool/build/lib/elfwriter.c:168: check failed: -1 != -1 errno= 2

make MODE= -j128 o//third_party/python/Lib/venv/scripts/nt/Activate.ps1.zip.o exited with 68: build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/venv/scripts/nt/Activate.ps1.zip.o third_party/python/Lib/venv/scripts/nt/Activate.ps1 consumed 227µs wall time ballooned to 164kb in size needed 189us cpu (0% kernel) caused 17 page faults (100% memcpy)

make: [build/rules.mk:77: o//third_party/python/Lib/venv/scripts/nt/Activate.ps1.zip.o] Error 68 make: Waiting for unfinished jobs.... error:tool/build/lib/elfwriter.c:168: check failed: -1 != -1 errno= 2

make MODE= -j128 o//third_party/python/Lib/venv/scripts/nt/activate.bat.zip.o exited with 68: build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/venv/scripts/nt/activate.bat.zip.o third_party/python/Lib/venv/scripts/nt/activate.bat consumed 378µs wall time ballooned to 156kb in size needed 257us cpu (0% kernel) caused 19 page faults (100% memcpy) 3 context switches (33% consensual)

make: *** [build/rules.mk:77: o//third_party/python/Lib/venv/scripts/nt/activate.bat.zip.o] Error 68 error:tool/build/lib/elfwriter.c:168: check failed: -1 != -1 errno= 2

make MODE= -j128 o//third_party/python/Lib/venv/scripts/nt/deactivate.bat.zip.o exited with 68: build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/venv/scripts/nt/deactivate.bat.zip.o third_party/python/Lib/venv/scripts/nt/deactivate.bat consumed 216µs wall time ballooned to 164kb in size needed 179us cpu (0% kernel) caused 17 page faults (100% memcpy)

make: *** [build/rules.mk:77: o//third_party/python/Lib/venv/scripts/nt/deactivate.bat.zip.o] Error 68 error:tool/build/lib/elfwriter.c:168: check failed: -1 != -1 errno= 2

make MODE= -j128 o//third_party/python/Lib/venv/scripts/posix/activate.csh.zip.o exited with 68: build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/venv/scripts/posix/activate.csh.zip.o third_party/python/Lib/venv/scripts/posix/activate.csh consumed 382µs wall time ballooned to 164kb in size needed 211us cpu (0% kernel) caused 18 page faults (100% memcpy) 3 context switches (33% consensual)

make: *** [build/rules.mk:77: o//third_party/python/Lib/venv/scripts/posix/activate.csh.zip.o] Error 68 error:tool/build/lib/elfwriter.c:168: check failed: -1 != -1 errno= 2

make MODE= -j128 o//third_party/python/Lib/venv/scripts/posix/activate.fish.zip.o exited with 68: build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/venv/scripts/posix/activate.fish.zip.o third_party/python/Lib/venv/scripts/posix/activate.fish consumed 331µs wall time ballooned to 164kb in size needed 196us cpu (0% kernel) caused 18 page faults (100% memcpy)

make: *** [build/rules.mk:77: o//third_party/python/Lib/venv/scripts/posix/activate.fish.zip.o] Error 68 527⏰ 479⏳ 276k 8iop build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/test/xmltestdata/.zip.o third_party/python/Lib/test/xmltestdata/ 619⏰ 540⏳ 488k 8iop build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/test/xmltestdata/simple.xml.zip.o third_party/python/Lib/test/xmltestdata/simple.xml 547⏰ 508⏳ 484k 8iop build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/test/xmltestdata/simple-ns.xml.zip.o third_party/python/Lib/test/xmltestdata/simple-ns.xml 662⏰ 624⏳ 484k 8iop build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/test/xmltestdata/expat224_utf8_bug.xml.zip.o third_party/python/Lib/test/xmltestdata/expat224_utf8_bug.xml 589⏰ 541⏳ 484k 8iop build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/test/xmltestdata/test.xml.out.zip.o third_party/python/Lib/test/xmltestdata/test.xml.out 639⏰ 591⏳ 484k 8iop build/bootstrap/zipobj.com -b0x400000 -P.python -C3 -o o//third_party/python/Lib/test/xmltestdata/test.xml.zip.o third_party/python/Lib/test/xmltestdata/test.xml 2,015,454⏰ 2,015,092⏳ 103,828k 12,528iop gcc -Og -c -o o//third_party/quickjs/quickjs.o third_party/quickjs/quickjs.c 2,536,675⏰ 2,536,403⏳ 253,512k 14,016iop g++ -o o//examples/nesemu1.o examples/nesemu1.cc 5,467,320⏰ 5,296,743⏳ 203,424k 20,040iop gcc -Og -O2 -c -o o//third_party/python/Objects/unicodeobject.o third_party/python/Objects/unicodeobject.c

[onefang]

'make -j128 -O' has no errors on todays git pull. B-)

Since this is officially closed, and it's working for me, I'll stop bothering this issue.