error building chezscheme openbsd 6.0

[rashadkm]

following this commit https://github.com/cisco/ChezScheme/commit/f81024e4979da28502ae634181b379083e419911. I got boot files for openbsd 'a64ob' from a linux.

I copied boot/a6ob directory to openbsd (6.0)

and run ./configure gmake

Build fails at this step with below error:

gmake all
echo '(reset-handler abort)'\
             '(base-exception-handler (lambda (c) (fresh-line) (display-condition c) (newline) (reset)))'\
             '(keyboard-interrupt-handler (lambda () (display "interrupted---aborting\n") (reset)))'\
             '(optimize-level 3)'\
             '(debug-level 0)'\
             '(generate-inspector-information #f)'\
             '(subset-mode (quote system))'\
             '(compile-file "cmacros.ss" "cmacros.so")'\
             | ../bin/a6ob/scheme -q
sizeof(time_t) * 8 [8] != time_t_bits [32]
gmake[4]: *** [Mf-base:312: cmacros.so] Error 1
gmake[3]: *** [Mf-base:146: allx] Error 2
gmake[2]: *** [Mf-base:157: bootstrap] Error 2
gmake[1]: *** [Makefile:21: build] Error 2
gmake: *** [Makefile:19: build] Error 2

build logs:

1. https://gist.github.com/rashadkm/123b40d014f5ca536f5350157d9f3a5b I replaced make -> gmake in c/Mf-a6ob, run gmake again and got below error
2. https://gist.github.com/rashadkm/c8cd26f8c0bbe2a1e18ccce1d31a0d5d

uname -a OpenBSD ab.my.domain 6.0 GENERIC.MP#2319 amd64

[rashadkm]

changing -#define time_t_bits 0x20 +#define time_t_bits 0x40 in boot/a6ob/equates.h

gives segfault

[akeep]

Hi @rashadkm,

Thanks for reporting this. I've not had a chance to commit a fix, but the file that needs to be changed is the a6ob.def in the s directory of the Chez Scheme repository.

In that file, you'll find a define-constant line that looks like the following:

(define-constant time-t-bits 32)

This needs to be changed to:

(define-constant time-t-bits 64)

Once you've made that change, you'll need to rebuild the boot file from a linux machine (which will also rebuild the equates.h and scheme.h files), and move this back onto your OpenBSD build machine.

Unfortunately just changing the equates.h file doesn't help, since the size of the time_t_bits has been compiled into the petite.boot and scheme.boot files to be 32-bits.

I'll try to get together a fix this evening, but hopefully this will get you up and running.

Thanks for the bug report! -andy:)

[gwatt]

It looks like the netbsd .def also needs this change.

[rashadkm]

thanks @akeep and @gwatt for your replies.

I tried your change but no luck. If I change boot/a6ob/equates.h -#define time_t_bits 0x20 +#define time_t_bits 0x40

This changes error in compilation. now it is segfault. see below gist. https://gist.github.com/rashadkm/380c2ee6bd0fa452c5b5b44574cc4d40

Any idea on how to debug?

btw, I also change ChezScheme/s/a6ob.def I copied boot/a6ob from a linux x86_64 machine using 'make bootfiles'. my target openbsd machine is OpenBSD 6.0 amd64.

[rashadkm]

here is the output from gdb

https://gist.github.com/rashadkm/a80f03f095c88d01dda0b5f89397012a you can see scheme was run with no arguments when it crashes.

when I pass --version or --help it has below output https://gist.github.com/rashadkm/3c7a2fc298fc975f776a60b92710630e

passing --script or during make it fails. I would appreciate if someone could provide any hints?

[rkanavath]

the call is failing in mmap (see gdb output) segment.c: 151 if ((addr = mmap(NULL, p_bytes, PROT_EXEC|PROT_WRITE|PROT_READ, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)) == (void *)-1) {

openbsd 6.0 has wx enabled. so PROT_EXEC|PROT_WRITE|PROT_READ might be a start when debugging this issue. I will try to find more information later. But I need someone from chezscheme team to fix this issue.

[rkanavath]

.... and I have this on dmesg

scheme(36686): mmap W^X violation scheme(80674): mmap W^X violation scheme(64380): mmap W^X violation scheme(46225): mmap W^X violation scheme(57800): mmap W^X violation scheme(45901): mmap W^X violation

adding -Wl,-z,wxneeded to LDFLAGS fixes the segfault issue. but still I cannot run this.

$ ./a6ob/bin/a6ob/scheme -b ./a6ob/boot/a6ob/petite.boot  
./a6ob/bin/a6ob/scheme[1]: ELF: not found

[gwatt]

Weird, I was able to run scheme in an OpenBSD 6 VM.

[rkanavath]

how!. I tried to latest git master branch and last release archive.

can you tell how you generated bootfiles on what machine and steps etc.. ?

[gwatt]

I built the bootfiles on a linux amd64 machine and uploaded them: https://github.com/gwatt/ChezScheme/tree/new-bootfiles Then I cloned the repository to my VM. To actually build chez I had to delete some of the locale stuff in c/expeditor.c. After that, it worked fine. I should probably try on bare metal and not a vm, but I don't expect that to matter.

[rkanavath]

the issue has to be with bootfiles I guess. I am still not able to make things work on my openbsd. can you give output of ldd ./a6ob/bin/a6ob/scheme

uname -a (openbsd) OpenBSD my.domain 6.0 GENERIC.MP#2319 amd64

uname -a (linux) Linux 3.13.0-108-generic #155-Ubuntu SMP Wed Jan 11 16:58:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

[gwatt]

OpenBSD openbsd 6.0 GENERIC#2148 amd64

[rkanavath]

hmm. that is same..

error keep changing. see here:

gmake all echo '(reset-handler abort)'\ '(base-exception-handler (lambda (c) (fresh-line) (display-condition c) (newline) (reset)))'\ '(keyboard-interrupt-handler (lambda () (display "interrupted---aborting\n") (reset)))'\ '(optimize-level 3)'\ '(debug-level 0)'\ '(generate-inspector-information #f)'\ '(subset-mode (quote system))'\ '(compile-file "cmacros.ss" "cmacros.so")'\ | ../bin/a6ob/scheme -q ../bin/a6ob/scheme[2]: syntax error: `(' unexpected

[gwatt]

I don't have an all target for gmake at the root directory. Where are you running gmake?

[rkanavath]

i am running from root directory of repo. where should I run gmake ?

I clone your repository, checkout out new-bootfiles branch. and ./configure gmake (has to change c/expeditor.c for locale changes as you said)

gmake (cd c ; gmake) gmake[1]: Nothing to be done for 'doit'. (cd s ; gmake bootstrap) gmake allx rm -f .a6ob xpatch patch .patch .so .asm script.all header.tmp *.html rm -rf nanopass cp -p -f ../boot/a6ob/petite.boot ../boot/a6ob/sbb cp -p -f ../boot/a6ob/scheme.boot ../boot/a6ob/scb gmake all echo '(reset-handler abort)'\ '(base-exception-handler (lambda (c) (fresh-line) (display-condition c) (newline) (reset)))'\ '(keyboard-interrupt-handler (lambda () (display "interrupted---aborting\n") (reset)))'\ '(optimize-level 3)'\ '(debug-level 0)'\ '(generate-inspector-information #f)'\ '(subset-mode (quote system))'\ '(compile-file "cmacros.ss" "cmacros.so")'\ | ../bin/a6ob/scheme -q ../bin/a6ob/scheme[1]: syntax error: `(' unexpected gmake[3]: [Mf-base:321: cmacros.so] Error 1 gmake[2]: [Mf-base:153: allx] Error 2 gmake[1]: [Mf-base:166: bootstrap] Error 2 gmake: [Makefile:21: build] Error 2

[gwatt]

That's where I'm running it, but if I try gmake all it tells me

gmake: *** No rule to make target 'all'. Stop.

Edit: After a second look, I think that's a recursive invocation of gmake in some subdirectory so my line of pursuit here is not useful.

[rkanavath]

okay. "syntax error: `(' unexpected" could be signalling a difference target vs host arch ?

[gwatt]

That looks like a shell error to me. Edit:

572 $ ls ()
-ksh: syntax error: `(' unexpected

[rkanavath]

No. /a6ob/bin/a6ob/scheme
./a6ob/bin/a6ob/scheme[1]: syntax error: `(' unexpected $ file ./a6ob/bin/a6ob/scheme ./a6ob/bin/a6ob/scheme: ELF 64-bit LSB shared object, x86-64, version 1

Edit: running from bash gives this error /a6ob/bin/a6ob/scheme bash: ./a6ob/bin/a6ob/scheme: cannot execute binary file: Exec format error

$ file ./a6ob/bin/a6ob/scheme ./a6ob/bin/a6ob/scheme: ELF 64-bit LSB shared object, x86-64, version 1

[rkanavath]

can you push your changes to expeditor.c in case changes weren't same on my side. just to be sure.

[gwatt]

Done.

[rkanavath]

thanks. I checkout out the code and still not working. I still believe there is something wroing with generated bootfiles and later using it to creating scheme binaries.

some more questions: is ./a6ob/bin/a6ob/scheme are supposed to scripts or executable files ? because you see the output of file command on my system.

and if that is a shell script then ldd gives some information depending on if there is any shared libraries. ? ldd ./a6ob/bin/a6ob/scheme

does nm ./a6ob/bin/a6ob/scheme gives you any undefined symbols? thanks again for your patience and good effort in helping

[gwatt]

ldd on ta6ob/bin/scheme yields fairly normal output: libiconv, libm, libncurses, libpthread, libc nm also looks normal. There are undefined symbols, but they're all provided by the shared libraries.

[rkanavath]

ah. in my case ldd gives error $ ldd ./a6ob/bin/a6ob/scheme
./a6ob/bin/a6ob/scheme: ./a6ob/bin/a6ob/scheme: Exec format error ./a6ob/bin/a6ob/scheme: exit status 1

[rkanavath]

finally, it is working. the fix was silly..

clone and build your repo on /usr/local. why? only this parttion is mounted with wxneeded and adding LDFLAGS won't help un running scheme binaries.. ufff

[gwatt]

Ha! That explains why I didn't have problems. My vm is small, and I just let the installer auto-partition the drive. /home was too small for the chez repo, so I moved it into /usr before building.

[rkanavath]

./ta6ob/bin/scheme -b ./boot/ta6ob/petite.boot
Petite Chez Scheme Version 9.4.1 Copyright 1984-2016 Cisco Systems, Inc.

(printf "hello world!") hello world! (exit)

Thanks a lot for your immense patience and support in helping me. You might merge changes to upstream. maybe add a ifdef openbsd for locale stuff?

[gwatt]

I could. I'd want to know upstream's interest in having and maintaining OpenBSD as a first class citizen before I do that.

[rkanavath]

Sure. you have my +1. and hope they include in upstream.

[vaspaese]

Would be great to have *BSD support in Chez Scheme. +1.

[akeep]

Hey All,

Just a quick follow-up on this. I'd love to get better support for the BSDs in as well. Currently, the boot files are one of the limiting factors here. Unfortunately, they are binary and already compressed, and each platform adds about 2 MB for each revision that requires a new boot file be built (which is essentially every time we make source code changes to the Scheme compiler or libraries).

So, we need to find a way to pair down our existing boot files, potentially going back to just having a single machine-type checked in, and then providing the other binaries for download after each check-in. I don't have a solution for this yet, but it is something we're thinking about so that we can provide a larger set of boot files, including boot files for the BSDs.

I do have a question for those of you using BSDs though, are you aware of any continuous integration resources for building on BSDs as commits come in? I've been considering automating the Linux and macOS builds, and I'd really like to get all supported platforms on this, but I'm not sure where to look for BSD automation resources.

[gwatt]

@akeep Is there a way to provide a single boot file per architecture, regardless of operating system? If that's not possible, what about smaller OS dependent boot files and a large boot file containing portable code?

Another issue is building the C parts on the BSDs. Obviously OpenBSD requires source changes. I have built Chez on FreeBSD, but don't have that system installed anymore. I remember there were some issues with iconv, but cannot remember specifics. I also don't know if this was before https://github.com/cisco/ChezScheme/pull/83, which causes build failures on OpenBSD. I have never tried building Chez on NetBSD.

[akeep]

Hey @gwatt

No, it is not possible to have one boot file per architecture. The boot file has compiled in calls to the C runtime routines, so it needs to have the calling conventions for the OS for that machine type. Even the threaded vs. non-threaded versions need to be independent because the handling of Scheme's context is different between the two and produces different boot files.

The C parts already make heavy use of #ifdefs to control how code is built on different platforms, and I think this should be perfectly sufficient for keeping the BSD code compiling (especially since we also build for Windows from this source tree).

Unfortunately, not having regular testing on the BSDs has lead to some code being added that has never been tested on a BSD, like the newlocale code for allowing the expression editor to accept unicode input. The system.h file contains the selectors for features that are only available on some platforms.

I've built Chez since the open source release on FreeBSD, OpenBSD, NetBSD, and DragonFlyBSD, but it is has probably been 6 months at least, and I've not gone through all the testing on these platforms. This is part of the reason I'd like to get all of this stuff on regular continuous integration. It is easy for me to drop the ball on this when I get busy with my day job, when the continuous integration isn't keeping my feet to the fire :).

[gwatt]

@akeep I feared that was the case for the bootfiles. Oh well.

[rkanavath]

@akeep , If you can have OpenBSD VM (always last stable release), is that enough to setup a daily CI build ? I have used ctest for continuous integration under obsd. Let me know what is required and I can try to setup this on my part

[akeep]

@rkanavath, part of what I'm looking for is a service like Travis-CI that can be connected with GitHub to initiate these builds in an autonomous fashion. Short of that, I can probably try to get a machine setup to do these builds, but I suspect a service will be a bit more reliable.

[rkanavath]

@akeep, I am not aware of any service like travis-CI that support BSD systems. other than a VM to do continuous build, i don't see any other option.

about openbsd boot files, is it possible to include all boot files in git repo and using git submodule?

[rkanavath]

@akeep, I am not aware of any service like travis-CI that support BSD systems. other than a VM to do continuous build, i don't see any other option.

about openbsd boot files, is it possible to include all boot files in git repo and using git submodule?

[mflatt]

Now that the build is based on platform-independent pb files, show we close this?

OpenBSD is still not part if CI, but it's supported. (I happen to have built on OpenBSD 7.x a few days ago.)