brew install not working

[bentxt]

Hi,

fresh clone in ~/local/cyclone on macos Catalina sw_vers

ProductName:    Mac OS X
ProductVersion: 10.15

Error: make: *** No rule to make targetlibs/cyclone/test.c', needed by cyclone'.

Here is the entire log:

brew install cyclone-scheme/cyclone/cyclone
Updating Homebrew...
==> Installing cyclone from cyclone-scheme/cyclone
==> Installing dependencies for cyclone-scheme/cyclone/cyclone: cyclone-bootstrap
==> Installing cyclone-scheme/cyclone/cyclone dependency: cyclone-bootstrap
==> Downloading https://github.com/justinethier/cyclone-bootstrap/archive/v0.11.5.tar.gz
Already downloaded: /Users/bkb/Library/Caches/Homebrew/downloads/4c11ef674416d5799a70f68c7e6cec378df6e17e9407986f92772c40484eedb8--cyclone-bootstrap-0.11.5.tar.gz
==> make
Last 15 lines from /Users/bkb/Library/Logs/Homebrew/cyclone-bootstrap/01.make:
2019-11-14 11:32:18 +0100

make

make: *** No rule to make target `libs/cyclone/test.c', needed by `cyclone'.  Stop.

[bentxt]

I think it has something to do with 'cyclone-bootstrap', because, there make test doesn't work

cd ~/local/cyclone/cyclone-bootstrap.git

make PREFIX=$HOME/local/cyclone/cyclone make PREFIX=$HOME/local/cyclone/cyclone install make PREFIX=$HOME/local/cyclone/cyclone test

Error:

ld: library not found for -lcyclone
clang: error: linker command failed with exit code 1 (use -v to see invocation)
/bin/sh: ./unit-tests: No such file or directory
make[1]: *** [unit-tests] Error 127
make: *** [test] Error 2

[justinethier]

Looks like there are two issues here. In the first case it looks like libs/cyclone/test.c is missing. Not sure what that is about. @adamfeuer - Have you noticed any issues with the brew install?

In the second case it looks like there is a path error, though the makefiles should have handled it properly. Can you verify the ~local/cyclone/cyclone directory exists?

[bentxt]

In the second case it looks like there is a path error, though the makefiles should have handled it properly. Can you verify the ~local/cyclone/cyclone directory exists?

yes, it looks the following:

tree -d /Users/bkb/local/cyclone/cyclone/
├── bin
├── include
│   └── cyclone
├── lib
└── share
    └── cyclone
        ├── cyclone
        ├── scheme
        │   └── cyclone
        └── srfi
            ├── list-queues
            └── sorting

12 directories

[justinethier]

Interesting, the above make commands work fine on Linux. There must be some difference in the environment when running them on MacOS.

[justinethier]

@bentxt Regarding the brew error there should be something similar to the following in the output (though you may need to run brew -v install cyclone-scheme/cyclone/cyclone):

Logs:
     /Users/runner/Library/Logs/Homebrew/cyclone-bootstrap/01.make
     /Users/runner/Library/Logs/Homebrew/cyclone-bootstrap/00.options.out
     /Users/runner/Library/Logs/Homebrew/cyclone-bootstrap/01.make.cc

Would it be possible for you to send those log files to me?

[adamfeuer]

@justinethier @bentxt Oops sorry folks, I was not paying attention. I'll look into this today.

[justinethier]

Cool, thanks @adamfeuer !

[adamfeuer]

@justinethier @bentxt I have repro'd the make test build failure. This is what I get:

/Applications/Xcode.app/Contents/Developer/usr/bin/make unit-tests
./cyclone unit-tests.scm && ./unit-tests
dyld: lazy symbol binding failed: Symbol not found: _mp_init
  Referenced from: /Users/adam/src/cyclone-bootstrap/./cyclone
  Expected in: flat namespace

dyld: Symbol not found: _mp_init
  Referenced from: /Users/adam/src/cyclone-bootstrap/./cyclone
  Expected in: flat namespace

/bin/sh: line 1: 36720 Abort trap: 6           ./cyclone unit-tests.scm
make[1]: *** [unit-tests] Error 134
make: *** [test] Error 2

Looking for the error now.

[justinethier]

@adamfeuer Thanks but it is concerning that is the third distinct error here :)

Do you get this error when doing make from a fresh copy of cyclone-bootstrap?

make: *** No rule to make target libs/cyclone/test.c', needed by cyclone'.

[adamfeuer]

@justinethier I think all the errors come from the same source. I got it to build using the brew version of libtommath instead of the version cyclone now includes.

The problem seems to be that cyclone is not building libs/cyclone/test.c because it can't link:

ld: warning: ignoring file /usr/local/lib/libcyclonebn.a, building for macOS-x86_64 but attempting to link with file built for unknown-unsupported file format ( 0x21 0x3C 0x61 0x72 0x63 0x68 0x3E 0x0A 0x2F 0x20 0x20 0x20 0x20 0x20 0x20 0x20 )

This is because it can't link against libcyclonebn.a – it's using the libtommath makefile which for some doesn't work on MacOS the way cyclone / cyclone-bootstap invokes it.

I think this changed in cyclone in commit https://github.com/justinethier/cyclone/commit/be8c534739d5574b4fbb0fe6c3f1284565b71937

Not sure what to do– the MacOS build was working because it was linking against a libtommath.a that was installed by the brew package manager. I'll have to look into how to build libtommath on MacOS if you want to use the version you're including with the cyclone source now.

Would you fill me in on the libcyclonebn change? Do you need to use this version of libtommath or can you use a version that's installed by a package manager? I would like the MacOS build to be as close to the Linux build as possible to keep maintenance low.

[justinethier]

@adamfeuer Interesting... the idea behind bundling libtommath with Cyclone is to (1) eliminate an external dependency as well as (2) avoid compilation issues with different versions of libtommath.

As a concept it seems like this should work fine. We build our own lcyclone library. But maybe there is a problem with the makefile(s) as provided by libtommath when compiling on macOS?

It is strange the compilation would fail where it does. At least on Linux all of our modules are compiled first before the files for libcyclonebn. So this should have happened before we encountered a problem later on with the math library:

cc  -O2 -fPIC -Wall -Iinclude   -c -o libs/cyclone/test.o libs/cyclone/test.c

[bentxt]

Would it be possible for you to send those log files to me?

cyclone-bootstrap.zip

[adamfeuer]

@justinethier I looked into it a little bit, while building libtommath on Linux is easy, building it MacOS is not very straightforward. I'll have to do some more research on this. In the meantime...

1. Maybe I could come up with a change to the Makefile that would do something OS dependent for MacOS? That would at least get us working again on MacOS. Would you be interested in an interim solution like this?
2. CI should have caught this... but I think we have no Mac CI set up. I'll also look into setting that up.

[justinethier]

@adamfeuer - Thank you for looking into this!

What kinds of problems are you running into with building libtommath on MacOS? If you go to https://github.com/justinethier/cyclone-bootstrap and scroll down a bit you will see CI jobs for both Travis and Github that are both successfully using the bootstrap makefile and building Cyclone (along with libtommath) on Mac OS.

For example: https://github.com/justinethier/cyclone-bootstrap/commit/f1a596bfd6f2a03b2a91256d0666bca0ea3367f7/checks?check_suite_id=315455564

There must be some kind of complication when using brew??

[justinethier]

@bentxt - Thank you for the logs, and thanks again for bringing this issue to our attention!

[adamfeuer]

@justinethier I don't remember exactly what the problem with building the way the CI server does it– but it's not the same as my Mac. I couldn't replicate the CI server's build environment, which is why I took the approach I did.

For libtommath– make; make install doesn't work because it doesn't detect the OS and use libtool – the docs point me to a PDF for install instructions, but then you can't build the docs on a Mac! So I need to install a latex toolchain on my VMWare linux box, but ran into disk space issues... anyway, I'll get tex installed, build the docs, and check out what the recommended way to build is.

As for the CI box, I don't know exactly what's going on there. Maybe the travis CI boxes are very out of date...? Or it could be some other difference. I'll also look into that, but it might take a few days.

[justinethier]

@adamfeuer Does make work? That will output libcyclonebn.a locally which is integrated into Cyclone's make install. No need to install libtommath separately anymore.

Anyway, sorry for the headache on all this :(. I appreciate your help on this and please take the time you need.

For what it's worth I could repro the issue using this Github action: https://github.com/justinethier/cyclone-bootstrap/blob/master/.github/workflows/test-brew.yml

The same CI environment will build Cyclone successfully using these steps: https://github.com/justinethier/cyclone-bootstrap/blob/master/.github/workflows/ci-macos.yml

[adamfeuer]

@justinethier make does not work on my box with the current version of cyclone or cyclone-bootrstrap. I see you just added an Xcode 11.2 Travis build. It runs there. And I have Xcode 11.2, the same as the travis build box. So it's a puzzle. Of course my Mac is a heavily customized and well-used developer workstation, and not a clean install.

I'll work on this some more over the next couple of days and report back.

[justinethier]

@bentxt

This should work for you now if you start from a git clone from cyclone-bootstrap and place everything in ~/local/cyclone/cyclone-bootstrap:

cd ~/local/cyclone/cyclone-bootstrap
make PREFIX=$HOME/local/cyclone/cyclone
make PREFIX=$HOME/local/cyclone/cyclone install
make PREFIX=$HOME/local/cyclone/cyclone test

Brew still needs to be fixed...

[justinethier]

This is working better now, though there may still be path issues, so we are not done yet. You would want to use the following commands:

brew update; brew tap cyclone-scheme/cyclone ; brew install cyclone-scheme/cyclone/cyclone-bootstrap

[justinethier]

@bentxt - Apologies for the delay on this issue. Everything should be working now using brew install cyclone-scheme/cyclone/cyclone-bootstrap. When you have a moment can you please confirm?

[bentxt]

I can confirm that it works now. Great :-)

[justinethier]

@bentxt - Excellent, I will go ahead and close this ticket. Thank you for reporting this issue and helping us track it down!