search

dom96 / choosenim

Tool for easily installing and managing multiple versions of the Nim programming language.
BSD 3-Clause "New" or "Revised" License
682 stars 65 forks source link

SIGBUS: Illegal storage access. (Attempt to read from nil?) on macOS #122

Closed metasyn closed 4 years ago

metasyn commented 5 years ago 
❯ curl https://nim-lang.org/choosenim/init.sh -sSf | sh
choosenim-init: Downloading choosenim-0.4.0_macosx_amd64
SIGBUS: Illegal storage access. (Attempt to read from nil?)

on zsh/bash on macOS 10.14.2

dom96 commented 5 years ago

That's a shame. @genotrance any chance you could take a look?

metasyn commented 5 years ago

Happy to help debug if there is anything I can do

metasyn commented 5 years ago

Using Nim:

❯ nim --version
Nim Compiler Version 0.20.99 [MacOSX: amd64]
Compiled at 2019-06-11
Copyright (c) 2006-2019 by Andreas Rumpf

git hash: ced0527ae334439a10e1719d1eccb727c19dc781
active boot switches: -d:release

and Nimble:

❯ nimble --version
nimble v0.10.2 compiled at 2019-06-11 00:09:59
git hash: d15c8530cb7480ce39ffa85a2dd9819d2d4fc645

If I run tests:

~/code/choosenim master
❯ nimble test
  Executing task test in /Users/aljohnson/code/choosenim/choosenim.nimble
Hint: used config file '/Users/aljohnson/code/Nim/config/nim.cfg' [Conf]
Hint: used config file '/Users/aljohnson/code/Nim/config/config.nims' [Conf]
Hint: system [Processing]
Hint: widestrs [Processing]
Hint: io [Processing]
Hint: tester [Processing]
Hint: osproc [Processing]
Hint: strutils [Processing]
Hint: parseutils [Processing]
Hint: math [Processing]
Hint: bitops [Processing]
Hint: macros [Processing]
Hint: algorithm [Processing]
Hint: unicode [Processing]
Hint: os [Processing]
Hint: pathnorm [Processing]
Hint: osseps [Processing]
Hint: posix [Processing]
Hint: times [Processing]
Hint: options [Processing]
Hint: typetraits [Processing]
Hint: strtabs [Processing]
Hint: hashes [Processing]
Hint: streams [Processing]
Hint: cpuinfo [Processing]
Hint: kqueue [Processing]
Hint: unittest [Processing]
Hint: sets [Processing]
Hint: terminal [Processing]
Hint: strformat [Processing]
Hint: colors [Processing]
Hint: tables [Processing]
Hint: termios [Processing]
Hint: sequtils [Processing]
Hint: future [Processing]
/Users/aljohnson/code/choosenim/tests/tester.nim(3, 59) Warning: Use the new 'sugar' module instead; future is deprecated [Deprecated]
CC: stdlib_assertions.nim
CC: stdlib_io.nim
CC: stdlib_system.nim
CC: stdlib_parseutils.nim
CC: stdlib_math.nim
CC: stdlib_unicode.nim
CC: stdlib_strutils.nim
CC: stdlib_pathnorm.nim
CC: stdlib_posix.nim
CC: stdlib_times.nim
CC: stdlib_os.nim
CC: stdlib_hashes.nim
CC: stdlib_strtabs.nim
CC: stdlib_streams.nim
CC: stdlib_cpuinfo.nim
CC: stdlib_osproc.nim
CC: stdlib_sets.nim
CC: stdlib_strformat.nim
CC: stdlib_terminal.nim
CC: stdlib_unittest.nim
CC: stdlib_future.nim
CC: tester.nim
Hint:  [Link]
Hint: operation successful (68147 lines compiled; 2.008 sec total; 72.781MiB peakmem; Debug Build) [SuccessX]
Hint: /Users/aljohnson/code/choosenim/tests/tester  [Exec]
  Verifying dependencies for choosenim@0.4.0
      Info: Dependency on nimble@>= 0.8.5 already satisfied
  Verifying dependencies for nimble@0.10.2
      Info: Dependency on untar@>= 0.1.0 already satisfied
  Verifying dependencies for untar@0.1.0
      Info: Dependency on libcurl@>= 1.0.0 already satisfied
  Verifying dependencies for libcurl@1.0.0
      Info: Dependency on analytics@>= 0.2.0 already satisfied
  Verifying dependencies for analytics@0.2.0
      Info: Dependency on uuids@>= 0.1.7 already satisfied
  Verifying dependencies for uuids@0.1.10
      Info: Dependency on isaac@>= 0.1.3 already satisfied
  Verifying dependencies for isaac@0.1.3
      Info: Dependency on osinfo@>= 0.3.0 already satisfied
  Verifying dependencies for osinfo@0.3.0
      Info: Dependency on osinfo@>= 0.3.0 already satisfied
  Verifying dependencies for osinfo@0.3.0
   Building choosenim/choosenim using c backend
[OK] can compile choosenim
/Users/aljohnson/code/choosenim/tests/tester.nim(117, 17): Check failed: inLines(output.processOutput, "invalid")
/Users/aljohnson/code/choosenim/tests/tester.nim(118, 17): Check failed: inLines(output.processOutput, "version")
/Users/aljohnson/code/choosenim/tests/tester.nim(119, 17): Check failed: inLines(output.processOutput, "path")
/Users/aljohnson/code/choosenim/tests/tester.nim(124, 17): Check failed: inLines(output.processOutput, "no")
/Users/aljohnson/code/choosenim/tests/tester.nim(125, 17): Check failed: inLines(output.processOutput, "binary")
/Users/aljohnson/code/choosenim/tests/tester.nim(126, 17): Check failed: inLines(output.processOutput, "found")
[FAILED] refuses invalid path
/Users/aljohnson/code/choosenim/tests/tester.nim(132, 15): Check failed: inLines(output.processOutput, "unknown")
/Users/aljohnson/code/choosenim/tests/tester.nim(133, 15): Check failed: inLines(output.processOutput, "flag")
[FAILED] fails on bad flag
    Prompt: Can choosenim record and send anonymised telemetry data? [y/n]
        ... Anonymous aggregate user analytics allow us to prioritise
        ... fixes and features based on how, where and when people use Nim.
        ... For more details see: https://goo.gl/NzUEPf.
    Answer: Forced Yes
Traceback (most recent call last)
/Users/aljohnson/code/choosenim/src/choosenim.nim(268) choosenim
/Users/aljohnson/code/choosenim/src/choosenim/telemetry.nim(120) loadAnalytics
/Users/aljohnson/.nimble/pkgs/analytics-0.2.0/analytics.nim(63) newAsyncAnalytics
/Users/aljohnson/code/Nim/lib/pure/httpclient.nim(658) newAnalyticsRef
/Users/aljohnson/code/Nim/lib/pure/httpclient.nim(412) getDefaultSSL
/Users/aljohnson/code/Nim/lib/pure/net.nim(532) newContext
SIGBUS: Illegal storage access. (Attempt to read from nil?)
/Users/aljohnson/code/choosenim/tests/tester.nim(139, 19): Check failed: exitCode == QuitSuccess
exitCode was 1
QuitSuccess was 0
/Users/aljohnson/code/choosenim/tests/tester.nim(141, 17): Check failed: inLines(output.processOutput, "building")
/Users/aljohnson/code/choosenim/tests/tester.nim(142, 17): Check failed: inLines(output.processOutput, "downloading")
/Users/aljohnson/code/choosenim/tests/tester.nim(146, 19): Check failed: inLines(output.processOutput, "building tools")
/Users/aljohnson/code/choosenim/tests/tester.nim(147, 17): Check failed: hasLine(output.processOutput, "switched to nim 0.16.0")
/Users/aljohnson/code/choosenim/tests/tester.nim(151, 19): Check failed: exitCode == QuitSuccess
exitCode was 1
QuitSuccess was 0
/Users/aljohnson/code/choosenim/tests/tester.nim(153, 17): Check failed: hasLine(output.processOutput, "info: version 0.16.0 already selected")
/Users/aljohnson/code/choosenim/tests/tester.nim(157, 19): Check failed: exitCode == QuitSuccess
exitCode was 127
QuitSuccess was 0
/Users/aljohnson/code/choosenim/tests/tester.nim(158, 17): Check failed: inLines(output.processOutput, "v0.8.2")
[FAILED] can choose v0.16.0
Error: execution of an external program failed: '/Users/aljohnson/code/choosenim/tests/tester '
stack trace: (most recent call last)
/private/var/folders/cy/fp9dph0x14x9s7fszwjjjh4c0000gn/T/nimblecache/nimscriptapi.nim(164, 16)
/Users/aljohnson/code/choosenim/choosenim_8672.nims(28, 10) testTask
/Users/aljohnson/code/Nim/lib/system/nimscript.nim(242, 7) exec
/Users/aljohnson/code/Nim/lib/system/nimscript.nim(242, 7) Error: unhandled exception: FAILED: nim c -r tester
     Error: Exception raised during nimble script execution

Not positive if this is helpful

dom96 commented 5 years ago

Seems like this is where the crash is coming from:

Traceback (most recent call last)
/Users/aljohnson/code/choosenim/src/choosenim.nim(268) choosenim
/Users/aljohnson/code/choosenim/src/choosenim/telemetry.nim(120) loadAnalytics
/Users/aljohnson/.nimble/pkgs/analytics-0.2.0/analytics.nim(63) newAsyncAnalytics
/Users/aljohnson/code/Nim/lib/pure/httpclient.nim(658) newAnalyticsRef
/Users/aljohnson/code/Nim/lib/pure/httpclient.nim(412) getDefaultSSL
/Users/aljohnson/code/Nim/lib/pure/net.nim(532) newContext
SIGBUS: Illegal storage access. (Attempt to read from nil?)

Can you look into that code and debug why that's nil?

genotrance commented 5 years ago

Sorry I've not had OSX access for some time, but feels like an SSL setup issue. Might be worth checking which version of openssl is present, updating with brew, and checking the order of versions in the Nim openssl wrapper.

metasyn commented 5 years ago

Hmm - it could be - here is my version:

❯ openssl version
OpenSSL 1.1.0j  20 Nov 2018

At the end of httpclient.nim, I added:

when isMainModule:
  let j = getDefaultSSL()

with no error when I run it.

When I check the the wrapper (lib/wrappers/openssl.nim):

  when defined(osx):
    const versions = "(.1.1|.38|.39|.41|.43|.44|.45|.46|.10|.1.0.2|.1.0.1|.1.0.0|.0.9.9|.0.9.8|)"
  else:
    const versions = "(.1.1|.1.0.2|.1.0.1|.1.0.0|.0.9.9|.0.9.8|.46|.45|.44|.43|.41|.39|.38|.10|)"

When I build and test the wrapper:

❯ ./bin/nim c -d:ssl -p:. -r tests/untestable/tssl.nim
Hint: used config file '/Users/aljohnson/code/Nim/config/nim.cfg' [Conf]
Hint: used config file '/Users/aljohnson/code/Nim/config/config.nims' [Conf]
Hint: system [Processing]
Hint: widestrs [Processing]
Hint: io [Processing]
Hint: tssl [Processing]
Hint: httpclient [Processing]
Hint: net [Processing]
Hint: nativesockets [Processing]
Hint: os [Processing]
Hint: strutils [Processing]
Hint: parseutils [Processing]
Hint: math [Processing]
Hint: bitops [Processing]
Hint: macros [Processing]
Hint: algorithm [Processing]
Hint: unicode [Processing]
Hint: pathnorm [Processing]
Hint: osseps [Processing]
Hint: posix [Processing]
Hint: times [Processing]
Hint: options [Processing]
Hint: typetraits [Processing]
Hint: sets [Processing]
Hint: hashes [Processing]
Hint: openssl [Processing]
Hint: dynlib [Processing]
Hint: uri [Processing]
Hint: strtabs [Processing]
Hint: base64 [Processing]
Hint: mimetypes [Processing]
Hint: random [Processing]
Hint: httpcore [Processing]
Hint: tables [Processing]
Hint: streams [Processing]
Hint: asyncnet [Processing]
Hint: asyncdispatch [Processing]
Hint: heapqueue [Processing]
Hint: lists [Processing]
Hint: asyncstreams [Processing]
Hint: asyncfutures [Processing]
Hint: deques [Processing]
Hint: cstrutils [Processing]
Hint: selectors [Processing]
Hint: kqueue [Processing]
Hint: asyncfile [Processing]
CC: stdlib_assertions.nim
CC: stdlib_io.nim
CC: stdlib_system.nim
CC: stdlib_parseutils.nim
CC: stdlib_math.nim
CC: stdlib_strutils.nim
CC: stdlib_pathnorm.nim
CC: stdlib_posix.nim
CC: stdlib_options.nim
CC: stdlib_times.nim
CC: stdlib_os.nim
CC: stdlib_nativesockets.nim
CC: stdlib_hashes.nim
CC: stdlib_sets.nim
CC: stdlib_dynlib.nim
CC: stdlib_openssl.nim
CC: stdlib_net.nim
CC: stdlib_uri.nim
CC: stdlib_base64.nim
CC: stdlib_random.nim
CC: stdlib_tables.nim
CC: stdlib_httpcore.nim
CC: stdlib_streams.nim
CC: stdlib_heapqueue.nim
CC: stdlib_deques.nim
CC: stdlib_asyncfutures.nim
CC: stdlib_kqueue.nim
CC: stdlib_selectors.nim
CC: stdlib_asyncdispatch.nim
CC: stdlib_httpclient.nim
CC: tssl.nim
Hint:  [Link]
Hint: operation successful (77530 lines compiled; 2.348 sec total; 90.57MiB peakmem; Debug Build) [SuccessX]
Hint: /Users/aljohnson/code/Nim/tests/untestable/tssl  [Exec]
version: 0x00000000101000AF
done

Test:

❯ ./bin/nim c -d:ssl -p:. --dynlibOverride:ssl --passL:-lcrypto --passL:-lssl -r tests/untestable/tssl.nim
Hint: used config file '/Users/aljohnson/code/Nim/config/nim.cfg' [Conf]
Hint: used config file '/Users/aljohnson/code/Nim/config/config.nims' [Conf]
Hint: system [Processing]
Hint: widestrs [Processing]
Hint: io [Processing]
Hint: tssl [Processing]
Hint: httpclient [Processing]
Hint: net [Processing]
Hint: nativesockets [Processing]
Hint: os [Processing]
Hint: strutils [Processing]
Hint: parseutils [Processing]
Hint: math [Processing]
Hint: bitops [Processing]
Hint: macros [Processing]
Hint: algorithm [Processing]
Hint: unicode [Processing]
Hint: pathnorm [Processing]
Hint: osseps [Processing]
Hint: posix [Processing]
Hint: times [Processing]
Hint: options [Processing]
Hint: typetraits [Processing]
Hint: sets [Processing]
Hint: hashes [Processing]
Hint: openssl [Processing]
Hint: dynlib [Processing]
Hint: uri [Processing]
Hint: strtabs [Processing]
Hint: base64 [Processing]
Hint: mimetypes [Processing]
Hint: random [Processing]
Hint: httpcore [Processing]
Hint: tables [Processing]
Hint: streams [Processing]
Hint: asyncnet [Processing]
Hint: asyncdispatch [Processing]
Hint: heapqueue [Processing]
Hint: lists [Processing]
Hint: asyncstreams [Processing]
Hint: asyncfutures [Processing]
Hint: deques [Processing]
Hint: cstrutils [Processing]
Hint: selectors [Processing]
Hint: kqueue [Processing]
Hint: asyncfile [Processing]
CC: stdlib_system.nim
CC: stdlib_strutils.nim
CC: stdlib_openssl.nim
CC: stdlib_net.nim
Hint:  [Link]
Undefined symbols for architecture x86_64:
  "_OPENSSL_init_ssl", referenced from:
      _SSL_library_init_6SfBfS64wp6JZihQGGrnKg in stdlib_openssl.nim.c.o
  "_OpenSSL_version_num", referenced from:
      _getOpenSSLVersion_dJsO04BYfgh1GDCx4NNEgA in stdlib_openssl.nim.c.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: execution of an external program failed: 'clang   -o /Users/aljohnson/code/Nim/tests/untestable/tssl  /Users/aljohnson/.cache/nim/tssl_d/stdlib_assertions.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_io.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_system.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_parseutils.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_math.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_strutils.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_pathnorm.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_posix.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_options.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_times.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_os.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_nativesockets.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_hashes.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_sets.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_openssl.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_net.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_uri.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_base64.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_random.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_tables.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_httpcore.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_streams.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_heapqueue.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_deques.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_asyncfutures.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_kqueue.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_selectors.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_asyncdispatch.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/stdlib_httpclient.nim.c.o /Users/aljohnson/.cache/nim/tssl_d/tssl.nim.c.o  -lm  -lcrypto -lssl  -ldl'

which looks like it could be part of the issue. I'll try changing some openssl stuff and see whats going on...

dom96 commented 5 years ago

I don't think that's an issue, don't mess with --dynlibOverride. Do try using httpclient and see if you can reproduce this.

metasyn commented 5 years ago

I've since moved to using a different computer (also macOS) which does not have this problem. I'm going to chock it up to my weird setup on the old machine and close; someone else can re-open if this becomes an issue or something :)

Thanks!

Milerius commented 4 years ago

@dom96

can we reopen:

➜ bin curl https://nim-lang.org/choosenim/init.sh -sSf | sh choosenim-init: Downloading choosenim-0.4.0_macosx_amd64 SIGBUS: Illegal storage access. (Attempt to read from nil?)

OSX Catalina, AppleClang 11.

dom96 commented 4 years ago

@Milerius can you have a look at the above posts by me and the original author and see if you can reproduce by using httpclient? (you can install Nim some other way to try compiling a program which uses httpclient).

genotrance commented 4 years ago

I just tried on Mojave with openssl 1.1.1d and it all works fine. This is with Nim 1.0.4 and choosenim 0.5.0+.

dom96 commented 4 years ago

We need a debug build of choosenim so that we can ask users who receive such crashes to try it and give us a stack trace.

genotrance commented 4 years ago

--debuginfo --debugger:native or just no -d:release / -d:danger?

dom96 commented 4 years ago

just no -d:release / -d:danger

Just this.

genotrance commented 4 years ago

Okay, I'll add it to the CI and debug builds will get posted for every release.

genotrance commented 4 years ago

PR #160 now posts debug builds. Next minor release will have debug builds which can be used for such issues going forward.

timotheecour commented 4 years ago

dyld: Symbol not found: _futimens Referenced from: /var/folders/bb/0tr5gkzx6s9dvb2t67_pfpj80000gn/T//choosenim-0.5.1_macosx_amd64 (which was built for Mac OS X 10.13) Expected in: /usr/lib/libSystem.B.dylib

SIGABRT: Abnormal termination.


* here's more debugging information:
curl https://nim-lang.org/choosenim/init.sh -sSf > /tmp/z01.sh
instrument `/tmp/z01.sh` to see where it fails
=> it fails here: `"$temp_prefix/$filename" $CHOOSE_VERSION --firstInstall ${debug} < /dev/tty`

lldb -- /var/folders/bb/0tr5gkzx6s9dvb2t67_pfpj80000gn/T//choosenim-0.5.1_macosx_amd64 stable --firstInstall (lldb) target create "/var/folders/bb/0tr5gkzx6s9dvb2t67_pfpj80000gn/T//choosenim-0.5.1_macosx_amd64" Current executable set to '/var/folders/bb/0tr5gkzx6s9dvb2t67_pfpj80000gn/T//choosenim-0.5.1_macosx_amd64' (x86_64). (lldb) settings set -- target.run-args "stable" "--firstInstall" (lldb) br s -r raiseEIO Breakpoint 1: where = choosenim-0.5.1_macosx_amd64`raiseEIO__ZYk14k3sVNZUIjJjtqzFZQ, address = 0x0000000100000970 (lldb) r Process 34986 launched: '/var/folders/bb/0tr5gkzx6s9dvb2t67_pfpj80000gn/T/choosenim-0.5.1_macosx_amd64' (x86_64) Process 34986 stopped


EDIT: this is probably a red herring, caused by https://github.com/nim-lang/Nim/issues/9634

## EDIT
after instrumenting the sh file to instead use debug build choosenim-0.5.1_macosx_amd64_debug I get:

Traceback (most recent call last) /Users/travis/build/dom96/choosenim/src/choosenim.nim(267) choosenim /Users/travis/build/dom96/choosenim/src/choosenim.nim(248) performAction /Users/travis/build/dom96/choosenim/src/choosenim.nim(66) choose /Users/travis/build/dom96/choosenim/src/choosenim.nim(53) chooseVersion /Users/travis/build/dom96/choosenim/src/choosenim.nim(19) installVersion /Users/travis/build/dom96/choosenim/src/choosenimpkg/utils.nim(47) extract /Users/travis/.nimble/pkgs/nimarchive-0.3.5/nimarchive.nim(84) extract SIGABRT: Abnormal termination.


which means error isn't caught here, which points to your code @genotrance :)
```nim
proc extract*(path: string, extractDir: string) =
  display("Extracting", path.extractFilename(), priority = HighPriority)

  try:
    nimarchive.extract(path, extractDir)
  except Exception as exc:
    raise newException(ChooseNimError, "Unable to extract. Error was '$1'." %
                       exc.msg)
nixfreak commented 4 years ago

./choosenim-0.5.1_macosx_amd64_debug show [10:35:17] Traceback (most recent call last) /Users/travis/build/dom96/choosenim/src/choosenim.nim(266) choosenim /Users/travis/build/dom96/choosenim/src/choosenimpkg/telemetry.nim(120) loadAnalytics /Users/travis/.nimble/pkgs/analytics-0.2.0/analytics.nim(63) newAsyncAnalytics /Users/travis/.choosenim/toolchains/nim-1.0.4/lib/pure/httpclient.nim(534) newAnalyticsRef /Users/travis/.choosenim/toolchains/nim-1.0.4/lib/pure/httpclient.nim(288) getDefaultSSL /Users/travis/.choosenim/toolchains/nim-1.0.4/lib/pure/net.nim(543) newContext SIGBUS: Illegal storage access. (Attempt to read from nil?)

nixfreak commented 4 years ago

./choosenim-0.5.1_macosx_amd64_debug devel [10:33:20] Traceback (most recent call last) /Users/travis/build/dom96/choosenim/src/choosenim.nim(266) choosenim /Users/travis/build/dom96/choosenim/src/choosenimpkg/telemetry.nim(120) loadAnalytics /Users/travis/.nimble/pkgs/analytics-0.2.0/analytics.nim(63) newAsyncAnalytics /Users/travis/.choosenim/toolchains/nim-1.0.4/lib/pure/httpclient.nim(534) newAnalyticsRef /Users/travis/.choosenim/toolchains/nim-1.0.4/lib/pure/httpclient.nim(288) getDefaultSSL /Users/travis/.choosenim/toolchains/nim-1.0.4/lib/pure/net.nim(543) newContext SIGBUS: Illegal storage access. (Attempt to read from nil?)

nixfreak commented 4 years ago

Just tried to install via Curl curl https://nim-lang.org/choosenim/init.sh -sSf | sh choosenim-init: Downloading choosenim-0.5.1_macosx_amd64 SIGBUS: Illegal storage access. (Attempt to read from nil?)

Is there another way I can update nim to the latest without building ? Should I use brew ?

genotrance commented 4 years ago

On OSX, we do not ship official binaries yet, it has to be built even with choosenim. I'm not sure if Nim is in brew or what version it is at either, easiest is to build from source - shouldn't take too long.

nixfreak commented 4 years ago

Ok thanks for the fast update

nixfreak commented 4 years ago

Not sure if this is relevant or not but I have pasted below the build script using brew on Osx. brew doesn't have the latest release, after this I will edit the ruby files and build again.

brew info nim [12:03:12] nim: stable 1.0.4 (bottled), HEAD Statically typed compiled systems programming language https://nim-lang.org/ Not installed From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/nim.rb ==> Options --HEAD Install HEAD version ==> Analytics install: 498 (30 days), 3,006 (90 days), 10,891 (365 days) install-on-request: 479 (30 days), 2,830 (90 days), 10,010 (365 days) build-error: 0 (30 days)

brew install nim --HEAD

==> Cloning https://github.com/nim-lang/Nim.git Cloning into '/Users/aaronm/Library/Caches/Homebrew/nim--git'... ==> Checking out branch devel Already on 'devel' Your branch is up to date with 'origin/devel'. ==> Cloning https://github.com/nim-lang/csources.git Cloning into '/Users/aaronm/Library/Caches/Homebrew/nim--csources--git'... Checking out files: 100% (10876/10876), done. ==> Checking out branch master Already on 'master' Your branch is up to date with 'origin/master'. ==> /bin/sh build.sh ==> bin/nim c -d:release koch ==> ./koch boot -d:release -d:useLinenoise ==> ./koch tools ==> ./koch geninstall ==> /bin/sh install.sh /usr/local/Cellar/nim/HEAD-32f0910 🍺 /usr/local/Cellar/nim/HEAD-32f0910: 436 files, 14.4MB, built in 14 minutes 6 seconds

nim -v [12:18:17] Nim Compiler Version 1.0.4 [MacOSX: amd64] Compiled at 2019-11-27 Copyright (c) 2006-2019 by Andreas Rumpf

git hash: c8998c498f5e2a0874846eb31309e1d1630faca6 active boot switches: -d:release

Edited the ruby file for brew -- NOTE you have to uninstall first brew uninstall nim Didn't work , I did something wrong , just going to build instead.

dom96 commented 4 years ago

So this is the line that is crashing: https://github.com/nim-lang/Nim/blob/devel/lib/pure/net.nim#L543

I wonder if some versions of macOS have an OpenSSL which disallows this. @nixfreak what's your macOS version? Can you try the following code and see whether it works for you?

import httpclient
let client = newHttpClient()
echo(client.getContent("https://google.com")

Make sure to compile with -d:ssl.

nixfreak commented 4 years ago

yeah I'll try now Osx Version 10.15.2

$ ./getContent [8:28:46] <!doctype html>Google




 

Advanced search

<div class="szppmdbYuttmiddle-slot-promo" data-ved="0ahUKEwjumpHYwKbnAhXJSt8KHTBrDasQnIcBCAQ">Complete a Privacy Checkup to make sure your settings are still right for you

© 2020 - Privacy - Terms

dom96 commented 4 years ago

That's odd and suggests to me that there may be a bug in how choosenim was built.

nixfreak commented 4 years ago

Yeah I still can't install via choosnim I had to build nim and now I have 1.1.1 installed. Is there anyway we can install multiple versions of nim on one system? Sort of of like a virtualenv ? Then I can test quite a bit.

genotrance commented 4 years ago

https://github.com/dom96/choosenim/issues/139 is also similar.

genotrance commented 4 years ago

I did some additional testing and was able to reproduce this issue.

My macOS 10.14.6 with clang 11.0.0 has LibreSSL 2.6.5 installed in /usr/lib. That is what I get when I run openssl version. Both libssl.dylib and libcrypto.dylib are linked to libxxx.35.dylib, whatever that version means. I also have 0.9.7, 0.9.8, 41, 42, 43 and 44 in the folder. The Nim openssl wrapper loads libxxx.dylib first so v35 is getting loaded and it all works.

Now, brew has installed two additional versions of openssl in the cellar, 1.0.2s and 1.1.1.d. I tried testing both those versions with export DYLD_LIBRARY_PATH=/usr/local/opt/openssl@1.x/lib, one at a time, and could get the system to crash with 1.1.1.d as reported but it works fine with 1.0.2s.

I don't know why this version of openssl is not working with choosenim but hope someone has some ideas to proceed here.

timotheecour commented 4 years ago

@genotrance did you see https://github.com/dom96/choosenim/issues/122#issuecomment-576043187

which means error isn't caught here, which points to your code @genotrance :)

there could be more than 1 bug involved here, potentially.

genotrance commented 4 years ago

Yes but I focused first on the openssl variation since that's been reported more often.

genotrance commented 4 years ago

As suggested by @timotheecour in IRC, I recompiled choosenim with openssl 1.1.1d as the active version using DYLD_LIBRARY_PATH and it compiled fine. However, it crashes in that window just like before.

If I switch to another window using the default libressl version or the openssl version of 1.0.2s, it works again. So it doesn't matter what version of library we build against, only runtime matters.

It appears our Nim openssl wrapper has some bug with the 1.1.1d version of openssl on OSX.

cc @Araq @treeform @krux02 @SolitudeSF

treeform commented 4 years ago

I spent a ton of time on openssl crap. My opinion openssl project breaks their API all the time but you are forced to use it, as it's practically part of the os. Just find nim-openssl pairing that works and stick with it. It's very fregile. It's easy to get openssl microversion that is not compatible or just plain buggy.

genotrance commented 4 years ago

This is a duplicate of https://github.com/nim-lang/Nim/issues/9419 and can be tracked there. Just replace libpq with libcurl in this case.

dom96 commented 4 years ago

This is a duplicate of nim-lang/Nim#9419 and can be tracked there. Just replace libpq with libcurl in this case.

So the root cause is loading libcurl + openssl in the same program (where libcurl loads a different openssl version)? We should start a bug hall of fame just for this :)

Thank you for testing this @genotrance, can you confirm by compiling choosenim without libcurl and see if that fixes the problem? (you can just comment this out to do that: https://github.com/dom96/choosenim/blob/master/src/choosenim.nims#L2).

genotrance commented 4 years ago

can you confirm by compiling choosenim without libcurl and see if that fixes the problem?

Just tried this test and it works fine without libcurl. Compile it back in and it crashes again.

genotrance commented 4 years ago

As mentioned in the Nim issue, we can fix this quite easily for choosenim.

metasyn commented 4 years ago

Nice job ya'll! Been following this just to see what happens after opening it half a year ago

D-Nice commented 4 years ago

Issue is closed, but I'm still experiencing it on github actions:

https://github.com/D-Nice/argon2_bind/runs/456797271?check_suite_focus=true

Run curl https://nim-lang.org/choosenim/init.sh -sSf | sh -s -- -y
  curl https://nim-lang.org/choosenim/init.sh -sSf | sh -s -- -y
  shell: /bin/bash -e {0}
choosenim-init: Downloading choosenim-0.5.1_macosx_amd64
    Prompt: Can choosenim record and send anonymised telemetry data? [y/n]
        ... Anonymous aggregate user analytics allow us to prioritise
        ... fixes and features based on how, where and when people use Nim.
        ... For more details see: https://goo.gl/NzUEPf.
SIGBUS: Illegal storage access. (Attempt to read from nil?)
    Answer: Forced Yes
##[error]Process completed with exit code 1.

OT: but especially for CI/headless environments, shouldn't the analytics prompt be opt-out by default, or can a -n flag be passed.

genotrance commented 4 years ago

Issue has been fixed in #head but a new release hasn't been cut yet. We are close.