search

kylef / heroku-buildpack-swift

Heroku build pack for Swift
BSD 3-Clause "New" or "Revised" License
508 stars 0 forks source link

error while loading shared libraries: libatomic.so.1: cannot open shared object file: No such file or directory #32

Closed funkenstrahlen closed 7 years ago

funkenstrahlen commented 7 years ago

I would like to build my swift code with the standard library linked statically. Because currently I get an error when I try to run my build:

./my-build-binary: error while loading shared libraries: libatomic.so.1: cannot open shared object file: No such file or directory

I think this is because it can not find the standard libraries.

How can I set the compiler flags? swift build -Xswiftc -static-stdlib

kylef commented 7 years ago

How can I set the compiler flags?

You can set SWIFT_BUILD_FLAGS in your environment variables.

funkenstrahlen commented 7 years ago

I checked my .swift-lib folder contains libFoundation.so libXCTest.so libdispatch.so libswiftCore.so libswiftGlibc.so libswiftRemoteMirror.so libswiftSwiftOnoneSupport.so

So it is actually missing. I do not know why it is missing.

I did set SWIFT_BUILD_FLAGS: -Xswiftc -static-stdlib -c release. When I try to build on heroku with this setting I get many compile time errors:

This is just an extract.

[...]
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFDate.c.o):CoreFoundation/NumberDate.subproj/CFDate.c:function __CFTSRToDispatchTime: error: undefined reference to 'dispatch_time'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFTimeZone.c.o):CoreFoundation/NumberDate.subproj/CFTimeZone.c:function __nameStringOK: error: undefined reference to 'ucal_getCanonicalTimeZoneID_55'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFTimeZone.c.o):CoreFoundation/NumberDate.subproj/CFTimeZone.c:function CFTimeZoneCopyLocalizedName: error: undefined reference to 'ucal_getTimeZoneDisplayName_55'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function ___CFSetupXMLInterface_block_invoke: error: undefined reference to 'xmlInitParser'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function ___CFSetupXMLInterface_block_invoke: error: undefined reference to 'xmlGetExternalEntityLoader'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function ___CFSetupXMLInterface_block_invoke: error: undefined reference to 'xmlSetExternalEntityLoader'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceNoNetExternalEntityLoader: error: undefined reference to 'xmlNoNetExternalEntityLoader'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _errorCallback: error: undefined reference to 'xmlCtxtGetLastError'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceSetStructuredErrorFunc: error: undefined reference to 'xmlSetStructuredErrorFunc'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceCreatePushParserCtxt: error: undefined reference to 'xmlCreatePushParserCtxt'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceCtxtUseOptions: error: undefined reference to 'xmlCtxtUseOptions'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceParseChunk: error: undefined reference to 'xmlParseChunk'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceStopParser: error: undefined reference to 'xmlStopParser'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceDestroyContext: error: undefined reference to 'xmlFreeDoc'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceDestroyContext: error: undefined reference to 'xmlFreeParserCtxt'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceSAX2GetLineNumber: error: undefined reference to 'xmlSAX2GetLineNumber'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceSAX2GetColumnNumber: error: undefined reference to 'xmlSAX2GetColumnNumber'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceSAX2InternalSubset: error: undefined reference to 'xmlSAX2InternalSubset'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceSAX2ExternalSubset: error: undefined reference to 'xmlSAX2ExternalSubset'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLInterfaceGetPredefinedEntity: error: undefined reference to 'xmlGetPredefinedEntity'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLDTDNewElementDesc: error: undefined reference to 'xmlAddElementDecl'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLDTDNewElementDesc: error: undefined reference to 'xmlNewDtd'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLDTDNewElementDesc: error: undefined reference to 'xmlAddElementDecl'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLDTDNewElementDesc: error: undefined reference to 'xmlFreeDtd'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLUnlinkNode: error: undefined reference to 'xmlHashLookup'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLUnlinkNode: error: undefined reference to 'xmlHashLookup'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLUnlinkNode: error: undefined reference to 'xmlHashLookup3'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLUnlinkNode: error: undefined reference to 'xmlHashRemoveEntry3'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLUnlinkNode: error: undefined reference to 'xmlHashLookup'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLUnlinkNode: error: undefined reference to 'xmlHashRemoveEntry'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLUnlinkNode: error: undefined reference to 'xmlHashLookup'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLUnlinkNode: error: undefined reference to 'xmlHashRemoveEntry'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLUnlinkNode: error: undefined reference to 'xmlUnlinkNode'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLUnlinkNode: error: undefined reference to 'xmlHashRemoveEntry'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLDTDNewAttributeDesc: error: undefined reference to 'xmlAddAttributeDecl'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLDTDNewAttributeDesc: error: undefined reference to 'xmlNewDtd'
remote: /app/tmp/cache/swiftenv/versions/4.0/usr/lib/swift_static/linux/libFoundation.a(CFXMLInterface.c.o):CoreFoundation/Parsing.subproj/CFXMLInterface.c:function _CFXMLDTDNewAttributeDesc: error: undefined reference to 'xmlAddAttributeDecl'
[...]

I think dynamically linking the libs is the correct way. But why does it not find the required libraries? When I build + run my code locally with dynamically linked libraries it works fine.

kylef commented 7 years ago

I've updated the example project to Swift 4.0 and I'm able to reproduce this problem (after fixing a separate problem d7f0218064d3448dedd8e796a4669bf6126a9d57):

HelloWorld: error while loading shared libraries: libatomic.so.1: cannot open shared object file: No such file or directory

Does look like there is a missing expected system library (libatomic.so):

$ heroku run bash -a testingcurassow
 ▸    heroku-cli: update available from 6.13.4 to 6.14.34-1fcf80e
Running bash on ⬢ testingcurassow... up, run.4598 (Free)
~ $ ls -al .swift-bin/
total 1816
drwx------ 2 u29672 dyno   4096 Oct  9 17:01 .
drwx------ 7 u29672 dyno   4096 Oct  9 17:03 ..
-rwx------ 1 u29672 dyno 925408 Oct  9 17:01 HelloWorld
-rwx------ 1 u29672 dyno 925328 Oct  9 17:01 example
~ $ ls -al .swift-lib/
total 22468
drwx------ 2 u29672 dyno     4096 Oct  9 17:01 .
drwx------ 7 u29672 dyno     4096 Oct  9 17:03 ..
-rwx------ 1 u29672 dyno 12472888 Oct  9 17:01 libFoundation.so
-rwx------ 1 u29672 dyno   353256 Oct  9 17:01 libXCTest.so
-rwx------ 1 u29672 dyno   772536 Oct  9 17:01 libdispatch.so
-rwx------ 1 u29672 dyno  8327584 Oct  9 17:01 libswiftCore.so
-rwx------ 1 u29672 dyno    32752 Oct  9 17:01 libswiftGlibc.so
-rwx------ 1 u29672 dyno   467864 Oct  9 17:01 libswiftRemoteMirror.so
-rwx------ 1 u29672 dyno   555216 Oct  9 17:01 libswiftSwiftOnoneSupport.so
~ $ ldd .swift-lib/libFoundation.so
...
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f754ee0f000)
    libatomic.so.1 => not found
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f754ea8d000)
    libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11 (0x00007f754e85a000)
...
    /lib64/ld-linux-x86-64.so.2 (0x00007f755348e000)
    libatomic.so.1 => not found
    libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f754d296000)

I would suspect that it is available in the cedar-14 Heroku stack and available during build time for heroku-16 but not runtime. If that is true, you should be able to build using the cedar-14 stack (https://devcenter.heroku.com/articles/cedar-14-stack) as a short term solution.

funkenstrahlen commented 7 years ago

@kylef Thank you a lot for taking the time to take a look at it!

I do not need a short term solution right now. I can wait some time until I actually need to deploy my app. Do you think it will be possible to get this running on heroku-16 in the future? Or is this out of your control?

kylef commented 7 years ago

Do you think it will be possible to get this running on heroku-16 in the future?

I certainly hope so, if it is present at build time we can copy it over the the .swift-libs directory. If it isn't then we can likely download it from Ubuntu or built ourselves. Will need to take caution for the cases when there are newer stacks which may be incompatible.

I don't have much free time to dig into this further this week but hopefully at the weekend I can invest an hour into fixing this.

funkenstrahlen commented 7 years ago

I don't have much free time to dig into this further this week but hopefully at the weekend I can invest an hour into fixing this.

Awesome! Thank you!

kylef commented 7 years ago

5e91d32db352a0a28f628f856525924caf738adc works around the missing file on heroku-16 stack, my suspicion was correct.

Please give this a try now.

vzsg commented 7 years ago

I can confirm that this change fixes the issue - I've applied the same thing directly in vapor-community/heroku-buildpack a month ago.

funkenstrahlen commented 7 years ago

Can confirm this works now. Thank you for fixing it! 😃

sharplet commented 6 years ago

I've recently created a new app on heroku-18, and it looks like this is still a problem. I've downgraded to heroku-16 (and Swift 4.1) as a workaround. Would adding another special case for heroku-18 be appropriate, or is there a better solution?

JeromeCHA commented 6 years ago

I still get the error with the lastest version and swift 4.2 and heroku-16

➜  slackBot git:(master)✗ heroku run:detached slackbot
Running slackbot on ⬢ slack-bot... done, run.3646 (Free)
Run heroku logs --app slack-bot --dyno run.3646 to view the output.
➜  slackBot git:(master)✗ heroku logs --app slack-bot --dyno run.3646
2018-10-31T16:49:02.466717+00:00 heroku[run.3646]: Starting process with command `.build/release/slack-bot`
2018-10-31T16:49:03.162037+00:00 heroku[run.3646]: State changed from starting to up
2018-10-31T16:49:06.044276+00:00 heroku[run.3646]: State changed from up to complete
2018-10-31T16:49:06.025404+00:00 heroku[run.3646]: Process exited with status 127
2018-10-31T16:49:05.946164+00:00 app[run.3646]: bash: .build/release/slack-bot: No such file or directory
kylef commented 6 years ago

@JeromeCHA That is unrelated to the original post here, inside your Procfile you should specify slack-bot as the binary and not .build/release/slack-bot

shawnce commented 5 years ago

I appear to be hitting this issue when attempting to use Swift 4.2 on heroku-18

2018-11-26T19:43:11.434784+00:00 heroku[web.1]: State changed from crashed to starting
2018-11-26T19:43:12.586336+00:00 heroku[web.1]: Starting process with command `KituraTest`
2018-11-26T19:43:14.522273+00:00 heroku[web.1]: Process exited with status 127
2018-11-26T19:43:14.440434+00:00 app[web.1]: KituraTest: error while loading shared libraries: libatomic.so.1: cannot open shared object file: No such file or directory
2018-11-26T19:43:14.540423+00:00 heroku[web.1]: State changed from starting to crashed
~ $ ls -al .swift-bin/
total 3188
drwx------ 2 u55245 dyno    4096 Nov 21 00:47 .
drwx------ 5 u55245 dyno    4096 Nov 26 19:45 ..
-rwx------ 1 u55245 dyno 3255288 Nov 21 00:47 KituraTest
~ $ ls -al .swift-lib/
total 21352
drwx------ 2 u55245 dyno     4096 Nov 21 00:47 .
drwx------ 5 u55245 dyno     4096 Nov 26 19:45 ..
-rwx------ 1 u55245 dyno 12340880 Nov 21 00:47 libFoundation.so
-rwx------ 1 u55245 dyno   313696 Nov 21 00:47 libXCTest.so
-rw------- 1 u55245 dyno   785600 Nov 21 00:47 libdispatch.so
-rwx------ 1 u55245 dyno  7400000 Nov 21 00:47 libswiftCore.so
-rwx------ 1 u55245 dyno    70936 Nov 21 00:47 libswiftGlibc.so
-rwx------ 1 u55245 dyno   518088 Nov 21 00:47 libswiftRemoteMirror.so
-rwx------ 1 u55245 dyno   416496 Nov 21 00:47 libswiftSwiftOnoneSupport.so
~ $ ldd .swift-lib/libFoundation.so
    libatomic.so.1 => not found
    /lib64/ld-linux-x86-64.so.2 (0x00007f1698f57000)
    libatomic.so.1 => not found

Adding heroku-18 to the list in https://github.com/kylef/heroku-buildpack-swift/commit/5e91d32db352a0a28f628f856525924caf738adc corrected things in my fork of the build pack.