Closed cstrahan closed 7 months ago
cstrahan
commented
8 years ago I'm using v8 version 3.16.14.11. I realize now that, a couple months ago, we updated from 3.16.14 in NixOS's packages. So I tried with 3.16.14, and I can still reproduce this.
cstrahan
commented
8 years ago I wrote the following bit of C++ to rule out problems with my libv8.so:
#include <v8.h>
int main (int argc, char *argv[]) {
v8::HandleScope handle_scope;
v8::Persistent<v8::Context> context = v8::Context::New();
v8::Context::Scope context_scope(context);
v8::TryCatch trycatch;
auto source =
v8::String::New(
"/*”*/function f(){ }f()"
);
auto script = v8::Script::New(source);
script->Run();
}That runs fine, so I believe the problem resides with therubyracer. @cowboyd are you able to reproduce this? I'm willing to lend a hand in any way I can, just let me know if there's something I can do to help isolate the cause of this problem.
cowboyd
commented
8 years ago Do you find this problem on the v8 4.5 branch? https://github.com/cowboyd/therubyracer/tree/upgrade-to-v8-4.5
cstrahan
commented
8 years ago @cowboyd I can give that a go. Do you know if Google provides a tarball for the 4.5.x series of v8? The following has worked for 3.x: https://commondatastorage.googleapis.com/chromium-browser-official
cstrahan
commented
8 years ago @cowboyd I've built v8-4.5.95.1 from source, and tried using your 4.5 branch. Unfortunately, I get the following after require 'v8':
lib/v8/init.so: undefined symbol: _ZN2v88platform21CreateDefaultPlatformEiHere's the output when I run nm libv8.so: http://sprunge.us/IZUE
I'm not sure why the platform namespace appears to be unaccounted for...
cstrahan
commented
8 years ago So, I've spent some time studying the *.gyp files, and I think I'm slowly getting a sense of how it works, but even then, I'm still pretty lost.
From what I've gathered, the $prefix/bin/d8 executable should have the v8_libplatform target statically linked in, and from running nm on it, I know that assumption is correct.
I'm now curious what we're doing differently that would result in v8_libplatform being linked into your libv8.so, but not in mine.
cstrahan
commented
8 years ago I tried adding a dependency to v8_libplatform on the v8 target: https://github.com/v8/v8/blob/ea6801b77e285926ac41cc18a3b34d8b42eb3134/tools/gyp/v8.gyp#L42
That seems to result in linking in the platform stuff: http://sprunge.us/VgBQ
But I still get the "undefined symbol" error. It looks like those symbols are local instead of global.
cstrahan
commented
8 years ago Also, I don't see where you're passing component=shared as a make flag, which makes me wonder how you create the shared library.
cowboyd
commented
8 years ago The answer is that by default, the libv8 gem will build a static library, not a shared one. Did you build the v8 source directly from v8, or using the libv8 gem?
By default, the libv8 gem will compile static archives libv8_platform.a, libv8_parser.a, etc... which we then link into the v8/init.so in the ruby extension, so that the ruby extension is a dynamically linked library, that contains the entirety of the v8 object code.
If you are compiling therubyracer against system libraries (using --with-system-v8 option on libv8), and you're using component=shared then my question is, how many shared libraries does the v8 build generate, and we'll probably have to modify the libv8 gem to link against them (beyond just -lv8) so we'd have to add -lv8_platform -lv8_parser -lv8_snapshot
Anyhow, that's my guess.
cstrahan
commented
8 years ago By default, the libv8 gem will compile static archives libv8_platform.a, libv8_parser.a, etc... which we then link into the v8/init.so in the ruby extension, so that the ruby extension is a dynamically linked library, that contains the entirety of the v8 object code.
Ah, that makes sense.
Here's a look at the relevant bits from the build directory (I pruned unrelated stuff):
build-dynamic
├── build
│ └── All.target.mk
├── Makefile
└── Release
├── cctest
├── d8
├── hello-world
├── lib.target
│ └── libv8.so
├── mksnapshot
└── obj.target
└── tools
└── gyp
├── js2c.stamp
├── libv8_base.a
├── libv8_libbase.a
├── libv8_libplatform.a
├── libv8_nosnapshot.a
├── libv8_snapshot.a
├── libv8.so
└── v8_maybe_snapshot.stampSo, the dynamic build doesn't link libv8_libplatform.a into libv8.so - although it does link it into executables like d8. When I try to hack the build's gyp files per my earlier comment
, the symbols are there, but nm reports that they are local symbols and thus not visible.
I'm not sure what the "right" thing to do is. Have therubyracer dynamically link libv8.so while statically linking in libv8_libplatform.a (that's how the binaries like d8 are compiled)? Or try to produce a libv8.so with libv8_libplatform.a linked in (my C++ compiler tool-chain knowledge is too rusty to know how to fix the visibility of the symbols, and that also deviates from what the authors are doing)?
cowboyd
commented
8 years ago My inclination is to say that it should dynamically link libv8.so, and then statically link libv8_libplatform.a into the ruby extension.
cstrahan
commented
8 years ago I tried looking through the gyp file for any differences between how the libv8_libplatform and v8_base targets are defined, and I found this under v8_base:
['component=="shared_library"', {
'defines': [
'BUILDING_V8_SHARED',
'V8_SHARED',
],
}]Which lead me to the definition of V8_EXPORT:
#if V8_HAS_ATTRIBUTE_VISIBILITY && defined(V8_SHARED)
# ifdef BUILDING_V8_SHARED
# define V8_EXPORT __attribute__ ((visibility("default")))
# else
# define V8_EXPORT
# endif
#else
# define V8_EXPORT
#endifWhich then lead me to reading the GCC docs on C++ visibility.
If I'm reading include/libplatform/libplatform.h correctly, it looks like CreateDefaultPlatform isn't exported. That header includes include/v8-platform.h, which doesn't include include/v8.h, and thus neither of those two headers export anything via V8_EXPORT.
I'm not entirely sure what to make of that, but I figured I'd share what I've found. If I'm right about the whole exporting thing, then I would think it would be unreasonable to expect any distributions to modify both the header files and the gyp build files in order to have a unified libv8.so, so perhaps dynamically linking libv8.so (if available, perhaps falling back to statically linking the *.a if not) and statically linking libv8_libplatform.a is the way to go.
In the NixOS package I'm writing, I think I'll build a dynamic library while also including all of the *.a in the package too.
cstrahan
commented
8 years ago My inclination is to say that it should dynamically link libv8.so, and then statically link libv8_libplatform.a into the ruby extension.
Ah, just saw your comment. Yeah, I think that's the way to go.
cstrahan
commented
8 years ago I modified the libv8 gem's System configuration to add:
$libs = context.send(:append_library, $libs, 'v8_libplatform')And when I try to build therubyracer, I get the following:
linking shared-object v8/init.so
/nix/store/6sqj15q4snqmydj5cq5hz9fzr6a031qs-v8-4.5.95.1/lib/libv8_libplatform.a: could not read symbols: Malformed archiveI've tried using the archives from both a dynamic and static build of v8, and both approached result in the above error.
From googling, it looks like you've seen similar reports in the past. Do you know what the root cause was?
cowboyd
commented
8 years ago Indeed, this is usually because the build generated a thin archive that only contains symbolic references to the filesystem and not the actuall object code. Althought it should work if you're building therubyracer and libv8 on the same machine...
cstrahan
commented
8 years ago Indeed, this is usually because the build generated a thin archive that only contains symbolic references to the filesystem and not the actuall object code.
Ah, that makes perfect sense. I believe that explains why, when enumerating with ar -t, the full paths are given to the original object files. I wasn't familiar with with thin archives, so I'm glad you pointed that out - I would have been stumped for a long while (tried fruitlessly to use readelf and others to gain some insight).
Althought it should work if you're building therubyracer and libv8 on the same machine...
I'm using the (as of yet unfinished) package I wrote for v8, and any build artifacts that I don't explicitly copy over to the prefix are deleted at the end of the build process, so those object files are gone by the time I try to build the therubyracer.
I think I can convert the thin archives to regular ones and have everything work. I'll give that a go after work today and let you know if that fixes the build, and if the 4.5 branch also (hopefully) fixes the segfault I reported.
Thanks for your help!
ignisf
commented
8 years ago I think I can convert the thin archives to regular ones and have everything work.
If you'll be trying to produce non-thin archives on the 4.5 branch, take a look at these:
https://github.com/cowboyd/libv8/commit/9999e40d597110bef5905a7716bebe4462eb83e6 https://github.com/cowboyd/libv8/pull/178#issuecomment-120828269 https://github.com/cowboyd/libv8/pull/185
cstrahan
commented
8 years ago @ignisf Awesome - thanks so much for pointing out those patches! That should solve the thin archive problem, and probably also fix unforeseen problems regarding PIC. Also, disabling tests is nice while I'm trying to iterate on this quickly.
cstrahan
commented
8 years ago Hm.... therubyracer now builds successfully, but fails with:
symbol lookup error: /home/cstrahan/.gem/extensions/x86_64-linux/2.2.0/therubyracer-0.12.1/v8/init.so: undefined symbol: _ZN2v84base5MutexC1EvPer nm, only the v8_libplatform symbols are defined - the rest are undefined. ldd reports the correct rpath for libv8.so. Here are the linker flags from the Makefile:
LIBS = $(LIBRUBYARG_SHARED) -lv8 -lv8_libplatform -lpthread -lpthread -ldl -lcrypt -lm -lc(In what order should the the two v8 libs be specified? Does it matter here?)
cstrahan
commented
8 years ago I tried flipping the order of the two libs, but that didn't help.
cstrahan
commented
8 years ago Oh god:
00000000005fbfa0 t _ZN2v84base5MutexC1EvI really feel for you. Building v8 is an exercise in frustration.
cstrahan
commented
8 years ago Tried googling for "lv8 lv8_libplatform" to see if anyone else is doing what I'm trying to do. Looks like at least someone has some experience with this: https://github.com/pmed/v8pp/blob/9a5e206f1281a5c6e1cb7c414163b9a9619a3237/Makefile
@pmed do you think you could give some advice, pretty please? You'd be my hero :).
pmed
commented
8 years ago @cstrahan, V8 building might be a painful process. :) I have failed to use v8pp with Travis due v8 build issues.
Anyway, this base part in the missing symbol name _ZN2v84base5MutexC1Ev might give a clue to link against an additional libv8_libbase.a or libv8_base.a or both of them, which probably is used in libv8_libplatform.
@cstrahan, please let me know whether you managed to disable V8 tests. They take a lot of time and make the build process really slow.
ignisf
commented
8 years ago cstrahan
commented
8 years ago @pmed Aha! Thanks for pointing that out - now everything is working! And I can happily report that the segfault doesn't happen on this branch! I also tried building with v8 4.5.107, and that works too.
To help future people compiling v8, you can pass V=1 as a make flag when invoking the Makefile generated by gyp, which will give you verbose output. For reference, here's the line where the d8 executable is linked:
g++ -pthread -fuse-ld=gold -B/tmp/nix-build-v8-4.5.95.1.drv-17/v8-4.5.95.1-src/third_party/binutils/Linux_x64/Release/bin -fuse-ld=gold -B/tmp/nix-build-v8-4.5.95.1.drv-17/v8-4.5.95.1-src/third_party/binutils/Linux_x64/Release/bin -L/nix/store/80a3md2fnc41f8n30ss2g7g65v0sqmia-icu4c-55.1/lib -licui18n -licuuc -licudata -m64 -m64 -Wl,-rpath=\$ORIGIN/lib.target/ -Wl,-rpath-link=\/tmp/nix-build-v8-4.5.95.1.drv-17/v8-4.5.95.1-src/out/Release/lib.target/ -o /tmp/nix-build-v8-4.5.95.1.drv-17/v8-4.5.95.1-src/out/Release/d8 -Wl,--start-group /tmp/nix-build-v8-4.5.95.1.drv-17/v8-4.5.95.1-src/out/Release/obj.target/d8/src/d8.o /tmp/nix-build-v8-4.5.95.1.drv-17/v8-4.5.95.1-src/out/Release/obj.target/d8/src/startup-data-util.o /tmp/nix-build-v8-4.5.95.1.drv-17/v8-4.5.95.1-src/out/Release/obj.target/d8/src/d8-readline.o /tmp/nix-build-v8-4.5.95.1.drv-17/v8-4.5.95.1-src/out/Release/obj.target/d8/src/d8-posix.o /tmp/nix-build-v8-4.5.95.1.drv-17/v8-4.5.95.1-src/out/Release/obj.target/tools/gyp/libv8.so /tmp/nix-build-v8-4.5.95.1.drv-17/v8-4.5.95.1-src/out/Release/obj.target/tools/gyp/libv8_libplatform.a /tmp/nix-build-v8-4.5.95.1.drv-17/v8-4.5.95.1-src/out/Release/obj.target/tools/gyp/libv8_libbase.a -Wl,--end-group -lreadline -ldl -lrt -licui18n -licuuc -licudataHad I checked that earlier, I would have realized that I also needed to ensure -lv8_libbase was given.
Here's the patch that fixes ext/libv8/location.rb:
diff --git a/ext/libv8/location.rb b/ext/libv8/location.rb
index ddb63b2..3ce2fd5 100644
--- a/ext/libv8/location.rb
+++ b/ext/libv8/location.rb
@@ -53,6 +53,8 @@ module Libv8
context.send(:dir_config, 'v8')
context.send(:find_header, 'v8.h') or fail NotFoundError
context.send(:find_header, 'libplatform/libplatform.h') or fail NotFoundError
+ $libs = context.send(:append_library, $libs, 'v8_libbase')
+ $libs = context.send(:append_library, $libs, 'v8_libplatform')
context.send(:have_library, 'v8') or fail NotFoundError
end
I would use context.send(:have_library, 'v8_libbase'), but, unless explicitly given a symbol to find, have_library attempts to find a symbol main, which will fail when given a static library. I suppose I could use one of the mangled C++ symbols, but I don't know if that's a good idea (I think that source-compatible changes in v8 could still result in ABI changes, making the symbol check fragile at best).
@ignisf @pmed @cowboyd: thank you for the help!
@cowboyd: when are you thinking about releasing 4.5? This some exciting work in general, and more specifically, should get GitLab working on NixOS again.
cstrahan
commented
8 years ago Also, it looks like Chrome 4.6 went stable on Oct. 13th: http://googlechromereleases.blogspot.com/2015/10/stable-channel-update.html
Current version info available here: https://omahaproxy.appspot.com/
Assuming the API didn't change much, it might be worthwhile jumping straight to v8 4.6.x.
cstrahan
commented
8 years ago For reference, here's the package I wrote for NixOS: https://github.com/NixOS/nixpkgs/commit/abf7301cc9a41dd212f87db7e918c951ae3a6a69
pmed
commented
8 years ago @cstrahan, I finally found the way how to build only V8 libraries without .gyp files patching. See build-v8.sh script in a commit pmed/v8pp@9222535cdf944cc3dac6ba122686539b83334ebf
The idea is to generate a Makefile with gyp manually and then to run make for v8 and v8_libplatform targets only.
I had to create an additional v8_options.gypi file to override V8 gyp build options, in particular to set standalone_static_library: 1 for static libraries.
I found that
uglifierwas segfaulting in thetherubyracer, and managed to reduce the problematic JS code down to a rather bizarre one-liner:The important bits seem to be the smart-quote character, the number of characters/bytes in the function body, and the function invocation. I suspect that there's some sort of unicode/UTF-8 related problem at play.
Here's the debug output after segfaulting: