Support V8 C++ APIs for "nan" addons and other packages to work

xhyrom commented 1 year ago

Affected packages

[ ] better-sqlite3
[ ] node-canvas@v2 (node-canvas v3 is supported)
[ ] midi
[ ] libxmljs
[ ] bignum
[ ] zeromq
[ ] node-libcurl
[ ] bson-ext
[ ] heapdump
[ ] @datadog/native-metrics
[ ] tiny-secp256k1
[ ] tree-sitter-c-sharp
[ ] chacha-native
[ ] @sentry/profiling-node
[ ] pdfjs-dist
[ ] gl
[ ] leveldown / pouchdb
[x] sleep
[x] cpu-features
[ ] odbc
[ ] msgpackr
[ ] re2
[ ] node-pty
[ ] i2c

Functions implemented

[x] v8::Isolate::GetCurrent()
[x] v8::Isolate::TryGetCurrent()
[x] v8::Isolate::GetCurrentContext()
[x] v8::Number::New(v8::Isolate*, double)
[x] v8::Number::Value() const
[x] v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int)
[x] v8::String::WriteUtf8(v8::Isolate*, char*, int, int*, int) const
[x] v8::api_internal::ToLocalEmpty()
[x] v8::String::Length() const
[x] v8::External::New(v8::Isolate*, void*)
[x] v8::External::Value() const
[x] v8::Object::New(v8::Isolate*)
[x] v8::Object::Set(v8::Local<v8::Context>, v8::Local<v8::Value>, v8::Local<v8::Value>)
[x] v8::Object::SetInternalField(int, v8::Local<v8::Data>)
[x] v8::Object::SlowGetInternalField(int)
[x] v8::HandleScope::HandleScope(v8::Isolate*)
[x] v8::HandleScope::~HandleScope()
[x] v8::FunctionTemplate::GetFunction(v8::Local<v8::Context>)
[x] v8::FunctionTemplate::New(v8::Isolate*, void (*)(v8::FunctionCallbackInfo<v8::Value> const&), v8::Local<v8::Value>, v8::Local<v8::Signature>, int, v8::ConstructorBehavior, v8::SideEffectType, v8::CFunction const*, unsigned short, unsigned short, unsigned short) (only when all parameters past data are their default values)
[x] v8::ObjectTemplate::NewInstance(v8::Local<v8::Context>)
[x] v8::ObjectTemplate::SetInternalFieldCount(int)
[x] v8::ObjectTemplate::InternalFieldCount() const
[x] v8::ObjectTemplate::New(v8::Isolate*, v8::Local<v8::FunctionTemplate>) (only when no FunctionTemplate is provided)
[x] v8::EscapableHandleScopeBase::EscapeSlot(unsigned long*)
[x] v8::EscapableHandleScopeBase::EscapableHandleScopeBase(v8::Isolate*)
[x] node_module_register
[x] v8::Function::SetName(v8::Local<v8::String>)
[x] v8::Value::IsBoolean() const
[x] v8::Boolean::Value() const
[x] v8::Value::FullIsTrue() const
[x] v8::Value::FullIsFalse() const
[x] v8::EscapableHandleScope::EscapableHandleScope(v8::Isolate*)
[x] v8::EscapableHandleScope::~EscapableHandleScope()
[x] v8::Value::IsObject() const
[x] v8::Value::IsNumber() const
[x] v8::Value::IsUint32() const
[x] v8::Value::Uint32Value(v8::Local<v8::Context>) const
[x] v8::Value::IsUndefined() const
[x] v8::Value::IsNull() const
[x] v8::Value::IsNullOrUndefined() const
[x] v8::Value::IsTrue() const
[x] v8::Value::IsFalse() const
[x] v8::Value::IsString() const
[x] v8::Boolean::New(v8::Isolate*, bool)
[x] v8::Object::GetInternalField(int)
[ ] node::AddEnvironmentCleanupHook(v8::Isolate*, void (*)(void*), void*)
[ ] node::RemoveEnvironmentCleanupHook(v8::Isolate*, void (*)(void*), void*)
[x] v8::HandleScope::CreateHandle(v8::internal::Isolate*, unsigned long)
[ ] v8::internal::IsolateFromNeverReadOnlySpaceObject(unsigned long)
[ ] v8::Array::New(v8::Isolate*, v8::Local<v8::Value>*, unsigned long)
[ ] v8::Object::Get(v8::Local<v8::Context>, v8::Local<v8::Value>)

Major areas of unimplemented functionality

[ ] Array
[ ] ObjectTemplate with a constructor function
[ ] Less-simple cases of FunctionTemplate
[ ] ArrayBuffer
[ ] bun install fixes (need to pass node-gyp the right version)
[x] Doubles
[x] UTF-16 strings
[x] Persistent handles

What version of Bun is running?

0.8.0+ad326b77342dd3d8585a30b7da803d32f9c11fe2

What platform is your computer?

Linux 6.4.10-artix1-1 x86_64 unknown

What steps can reproduce the bug?

import Database from 'better-sqlite3';
const db = new Database('foobar.db', {});

console.log(db.prepare("CREATE TABLE TEST").run());

What is the expected behavior?

Run as normally

What do you see instead?

/usr/bin/bun: symbol lookup error: /home/hyro/Workspace/booo/node_modules/better-sqlite3/build/Release/better_sqlite3.node: undefined symbol: node_module_register

Additional information

Related thread: 1144181146809270332 Message: https://discord.com/channels/876711213126520882/1144181146809270332/1144184224006557747

All versions are affected

floydjones1 commented 1 year ago

This also happens for node-libcurl

issadarkthing commented 1 year ago

I also encounter this error

Zykatious commented 1 year ago

this also happens for bson-ext

autoscatto commented 1 year ago

also for heapdump

vroudge commented 1 year ago

same for @datadog/native-metrics

uditdc commented 1 year ago

Likewise for tiny-secp256k1

ozyman42 commented 1 year ago

Also for cdktf-cli #5306

obedm503 commented 1 year ago

Also happens with tree-sitter-c-sharp

Scitz0 commented 1 year ago

Same error for chacha-native

metawrap-dev commented 1 year ago

same for node-canvas

4i8 commented 1 year ago

bun: symbol lookup error: /root/rut/node_modules/canvas/build/Release/canvas.node: undefined symbol: node_module_register same for canvas

edi9999 commented 1 year ago

Same for libxmljs

floydjones1 commented 1 year ago

How can we get a little more attention to this. This is holding me back from using bun in production. :cry: node-libcurl

bun: symbol lookup error: /home/.../node_modules/node-libcurl/lib/binding/node_libcurl.node: undefined symbol: node_module_register

M-Gonzalo commented 1 year ago

bun: symbol lookup error: /root/rut/node_modules/canvas/build/Release/canvas.node: undefined symbol: node_module_register same for canvas

me too

frani commented 1 year ago

similar error:

bun: symbol lookup error: /usr/src/app/node_modules/@sentry/profiling-node/lib/sentry_cpu_profiler-linux-x64-glibc-115.node: undefined symbol: _ZN2v87Isolate10GetCurrentEv

xbura1han commented 1 year ago

Also happening for Sentry & bcrypt Sentry code: Code:

const Sentry = require('@sentry/bun');
const { ProfilingIntegration } = require("@sentry/profiling-node")

Error:

bun: symbol lookup error: /root/projects/userBackend/node_modules/@sentry/profiling-node/lib/sentry_cpu_profiler-linux-x64-glibc-115.node: undefined symbol: _ZN2v87Isolate10GetCurrentEv

movy commented 12 months ago

Same for https://github.com/TulipCharts/tulipnode

bun: symbol lookup error: node_modules/tulind/lib/binding/Release/node-v115-linux-x64/tulind.node: undefined symbol: node_module_register

Jarred-Sumner commented 12 months ago

node_module_register uses & expects V8 C++ APIs. Bun uses JavaScriptCore, an entirely different JavaScript engine. This is especially challenging for Bun to support, but not without precedent. node-jsc did it.

Wamy-Dev commented 8 months ago

Same with New Relic

Arilith commented 8 months ago

Same for https://github.com/zeromq/zeromq.js

bun: symbol lookup error: node_modules/zeromq/prebuilds/linux-x64/node.napi.glibc.node: undefined symbol: uv_async_init

bonald commented 8 months ago

node_libcurl.node: undefined symbol: node_module_register wen node-libcurl

deadblue22 commented 7 months ago

bun: symbol lookup error: .../node_modules/gl/build/Release/webgl.node: undefined symbol: node_module_register

ellis commented 7 months ago

Trying to use @testcontainers/postgresql also leads to the node_module_register error

ibilux commented 6 months ago

Same in pdfjs-dist (related to canvas).

terrablue commented 6 months ago

This blocks using node-gtk in combination with bun build --compile to produce standalone GTK apps.

celso-alexandre commented 6 months ago

Trying to use pcsclite also leads to the node_module_register error

jimying commented 6 months ago

Also node-pty , Release/pty.node: undefined symbol: node_module_register

ralyodio commented 6 months ago

I also need this for my project.

arthurwolf commented 6 months ago

I also get this error with the opencv bindings:

bun: symbol lookup error: /home/arthur/dev/ai/manga/node_modules/opencv4nodejs/build/Release/opencv4nodejs.node: undefined symbol: node_module_register

what is the status on this, is there anything I can do to get it to run ?

826327700 commented 6 months ago

I also encountered this problem. This question has been raised for a long time. Is there still no solution?

Jarred-Sumner commented 6 months ago

This question has been raised for a long time. Is there still no solution?

The solution is we need to hire more engineers because there's too much to do

We are hiring in San Francisco

826327700 commented 6 months ago

这个问题提了好久了，还是没有解决办法吗？

解决方案是我们需要雇佣更多的工程师，因为有太多事情要做

我们正在旧金山招聘

Sorry, I can't help with this, but I think we should pay attention to bun's compatibility with npm dependency packages generated by c++ static link libraries, otherwise bun will be difficult to apply to production projects. I also sincerely hope that bun can be liked and used by more people.

M-Gonzalo commented 6 months ago

This question has been raised for a long time. Is there still no solution?

The solution is we need to hire more engineers because there's too much to do

We are hiring in San Francisco

I'm sure you've already considered this, but hiring remote workers would allow you to find talent from all around the world. Not many people can (or want to) move to SF...

arthurwolf commented 6 months ago

I would expect this specific issue (and others like it?) have a huge cost/benefit ratio in terms of bringing new users to bun. it's the only thing at the moment blocking me from using bun for everything...

solve this, and many more will be able to use bun, meaning more eyes on bun, meaning more ease hiring...

On Tue, May 28, 2024 at 4:54 PM Gonzalo Muñoz @.***> wrote:

This question has been raised for a long time. Is there still no solution?

The solution is we need to hire more engineers because there's too much to do

We are hiring https://apply.workable.com/oven/j/A7A1388873/ in San Francisco

I'm sure you've already considered this, but hiring remote workers would allow you to find talent from all around the world. Not many people can (or want to) move to SF...

— Reply to this email directly, view it on GitHub https://github.com/oven-sh/bun/issues/4290#issuecomment-2135436843, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA2SFKJPCILBWEVZBLJRJTZESLC5AVCNFSM6AAAAAA34UURSSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZVGQZTMOBUGM . You are receiving this because you commented.Message ID: @.***>

--

勇気とユーモア

Wamy-Dev commented 6 months ago

Im interested, but my portfolio isnt wide enough in terms of the langs they use...

maietta commented 6 months ago

This question has been raised for a long time. Is there still no solution?

The solution is we need to hire more engineers because there's too much to do

We are hiring in San Francisco

If I didn't hate on SF, I'd be applying in a heartbeat.

trevorsargent commented 5 months ago

node_rdkafka is also affected. I'd also move to bun in a heartbeat if this was fixed.

😢

ralyodio commented 5 months ago

any update on this?

M-Gonzalo commented 5 months ago

any update on this?

yeah I got a notification about it but it was just some dude asking are we there yet

1bye commented 5 months ago

Same with cpu-features

Eyepan commented 5 months ago

Also for NX. trying to port an old repo from pnpm to nx causes the same error.

Tjerk-Haaye-Henricus commented 4 months ago

Holding me back right now. Its open since August 2023, Is there any eta on this ?

JonathanCabezas commented 3 months ago

Same problem happens with the odbc package.

dayblox commented 3 months ago

same for cap

190n commented 3 months ago

Now that #12821 is merged, the next version of Bun will include some of the first working V8 functions. See the top of this issue for more detailed status. Packages more complex than cpu-features or sleep are likely to still run into cases I have not covered yet, and there may be functions where we're never able to bridge the gap between how V8 and JavaScriptCore represent data, but we're excited to have the first version of this functionality landing soon for testing.

chriskrogh commented 3 months ago

@190n thank you for the hard work! based on https://github.com/oven-sh/bun/issues/5835#issuecomment-2229156850 this gets us closer to supporting node-canvas which unblocks jsdom! we can't wait to use bun at @faire

Jarred-Sumner commented 3 months ago

@190n thank you for the hard work! based on #5835 (comment) this gets us closer to supporting node-canvas which unblocks jsdom! we can't wait to use bun at @Faire

@chriskrogh you can use canvas@next instead of the latest version and it should work, as that uses napi instead of v8 APIs

chriskrogh commented 3 months ago

@Jarred-Sumner it works! 🥳 no more canvas issues on bun install!!! we only have 2 more offending packages:

llamaindex
- tries to link @datastax/astra-db-ts
- we don't have that many usages. might try to switch to ai by vercel
@grpc/grpc-js
- this one is actually used by most of our app servers, so we might need to wait for support here

but after removing those two, I was able to get a clean install which was lightning fast! thanks again for all the progress here! hopefully grpc doesn't take that much more? 🤞

KilianB commented 3 months ago

Is grpc related to v8 incompatibilities? I thought it was blocked by https2 server support

rgillan commented 3 months ago

@Jarred-Sumner et al, please note I've deleted a comment re ssh2 server issues as it is not related to the V8 APIs, will raise a separate ticket shortly (have identified a potential issue)

oven-sh / bun