Long Major GC events while using external buffers (Video Image Data)

david-metrica commented 2 hours ago

Version

v21.1.0

Platform

23.1.0 Darwin Kernel Version 23.1.0

Subsystem

No response

What steps will reproduce the bug?

I am sharing data from a native node C++ plugin using node-addon-api: return Napi::ArrayBuffer::New(env, frame_->data[i], width*height);

I checked that the memory being accessed from JS was the same as the one from C++ (And can confirm it works as expected).

However, whenever I access this data I get some Major GC events taking more than 100+ms in total.

The memory is freed by the C++ process and there is not any crash or weird behavior apart from this.

The ArrayBuffer is exposed using node-addon-api: InstanceAccessor("data", &Frame::data, nullptr), .

Writing frame.data in javascript causes this events.

node-addon-api uses napi_create_external_buffer when creating a Napi::ArrayBuffer

How often does it reproduce? Is there a required condition?

This occurs multiple times every 1-2s. Depends on the size of the data and how often is sent from C++ to JS.

What is the expected behavior? Why is that the expected behavior?

As the memory collection is not handled by the Major GC it should not expend time trying to free this objects.

What do you see instead?

Major GC trying to collect external buffers and spending more time depending how big this external buffers are.

Additional information

No response

RedYetiDev commented 2 hours ago

Node.js v21 is EoL. If you can reproduce this issue in v22 or v23, please leave a comment. Additionally, "a native node C++ plugin" isn't a minimal reproduction. Without a minimal reproduction, finding the root cause of the error is difficult.

david-metrica commented 1 hour ago

Replicated in v23.1.0. Minimal reproduction would be creating an ArrayBuffer with large malloc data every 20ms with a timer in a InstanceAccesor

RedYetiDev commented 1 hour ago

Can you provide some code?

david-metrica commented 1 hour ago

This is not our top priority right now and I can't provide the company's code. I will try to have a minimal reproducible example with code before end of next week.

Here is some pseudo-code until then.

JS: setTimeout every 20ms accessing a wrapper field (Example frame.data)

C++: Wrapper with InstanceAccessor("data", &Frame::data, nullptr)

Napi::Value Frame::data(const Napi::CallbackInfo &info)
{
  Napi::Env env = info.Env();
  -- Allocate Large Data (500kb or 1Mb) --
  return Napi::ArrayBuffer::New(env, data, data_size);
}

nodejs / node