[ASK] How To Use Arena in ffi

[azkadev]

Sorry, this is a question, I was looking for a way to immediately free up memory after calling the function so that the memory doesn't keep increasing, then I found a solution using arena but I can't use it,

How to use the Arena function?

Currently my script is like this

import 'dart:ffi';
import 'package:ffi/ffi.dart';

typedef MyLibStringNative = Pointer<Utf8>;
typedef MyLibReceiveNative = MyLibStringNative Function(Double timout);
typedef MyLibReceiveDart = MyLibStringNative Function(double timout);
void main(List<String> args) {
  DynamicLibrary myLib = DynamicLibrary.open("liblibrary.so");
  MyLibReceiveDart myLibReceiveFunction = myLib.lookupFunction<MyLibReceiveNative, MyLibReceiveDart>("my_function");
  MyLibStringNative update = myLibReceiveFunction(10);
  if (update.address != 0) {
    String updateString = update.toDartString();
    print(updateString);
  }
}

[azkadev]

is this code correct?

import 'dart:ffi';
import 'package:ffi/ffi.dart';

typedef MyLibStringNative = Pointer<Utf8>;
typedef MyLibReceiveNative = MyLibStringNative Function(Double timout);
typedef MyLibReceiveDart = MyLibStringNative Function(double timout);
void main(List<String> args) {
  // open library
  DynamicLibrary myLib = DynamicLibrary.open("liblibrary.so");
  // use arena
  Arena arena = Arena();
  MyLibReceiveDart myLibReceiveFunction = myLib.lookupFunction<MyLibReceiveNative, MyLibReceiveDart>("my_function");
  MyLibStringNative update = arena.using(myLibReceiveFunction(10), (p0) {});
  if (update.address != 0) {
    String updateString = update.toDartString();
    print(updateString);
  }
  // get free up memory
  arena.releaseAll();
}

[azkadev]

or is this code correct?

import 'dart:ffi';
import 'package:ffi/ffi.dart';

typedef MyLibStringNative = Pointer<Utf8>;
typedef MyLibReceiveNative = MyLibStringNative Function(Double timout);
typedef MyLibReceiveDart = MyLibStringNative Function(double timout);
void main(List<String> args) {
  // open library
  DynamicLibrary myLib = DynamicLibrary.open("liblibrary.so");
  // use arena
  //
  String? result = using((Arena arena) {
    MyLibReceiveDart myLibReceiveFunction = myLib.lookupFunction<MyLibReceiveNative, MyLibReceiveDart>("my_function");
    MyLibStringNative update = arena.using(myLibReceiveFunction(10), (p0) {});
    if (update.address != 0) {
      String updateString = update.toDartString();
      print(updateString);
      return updateString;
    }
    // get free up memory
    arena.releaseAll();
    return null;
  });

  print(result);
}

Sorry, I ask a lot of questions, I have tried various methods but after running the program for a long time the memory doesn't go down

[HosseinYousefi]

You are not providing a free method to using, so it will just run the empty (p0) {} closure.

For example if you have used malloc to allocate the Pointer<Utf8> in C, you can pass malloc.free to free it.

using ((arena) {
  arena.using(myLibReceiveFunction(10), malloc.free);
});

// The memory is now released.
// No need for writing arena.releaseAll(), the using block has taken care of calling that.

It's easier to use using ((arena) { /* ... */ }); instead of manually running arena.releaseAll(). As it accounts for exceptions and async.

[azkadev]

You are not providing a free method to using, so it will just run the empty (p0) {} closure.

For example if you have used malloc to allocate the Pointer<Utf8> in C, you can pass malloc.free to free it.

using ((arena) {
  arena.using(myLibReceiveFunction(10), malloc.free);
});

// The memory is now released.
// No need for writing arena.releaseAll(), the using block has taken care of calling that.

It's easier to use using ((arena) { /* ... */ }); instead of manually running arena.releaseAll(). As it accounts for exceptions and async.

I have tried but this error appears

free(): double free detected in tcache 2
[1] 485641 IOT instructions (core dumped) dart run

And the program immediately stops

[HosseinYousefi]

Can you post your full code?

[azkadev]

Can you post your full code?

i tried running tdlib (Telegram database library),

I managed to run it well without malloc.free but when the old program runs the memory will continue to increase, I have tried restarting / carrying out instructions from tdlib but to no avail, so I tried using arena

then add the code according to what you told me

using ((arena) {
  arena.using(myLibReceiveFunction(10), malloc.free);
});

this is my code

// ignore_for_file: empty_catches, unused_local_variable

import 'dart:ffi';
import 'package:ffi/ffi.dart';
import 'dart:convert' as convert;

typedef TdPointerNative = Pointer;
typedef TdPointerFunctionNative = TdPointerNative Function();

typedef TdStringNative = Pointer<Utf8>;
typedef TdReceiveNative = TdStringNative Function(Double timout);
typedef TdReceiveDart = TdStringNative Function(double timout);

typedef TdSendNative = Void Function(TdPointerNative client, TdStringNative request);
typedef TdSendDart = void Function(TdPointerNative client, TdStringNative request);

typedef TdExecuteNative = TdStringNative Function(TdStringNative parameters);

typedef TdDestroyNative = Void Function(Pointer clientId);
typedef TdDestroyDart = void Function(Pointer clientId);

int tdCreateClientId({
  required DynamicLibrary tdLib,
}) {
  int clientIdNew = using((Arena arena) {
    // https://core.telegram.org/tdlib/docs/td__json__client_8h.html#a7feda953a66eee36bc207eb71a55c490
    TdPointerFunctionNative tdPointerNativeFunction = tdLib.lookupFunction<TdPointerFunctionNative, TdPointerFunctionNative>('td_create_client_id');
    Pointer tdPointerNativeResult = arena.using(tdPointerNativeFunction(), freeMemory);
    int clientIdNew = tdPointerNativeResult.address;

    return clientIdNew;
  });
  return clientIdNew;
}

/// td_send
void tdSend({
  required int clientId,
  Map? parameters,
  required DynamicLibrary tdLib,
}) {
  using((Arena arena) {
    Pointer clientIdAddresData = Pointer.fromAddress(clientId);
    TdStringNative requestData = convert.json.encode(parameters).toNativeUtf8();
    Arena arena = Arena();
    TdSendDart tdSendFunction = tdLib.lookupFunction<TdSendNative, TdSendDart>('td_send');
    void tdSendResult = arena.using(tdSendFunction(clientIdAddresData, requestData), (p0) {});

    malloc.free(requestData);
  });
  return;
}

Map<String, dynamic>? tdReceiveStatic({
  required DynamicLibrary tdLib,
  double timeout = 1.0,
  bool isAndroid = false,
}) {
  try {
    Map<String, dynamic>? result = using((Arena arena) {
      /// Docs: https://core.telegram.org/tdlib/docs/td__json__client_8h.html#a62715bea8e41a554d1bac763c187b662
      TdReceiveDart tdReceiveFunction = tdLib.lookupFunction<TdReceiveNative, TdReceiveDart>(
        '${isAndroid ? "_" : ""}td_receive',
      );
      TdStringNative update = arena.using(tdReceiveFunction(timeout), freeMemory);
      if (update.address != 0) {
        String updateString = update.toDartString();

        if (updateString.isEmpty) {
          return null;
        }
        Map<String, dynamic>? updateOrigin;
        try {
          updateOrigin = convert.json.decode(update.toDartString());
        } catch (e) {}
        if (updateOrigin != null) {
          return updateOrigin;
        }
      } else {}
      return null;
    });
    return result;
  } catch (e) {}
  return null;
}

void freeMemory(Pointer<NativeType> pointer) {
  malloc.free(pointer);
}

void main(List<String> args) async {
  // open library tdlib
  // Tdlib docs: https://core.telegram.org/tdlib/docs/td__json__client_8h.html
  // Tdlib is Telegram Database Library for interact with telegram api so you can make
  // Program Application, Bot, Userbot Custom
  DynamicLibrary tdLib = DynamicLibrary.open("libtdjson.so");
  int clientId = tdCreateClientId(tdLib: tdLib);
  tdSend(clientId: clientId, tdLib: tdLib, parameters: {
    "@type": "getOption",
    "name": "version",
  });
  while (true) {
    await Future.delayed(Duration(microseconds: 1));

    Map? update = tdReceiveStatic(tdLib: tdLib);
    if (update != null) {
      print(update);
    }
  }
}

current error code

===== CRASH =====
si_signo=Segmentation fault(11), si_code=SEGV_MAPERR(1), si_addr=0xfffffffffffffff9
version=3.3.3 (stable) (Tue Mar 26 14:21:33 2024 +0000) on "linux_x64"
pid=564959, thread=564986, isolate_group=main(0x5bdb6457f540), isolate=main(0x5bdb645858d0)
os=linux, arch=x64, comp=no, sim=no
isolate_instructions=5bdb6238d580, vm_instructions=5bdb6238d580
fp=7d1cf87fe0b8, sp=7d1cf87fe070, pc=7d1d0b2a881e
  pc 0x00007d1d0b2a881e fp 0x00007d1cf87fe0b8 free+0x1e
  pc 0x00007d1d09ba5dd7 fp 0x00007d1cf87fe0f0 Unknown symbol
  pc 0x00007d1d09ba5433 fp 0x00007d1cf87fe138 Unknown symbol
  pc 0x00007d1d09ba5288 fp 0x00007d1cf87fe170 Unknown symbol
  pc 0x00007d1d09ba51e9 fp 0x00007d1cf87fe1a0 Unknown symbol
  pc 0x00007d1d09ba50e3 fp 0x00007d1cf87fe1f8 Unknown symbol
  pc 0x00007d1d09ba4f80 fp 0x00007d1cf87fe250 Unknown symbol
  pc 0x00007d1d09ba4e8a fp 0x00007d1cf87fe2a0 Unknown symbol
  pc 0x00007d1d09ba4275 fp 0x00007d1cf87fe310 Unknown symbol
  pc 0x00007d1d09ba2cd8 fp 0x00007d1cf87fe3a0 Unknown symbol
  pc 0x00007d1d09ba28af fp 0x00007d1cf87fe400 Unknown symbol
  pc 0x00007d1d09ba20a1 fp 0x00007d1cf87fe458 Unknown symbol
  pc 0x00007d1d09ba1f99 fp 0x00007d1cf87fe488 Unknown symbol
  pc 0x00007d1d09ba1106 fp 0x00007d1cf87fe4f0 Unknown symbol
  pc 0x00007d1d09ba138c fp 0x00007d1cf87fe538 Unknown symbol
  pc 0x00007d1d09ba1106 fp 0x00007d1cf87fe5a0 Unknown symbol
  pc 0x00007d1d09ba01bc fp 0x00007d1cf87fe5f8 Unknown symbol
  pc 0x00007d1d0ac02e46 fp 0x00007d1cf87fe670 Unknown symbol
  pc 0x00005bdb624cb0e2 fp 0x00007d1cf87fe6d0 dart::DartEntry::InvokeFunction+0x162
  pc 0x00005bdb624ccad3 fp 0x00007d1cf87fe710 dart::DartLibraryCalls::HandleMessage+0x123
  pc 0x00005bdb624e944f fp 0x00007d1cf87feca0 dart::IsolateMessageHandler::HandleMessage+0x2bf
  pc 0x00005bdb6250b7d6 fp 0x00007d1cf87fed10 dart::MessageHandler::HandleMessages+0x116
  pc 0x00005bdb6250bdc8 fp 0x00007d1cf87fed60 dart::MessageHandler::TaskCallback+0x1e8
  pc 0x00005bdb626098e7 fp 0x00007d1cf87fede0 dart::ThreadPool::WorkerLoop+0x137
  pc 0x00005bdb62609b72 fp 0x00007d1cf87fee10 dart::ThreadPool::Worker::Main+0x72
  pc 0x00005bdb62593096 fp 0x00007d1cf87feed0 dart::ThreadStart+0xd6
-- End of DumpStackTrace
  pc 0x0000000000000000 fp 0x00007d1cf87fe0b8 sp 0x0000000000000000 Cannot find code object
  pc 0x00007d1d09ba5dd7 fp 0x00007d1cf87fe0f0 sp 0x00007d1cf87fe0c8 [Optimized] init:posixFree.#ffiClosure2
  pc 0x00007d1d09ba5433 fp 0x00007d1cf87fe138 sp 0x00007d1cf87fe100 [Unoptimized] MallocAllocator.free
  pc 0x00007d1d09ba5288 fp 0x00007d1cf87fe170 sp 0x00007d1cf87fe148 [Unoptimized] freeMemory
  pc 0x00007d1d09ba51e9 fp 0x00007d1cf87fe1a0 sp 0x00007d1cf87fe180 [Unoptimized] freeMemory
  pc 0x00007d1d09ba50e3 fp 0x00007d1cf87fe1f8 sp 0x00007d1cf87fe1b0 [Unoptimized] _RootZone@4048458.runUnary
  pc 0x00007d1d09ba4f80 fp 0x00007d1cf87fe250 sp 0x00007d1cf87fe208 [Unoptimized] _RootZone@4048458.bindUnaryCallback.<anonymous closure>
  pc 0x00007d1d09ba4e8a fp 0x00007d1cf87fe2a0 sp 0x00007d1cf87fe260 [Unoptimized] Arena.using.<anonymous closure>
  pc 0x00007d1d09ba4275 fp 0x00007d1cf87fe310 sp 0x00007d1cf87fe2b0 [Unoptimized] Arena.releaseAll
  pc 0x00007d1d09ba2cd8 fp 0x00007d1cf87fe3a0 sp 0x00007d1cf87fe320 [Unoptimized] using
  pc 0x00007d1d09ba28af fp 0x00007d1cf87fe400 sp 0x00007d1cf87fe3b0 [Unoptimized] tdCreateClientId
  pc 0x00007d1d09ba20a1 fp 0x00007d1cf87fe458 sp 0x00007d1cf87fe410 [Unoptimized] main
  pc 0x00007d1d09ba1f99 fp 0x00007d1cf87fe488 sp 0x00007d1cf87fe468 [Unoptimized] main
  pc 0x00007d1d09ba1106 fp 0x00007d1cf87fe4f0 sp 0x00007d1cf87fe498 [Unoptimized] _Closure@0150898.dyn:call
  pc 0x00007d1d09ba138c fp 0x00007d1cf87fe538 sp 0x00007d1cf87fe500 [Unoptimized] _delayEntrypointInvocation@1026248.<anonymous closure>
  pc 0x00007d1d09ba1106 fp 0x00007d1cf87fe5a0 sp 0x00007d1cf87fe548 [Unoptimized] _Closure@0150898.dyn:call
  pc 0x00007d1d09ba01bc fp 0x00007d1cf87fe5f8 sp 0x00007d1cf87fe5b0 [Unoptimized] _RawReceivePort@1026248._handleMessage@1026248
  pc 0x00007d1d0ac02e46 fp 0x00007d1cf87fe670 sp 0x00007d1cf87fe608 [Stub] InvokeDartCode
[1]    564959 IOT instruction (core dumped)  dart run 

[HosseinYousefi]

So I started reading your code and the first problem I found is the fact that based on the URL in the comment: td_create_client_id returns an integer and not a pointer. Trying to free some random integer will cause a segmentation fault.

Your original question of how to release memory using Arena is answered. For further questions please use Q&A sites like StackOverFlow. Closing this.