LibTorrent bindings

[Davenchy]

I am trying to bind libtorrent using ffigen

my ffigen.yaml:

# Run with `dart run ffigen --config ffigen.yaml`.
name: DartLibTorrentBindings
description: |
  Bindings for `LibTorrent`.

  Regenerate bindings with `dart run ffigen --config ffigen.yaml`.
output: "lib/bindings_generated.dart"
headers:
  entry-points:
    - "src2/include/libtorrent/libtorrent.hpp"
preamble: |
  // ignore_for_file: always_specify_types
  // ignore_for_file: camel_case_types
  // ignore_for_file: non_constant_identifier_names
comments:
  style: any
  length: full

the output of dart run ffigen --config ffigen.yaml is:

Input Headers: [src2/include/libtorrent/libtorrent.hpp]
[SEVERE] : Header src2/include/libtorrent/libtorrent.hpp: Total errors/warnings: 1.
[SEVERE] :     src2/include/libtorrent/libtorrent.hpp:4:10: fatal error: 'libtorrent/add_torrent_params.hpp' file not found [Lexical or Preprocessor Issue]
Finished, Bindings generated in /.../flutter/projects/dart_libtorrent/lib/bindings_generated.dart

the libtorrent.hpp file is something like this

// This header is generated by tools/gen_convenience_header.py

#include "libtorrent/add_torrent_params.hpp"
#include "libtorrent/address.hpp"
#include "libtorrent/alert.hpp"
#include "libtorrent/alert_types.hpp"
...

I was following this tutorial:

https://blog.logrocket.com/dart-ffi-native-libraries-flutter/

[liamappelbe]

clang can't find those headers because it needs an extra include directory. In ffigen.yaml, use the compiler-opts option to pass clang a -I flag:

compiler-opts:
  - '-Isrc2/include/'

[kirill-21]

I am trying to bind libtorrent using ffigen

my ffigen.yaml:

# Run with `dart run ffigen --config ffigen.yaml`.
name: DartLibTorrentBindings
description: |
  Bindings for `LibTorrent`.

  Regenerate bindings with `dart run ffigen --config ffigen.yaml`.
output: "lib/bindings_generated.dart"
headers:
  entry-points:
    - "src2/include/libtorrent/libtorrent.hpp"
preamble: |
  // ignore_for_file: always_specify_types
  // ignore_for_file: camel_case_types
  // ignore_for_file: non_constant_identifier_names
comments:
  style: any
  length: full

the output of dart run ffigen --config ffigen.yaml is:

Input Headers: [src2/include/libtorrent/libtorrent.hpp]
[SEVERE] : Header src2/include/libtorrent/libtorrent.hpp: Total errors/warnings: 1.
[SEVERE] :     src2/include/libtorrent/libtorrent.hpp:4:10: fatal error: 'libtorrent/add_torrent_params.hpp' file not found [Lexical or Preprocessor Issue]
Finished, Bindings generated in /.../flutter/projects/dart_libtorrent/lib/bindings_generated.dart

the libtorrent.hpp file is something like this

// This header is generated by tools/gen_convenience_header.py

#include "libtorrent/add_torrent_params.hpp"
#include "libtorrent/address.hpp"
#include "libtorrent/alert.hpp"
#include "libtorrent/alert_types.hpp"
...

I was following this tutorial:

https://blog.logrocket.com/dart-ffi-native-libraries-flutter/

Did you solve this, Davenchy? If yes - can you share your configuration for ffigen for libtorrent?

[Davenchy]

Hey @kirill-21 the answer helped to generate code but I still can't use the generated code because it doesn't contain the actual classes and methods found in libtorrent docs

also the generated code has errors for undeclared types

This is my ffigen.yaml:

# Run with `dart run ffigen --config ffigen.yaml`.
name: DartLibTorrentBindings
description: |
  Bindings for `LibTorrent`.

  Regenerate bindings with `dart run ffigen --config ffigen.yaml`.
output: "lib/bindings_generated.dart"
headers:
  entry-points:
    - "src2/include/libtorrent/libtorrent.hpp"
compiler-opts:
  - '-Isrc2/include'
preamble: |
  // ignore_for_file: always_specify_types
  // ignore_for_file: camel_case_types
  // ignore_for_file: non_constant_identifier_names
comments:
  style: any
  length: full

[dcharkes]

@Davenchy I'm assuming you used https://github.com/arvidn/libtorrent/blob/RC_2_0/include/libtorrent/libtorrent.hpp ?

As far as I can see this is C++ not C. dart:ffi and package:ffigen do not support C++ .

Unfortunately, C++ does not have a stable ABI. This means it is undefined what the register/stack layout is when one calls functions in a .dll, .so, .dylib. It might work if you happen to have compiled the caller with the same compiler, but it is not guaranteed to work like it is for C. We cannot know in Dart with which compiler you compiled your C++ code. C has defined the Application Binary Interface which every compiler adheres to, so that we can know how to call it.

For more info see https://github.com/dart-lang/sdk/issues/38788.

_Originally posted by @dcharkes in https://github.com/dart-lang/native/issues/244

I see a C header in https://github.com/arvidn/libtorrent/blob/RC_2_0/bindings/c/libtorrent.h. But it looks like that is a rather small subset of the API. (Also, dart:ffi and this package do not yet support variadic functions https://github.com/dart-lang/native/issues/398.)

[Kylart]

Hey @Davenchy,

Did you end up with something working in terms of libtorrent bindings? I'm also working on libtorrent bindings for Dart / Flutter and would appreciate any lead you have 🙌

[Davenchy]

Hey @Kylart, Unfortunately I don't have that much experience to deal with libtorrent and writing bindings to Dart.

[tanishbajaj101]

Hey @Davenchy,

Did you end up with something working in terms of libtorrent bindings? I'm also working on libtorrent bindings for Dart / Flutter and would appreciate any lead you have 🙌

did you find any alternative? @Kylart

[Kylart]

@tanishbajaj101

There is not alternative to this day using pure dart solutions so you'd have to build your own.

If I had to do it now, I would start by building pure C bindings for libtorrent that I would then use from Dart just like @dcharkes mentioned.

Depending on your needs though, I would say that you can take a look at the dtorrent_task package.

[tanishbajaj101]

@tanishbajaj101

There is not alternative to this day using pure dart solutions so you'd have to build your own.

If I had to do it now, I would start by building pure C bindings for libtorrent that I would then use from Dart just like @dcharkes mentioned.

Depending on your needs though, I would say that you can take a look at the dtorrent_task package.

yes, but how did you do it? not to snoop but out of curiosity i checked your profile that you had build an anime streaming app using torrents, also i did check out dtorrent, it doesn't have concurrency and hence was giving very slow results

[tanishbajaj101]

@tanishbajaj101

There is not alternative to this day using pure dart solutions so you'd have to build your own.

If I had to do it now, I would start by building pure C bindings for libtorrent that I would then use from Dart just like @dcharkes mentioned.

Depending on your needs though, I would say that you can take a look at the dtorrent_task package.

mind i ask how I could make pure C bindings from libtorrent as it is written majorly in C++

[Davenchy]

@tanishbajaj101 This answer on stack overflow could be helpful, https://stackoverflow.com/a/2045860/8046862

[tanishbajaj101]

@tanishbajaj101 This answer on stack overflow could be helpful, https://stackoverflow.com/a/2045860/8046862

would i need to do this only for the header files/functions that I use, or basically on every file (that those header files may further depend on) thanks for the link, I wasn't aware this was possible

[Davenchy]

@tanishbajaj101 This answer on stack overflow could be helpful, https://stackoverflow.com/a/2045860/8046862

would i need to do this only for the header files/functions that I use, or basically on every file (that those header files may further depend on) thanks for the link, I wasn't aware this was possible

Here is a ChatGPT answer:

To use C++ classes and functions in Dart, you can create C functions (wrapper functions) for the specific classes and functions you want to use. These C functions act as an interface between your Dart code and the C++ code.

Here's a basic example of how you can do it:

1. Define your C++ class and functions as usual (In your case this already done by libtorrent):

// my_class.hpp
#pragma once

class MyClass {
public:
    MyClass();
    int add(int a, int b);
};

// my_class.cpp
#include "my_class.hpp"

MyClass::MyClass() {}

int MyClass::add(int a, int b) {
    return a + b;
}

2. Create a C header file (my_class_c_wrapper.h) with C functions that wrap your C++ class and functions:

// my_class_c_wrapper.h
#ifdef __cplusplus
extern "C" {
#endif

typedef void* MyClassHandle;

MyClassHandle createMyClass();
int add(MyClassHandle handle, int a, int b);
void destroyMyClass(MyClassHandle handle);

#ifdef __cplusplus
}
#endif

3. Implement these C functions in a C++ source file (my_class_c_wrapper.cpp) using the C++ class and functions:

// my_class_c_wrapper.cpp
#include "my_class_c_wrapper.h"
#include "my_class.hpp"

extern "C" {

MyClassHandle createMyClass() {
    return reinterpret_cast<MyClassHandle>(new MyClass());
}

int add(MyClassHandle handle, int a, int b) {
    MyClass* instance = reinterpret_cast<MyClass*>(handle);
    return instance->add(a, b);
}

void destroyMyClass(MyClassHandle handle) {
    MyClass* instance = reinterpret_cast<MyClass*>(handle);
    delete instance;
}

}

4. Now you can compile your C++ code into a shared library (DLL on Windows, SO on Unix-like systems) and use it in your Dart code using FFI (Foreign Function Interface).

With this setup, you can use the C functions (createMyClass, add, destroyMyClass) in Dart through FFI to interact with your C++ code.

[Davenchy]

My friends @tanishbajaj101, @kirill-21, @Kylart , I've created this example project. I hope it proves useful to you. https://github.com/Davenchy/libtorrent-dart-bindings-example

[tanishbajaj101]

My friends @tanishbajaj101, @kirill-21, @Kylart , I've created this example project. I hope it proves useful to you. https://github.com/Davenchy/libtorrent-dart-bindings-example

thanks a lot for the example project! I just wanted to request you to share the intermediate C file or header file which you might have created to generate the wrapper.cpp file for a better understanding. Also in the stackoverflow answers they repeatedly make use of to decode data from class that was in .C file. How is your wrapper avoiding that? Also, the functions that wrapper.cpp is using from wrapper.hpp seem to be redefined in that file which defeats the purpose of bindings

[Davenchy]

My friends @tanishbajaj101, @kirill-21, @Kylart , I've created this example project. I hope it proves useful to you. https://github.com/Davenchy/libtorrent-dart-bindings-example

thanks a lot for the example project! I just wanted to request you to share the intermediate C file or header file which you might have created to generate the wrapper.cpp file for a better understanding. Also in the stackoverflow answers they repeatedly make use of to decode data from class that was in .C file. How is your wrapper avoiding that? Also, the functions that wrapper.cpp is using from wrapper.hpp seem to be redefined in that file which defeats the purpose of bindings

Hello there,

I didn't generate the wrapper.cpp/.hpp files; instead, I crafted them from scratch to suit the project's specific requirements.

Essentially, what I've done is create the storage_get(...) function. This function accepts a torrent magnet URI and returns a storage_t object containing all the files within the torrent file, along with their names, paths, and sizes.

The storage_get(...) function utilizes the download_torrent_info(...) function, which in turn employs the libtorrent library to download the torrent file from peers. For more details about libtorrent, please refer to their documentation, which I've conveniently linked in the README.md file.

Within the header file wrapper.hpp, I've included the storage_get and storage_free functions within an extern "C" scope. This informs the compiler to use the actual function names. For further information on C++ name mangling, you can check out this link: https://www.geeksforgeeks.org/extern-c-in-c/.

By utilizing the provided Makefile to build the wrapper.cpp/.hpp files using the command make build, you'll obtain a library file named libwrapper.so. You can adjust the Makefile to generate .dll libraries for Windows and .dylib for MacOS, although my focus has been primarily on Linux as it's my operating system.

Now, you can utilize the compiled library with Dart's FFI to invoke the storage_get function from Dart. I hope this clarifies everything.

I acknowledge that my C++ skills may not be top-notch, and my code may not adhere to the best practices. However, my intention was to provide a functional example. I'm still learning, and this might be my first real project, where I delved into documentation and utilized a library like libtorrent.

Regarding the casting, I've employed C-style casting to cast const storage_t* into void* in several instances. While this isn't considered good practice, I prioritized creating a functional example over focusing on the actual C++ code.

For further insights into casting, you can explore the following links:

• https://anteru.net/blog/2007/c-background-static-reinterpret-and-c-style-casts/#:~:text=The%20rule%20of%20the%20thumb,have%20to%20reinterpret%20the%20data.
• https://stackoverflow.com/questions/7831696/reinterpret-cast-vs-c-style-cast
• https://softwareengineering.stackexchange.com/questions/50442/c-style-casts-or-c-style-casts

Feel free to reach out if you have any further questions or need clarification on anything.

Yep, you got it! I'm relying on ChatGPT to jazz up my reply because, let's face it, my English is more of a mess than my C++ code! 😅😇