Possible to create a stream

[sakib1361]

Creating a stream out of the manager with a required file like monotorrent would allow streaming the file to media players directly. Seems a bit difficult to implement though. I have linked the possible documentation.

https://www.rasterbar.com/products/libtorrent/streaming.html

[aspriddell]

The library was originally designed to download files in their entirety then handle post-processing when the download is complete. I can see adding the sequential_download mode could be helpful, though as for providing a direct stream to the file(s) downloaded, you can just do using var fileStream = new FileStream(manager.Files[0].Path, FileAccess.Read, FileShare.ReadWrite); to open the file.

That said, if you wanted to add a native method to get detailed download stats for each file in a TorrentManager, the native files need:

• A struct to contain the individual file info (pieces, percentage, etc.)
• A struct containing information about the list of above structs (i.e. count, array pointer)
• A method that takes the active libtorrent manager and produces the list
• A method that takes the list and destroys it (along with destroying any allocated stuff in the nested structs)

Then map the methods/structs in the C#-side for use.

---

If you're interested in getting this in, I'm open to contributions though as it's not something I would use in my application that uses this library. I don't plan on adding this in the immediate future unless there is significant demand or I have spare time.

[sakib1361]

Thanks a lot. I think you already have FileInfo. I am a bit weak with cmake. I have done it in c++/cli which allows direct import in windows only. But may be better to have it as a cross platform feature like yours do nicely. I will be trying to add the following features which I think enough to create a custom stream.

bool has_piece(handler, pieceIndex) void focus_pieces(handler, start, end) void reset_focus(handler)

The logic is, we would start with current required file with high availability. So the piece picker logic would try to buffer next 20mb first. When the file is fully downloaded, a reset_focus can be called to allow downloading other parts. (Honors do no download). Obviously, this logic needs to be in the app, not in the csdl library. As different player, webapp might request focus_piece differently.

Thanks, I am checking the cmake build process now.

[aspriddell]

I've had some time to think about this and there's a couple things that would be ideal:

• Adding the ability to get download metrics on a specific file (piece count, bytes uploaded/downloaded)
• Setting the focus (as you mentioned)
• Being able to toggle sequential_download for a file (if it's possible to toggle, otherwise just when it's created)
• The ability to load the piece map for a file or the whole torrent

I've written this patch that may or may not be useful, basically adding the piece count to the torrent info, adding a struct to contain download stats for specific files and stubs for setting focus and getting the piece map (as you mentioned previously):

Index: native/src/library.cpp
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/native/src/library.cpp b/native/src/library.cpp
--- a/native/src/library.cpp    (revision afd7345d47318c57ad63374b20d42bf532b0c402)
+++ b/native/src/library.cpp    (date 1730022000821)
@@ -238,6 +238,7 @@
             index,
             files.file_offset(i),
             files.file_size(i),
+            files.file_num_pieces(i),
             files.mtime(i),
             file_name,
             file_path,
Index: native/include/library.h
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/native/include/library.h b/native/include/library.h
--- a/native/include/library.h  (revision afd7345d47318c57ad63374b20d42bf532b0c402)
+++ b/native/include/library.h  (date 1730023080806)
@@ -45,12 +45,18 @@
     CSDL_EXPORT uint8_t get_file_dl_priority(lt::torrent_handle* torrent, int32_t file_index);
     CSDL_EXPORT void set_file_dl_priority(lt::torrent_handle* torrent, int32_t file_index, uint8_t priority);

+    CSDL_EXPORT void set_torrent_dl_focus(lt::torrent_handle* torrent, int64_t offset, int64_t length);
+    CSDL_EXPORT void clear_torrent_dl_focus(lt::torrent_handle* torrent);
+
     // download control
     CSDL_EXPORT void start_torrent(lt::torrent_handle* torrent);
     CSDL_EXPORT void stop_torrent(lt::torrent_handle* torrent);
     CSDL_EXPORT void reannounce_torrent(lt::torrent_handle* torrent, const int32_t seconds, const uint8_t ignore_min_interval);

     CSDL_EXPORT void get_torrent_status(lt::torrent_handle* torrent, torrent_status* torrent_status);
+    CSDL_EXPORT void get_torrent_file_status(lt::torrent_handle* torrent, int32_t file_index, torrent_file_status* file_status);
+
+    CSDL_EXPORT void get_torrent_piece_map(lt::torrent_handle* torrent, int64_t piece_start, int64_t piece_end, uint8_t* map);

 #ifdef __cplusplus
 }
Index: native/include/structs.h
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/native/include/structs.h b/native/include/structs.h
--- a/native/include/structs.h  (revision afd7345d47318c57ad63374b20d42bf532b0c402)
+++ b/native/include/structs.h  (date 1730022535206)
@@ -19,6 +19,7 @@

     int64_t offset;
     int64_t file_size;
+    int64_t total_pieces;

     time_t modified_time;

@@ -74,6 +75,19 @@
     int64_t download_rate;
 } torrent_status;

+CSDL_STRUCT typedef struct cs_torrent_file_status {
+    cs_torrent_state state;
+
+    int32_t file_index;
+    int64_t offset;
+
+    int64_t pieces_total;
+    int64_t pieces_downloaded;
+
+    int64_t bytes_total;
+    int64_t bytes_downloaded;
+} torrent_file_status;
+
 #ifdef __cplusplus
 }
 #endif

These would provide enough information to work out what subsection of the torrent map you'd need to fetch to get the entries for a file, and can be used to get the state on a single piece.

As for CMake... the hardest part is getting it to work with vcpkg - I suggest getting it to work on your device then when you want to do the cross-platform stuff, use github actions to build you all the libraries in one go.

[sakib1361]

Wow, that looks interesting to see and I see if the headers are implemented, that would make a custom filestream possible.

Yes, you are absolutely right that I got stuck with cmake and vcpkg. But managed to build. I am checking things out.

bool has_piece(lt::torrent_handle* torrent, int piece_index)
{
    lt::piece_index_t piece(piece_index);
    return torrent->have_piece(piece);
}

void set_torrent_focus(lt::torrent_handle* torrent, int start, int end)
{
    torrent->clear_piece_deadlines();
    int index = 0;
    for (auto i : torrent->torrent_file().get()->piece_range())
    {
        if (index >= start && index <= end)
        {
            torrent->piece_priority(i, lt::top_priority);
            torrent->set_piece_deadline(i, 5000 * (1 + index - start));
        }
        else
        {
            torrent->piece_priority(i, lt::dont_download);
        }
        index++;
    }
}

void clear_torrent_focus(lt::torrent_handle* torrent)
{
    torrent->clear_piece_deadlines();
    for (auto i : torrent->torrent_file().get()->piece_range())
    {
        torrent->piece_priority(i, lt::default_priority);
    }
}

These are the things I had in mind. File status is also nice, but not as crucial as the has piece. Not sure about the piece_map though. Which function it would use internally?

Thanks again a lot for taking interest in this.

[sakib1361]

BTW this is more or less a working example of a custom stream. Underlying contains a filestream and a list of chunk(piece) already available. _start (file start index piece) can be calculated easily via filesizes, indexex and filelength. then it becomes simple as below.

public override async Task<int> ReadAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
{
    var startOffset = _start + offset + Position;
    int piece = (int)(startOffset / _piece);

    if (_chunks.Contains(piece))
    {
        //No need to do anything now. gets called very often. 
    }
    else if (_handler.HasPiece(piece))
    {
        _chunks.Add(piece);
    }
    else
    {
        int pieceEnd = (int)((startOffset + offset + Position + 10*1024*1024)/_piece);
        pieceEnd = Math.Max(piece + 1, pieceEnd);
        _handler.FocusPieces(piece, pieceEnd);
        while (_disposed == false && _handler.HasPiece(piece) == false)
        {
            await Task.Delay(2000, cancellationToken).ConfigureAwait(false);
        }
        _chunks.Add(piece);
        if(_disposed) return 0;
    }
    return await _fileStream.ReadAsync(buffer, offset, count, cancellationToken);
}

How much offset is required is really dependent on the player it needs. More accurately this should be dynamic. Thanks again.

[aspriddell]

File status is also nice, but not as crucial as the has piece.

That's true, but is for meeting expectations - if there's a function to get a stream to a file, I would expect there to be a way to find out how much of the file has been downloaded

Not sure about the piece_map though. Which function it would use internally?

It replaces the has_piece function but is expandable so you can use it to get the state of a single piece, but someone else can use it to build the map for an entire file. _{Afaik you can also get more detailed states on each piece like Completed, Verifying, Downloading, Skipped, etc., though I could be wrong.}

---

As for your stream example, you might need to look at events emitted by the torrent engine (might need a custom event adding too) when you end up waiting for pieces to be available and use an AutoResetEvent to release the waiting read asynchronously.

https://github.com/aspriddell/csdl/blob/afd7345d47318c57ad63374b20d42bf532b0c402/csdl/TorrentClient.cs#L115

I'm wondering if it's worth getting the base features (piece map/setting focus/file status) in without worrying about the stream implementation, as you can always integrate that directly into your own project then fine-tune it to your needs.

[sakib1361]

I'm wondering if it's worth getting the base features (piece map/setting focus/file status) in without worrying about the stream implementation, as you can always integrate that directly into your own project then fine-tune it to your needs.

I would agree that piece handling should be good enough to create a stream implementation. Stream creation shouldn't be the concern of this library as libtorrent itself also does not provide direct stream creation. It really needs fine tuning for the end developer. So I would say it's good enough to have the functions you mentioned.