frostwire / frostwire-jlibtorrent

A swig Java interface for libtorrent by the makers of FrostWire. Develop libtorrent based apps with the joy of coding in Java.
http://www.frostwire.com
MIT License
444 stars 137 forks source link

SessionManager: Error fetching magnet, java.lang.InterruptedException #238

Closed FunkyMuse closed 2 years ago

FunkyMuse commented 3 years ago

I/SessionManager: Error fetching magnet java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1065) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1358) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:278) at com.frostwire.jlibtorrent.SessionManager.fetchMagnet(SessionManager.java:680) at com.frostwire.jlibtorrent.SessionManager.fetchMagnet(SessionManager.java:704) at com.frostwire.jlibtorrent.SessionManager.fetchMagnet(SessionManager.java:716)

It doesn't work on Android 11 You can reproduce it using this repo by running it and clicking TEST button Jlib version used: 1.2.8.0

gubatron commented 3 years ago

awesome, we're now testing Android 11, so far we had no issues on our FrostWire client fetching magnets, will take a look, I wonder how you're using this, could you please link to the place where you instantiate that SessionManager, the "InterruptedException" might be due to Android destroying the context of wherever that's running, are you using a background service properly?

FunkyMuse commented 3 years ago

awesome, we're now testing Android 11, so far we had no issues on our FrostWire client fetching magnets, will take a look, I wonder how you're using this, could you please link to the place where you instantiate that SessionManager, the "InterruptedException" might be due to Android destroying the context of wherever that's running, are you using a background service properly?

Link

FunkyMuse commented 3 years ago

awesome, we're now testing Android 11, so far we had no issues on our FrostWire client fetching magnets, will take a look, I wonder how you're using this, could you please link to the place where you instantiate that SessionManager, the "InterruptedException" might be due to Android destroying the context of wherever that's running, are you using a background service properly?

This issue happens when I try to stop it. Otherwise downloading won't even start.

FunkyMuse commented 3 years ago
val sessionManager = SessionManager(true)
val torrentInfoByteArray = sessionManager.fetchMagnet(link1, 10, true)
//val torrentInfoByteArray = sessionManager.fetchMagnet(link1, 10)

this always returns null for the torrentInfoByteArray

FunkyMuse commented 3 years ago
java.lang.NullPointerException: Attempt to get length of null array
        at com.frostwire.jlibtorrent.Vectors.bytes2byte_vector(Vectors.java:40)
        at com.frostwire.jlibtorrent.TorrentInfo.bdecode0(TorrentInfo.java:679)
        at com.frostwire.jlibtorrent.TorrentInfo.bdecode(TorrentInfo.java:654)
gubatron commented 3 years ago

This issue happens when I try to stop it. that would explain the InterruptedException :)

val torrentInfoByteArray = sessionManager.fetchMagnet(link1, 10, true) 10 seconds is too litle for most magnets, raise that timeout to 30 secs.

That NPE seems to be because you had a null/invalid TorrentInfo

I'm very close to releasing jlibtorrent 1.2.10, hopefully in the next hours today.

However, it seems to me that all of these errors are due to a bad implementation.

I suggest you read FrostWire's source code for examples

These are two good starting points. https://github.com/frostwire/frostwire/blob/master/common/src/main/java/com/frostwire/bittorrent/BTEngine.java https://github.com/frostwire/frostwire/blob/86f8e16dfda1cbc3955e1aa1294cec5a0104d4a8/android/src/com/frostwire/android/gui/transfers/TorrentFetcherDownload.java

FunkyMuse commented 3 years ago

The repo is a fork of simple-torrent-android and the stream one found in the readme of this repo and i just make updates from this library's releases.

It's weird that it works on Android 10 but it doesn't on Android 11.

I'll look into the implementation and hopefully sort it out, thank you for your effort and time.

FunkyMuse commented 3 years ago

If you change the compiled version and targeted version to frostwire's Android repo (just did a pull and tested) to 30 but it works if it's 29 and installed on 30, there magnets don't work on 30. @gubatron

The status goes to Downloading torrent and after a while it says Error

Tested on Pixel XL 2 with Android 11

FunkyMuse commented 3 years ago

Doesn't work with 1.2.10.0 update either

gubatron commented 3 years ago

when you talk about target SDK, are you talking about your own app's target SDK, or are you talking about your compilation of jlibtorrent's sources under Android SDK 29 vs Android SDK 30?

gubatron commented 3 years ago

I wonder if this has something to do with further restrictions on Storage after SDK 30. So much bullshit coming out of Android, I miss the days where you could simply use a File and a stream...

https://developer.android.com/training/data-storage/use-cases#modify-delete-media

FunkyMuse commented 3 years ago

I wonder if this has something to do with further restrictions on Storage after SDK 30. So much bullshit coming out of Android, I miss the days where you could simply use a File and a stream...

https://developer.android.com/training/data-storage/use-cases#modify-delete-media

The READ_EXTERNAL_STORAGE was deprecated.

So the only way this works on Android 11 is if you compile it with SDK 29

If you compile it with SDK 30 it won't work on Android 11

FunkyMuse commented 3 years ago

when you talk about target SDK, are you talking about your own app's target SDK, or are you talking about your compilation of jlibtorrent's sources under Android SDK 29 vs Android SDK 30?

I'm talking about compiling jlib torrent under SDK 30, if you compile it with SDK 30 it doesn't work on Android 11.

I also compiled frostwire for Android with SDK 30 and it didn't work on Android 11, same error, the fetchmagnet didn't work.

gubatron commented 3 years ago

thanks for letting me know, I'll test, hopefully I'll be able to replicate and find a solution to the problem.

Google seems to be making access to external storage impossible.

FunkyMuse commented 3 years ago

thanks for letting me know, I'll test, hopefully I'll be able to replicate and find a solution to the problem.

Google seems to be making access to external storage impossible.

Indeed they are, if it's of any help look up SAF for writing everywhere else than own's app storage, the File access is only to app's own storage(context.getFilesDir) and cache (context.getCacheDir)

gubatron commented 3 years ago

Yes, we've been using SAF since they introduced it (on FrostWire + JLibTorrent) I read that Android 11 has a new Manage External Storage mechanism, I have to read more about it, perhaps it will allow you to write without SAF interfaces, and just get back to using File wherever the app has permission to write to.

gubatron commented 3 years ago

More on this (for whoever else reads this months later looking for answers):

https://www.androidpolice.com/2020/02/19/scoped-storage-in-android-11-will-have-exemptions-for-older-apis-and-core-apps-like-file-managers/

This gives me hope:

First, developers will be able to keep using the legacy storage access system as long 
as they target Android 10 (API level 29). That will only be a viable option until Google 
requires Android 11 support in 2021. In addition, Google will add a new permission for
 apps like file managers and backup apps with "core use cases" that require full 
storage access. 

These apps will be able to get full file access by declaring the MANAGE_EXTERNAL_STORAGE permission.

https://developer.android.com/about/versions/11/privacy/storage#scoped-storage

FunkyMuse commented 3 years ago

I've tried giving MANAGE_EXTERNAL_STORAGE permission to the sample but fetchMagnet still didn't work

<uses-permission android:name="android.permission.MANAGE_EXTERNAL_STORAGE"/>

@RequiresApi(Build.VERSION_CODES.R)
    private fun requestAllFilesAccessPermission() {
        val intent = Intent(Settings.ACTION_MANAGE_APP_ALL_FILES_ACCESS_PERMISSION)
        intent.data = Uri.parse("package:$packageName")
         try {
             startActivity(intent)
        } catch (e: Exception) {
            Log.w("PERMISSIONS", "Request all files access not supported", e)
        }
    }

After I granted it, for testing purposes and saving on time there's no startActivityForResult, restarted the app and tired again, it did not work.

If that permission is given to FrostWire then it won't be placed on the store since they'll reject it I think cause it doesn't fall under any of their categories, the best way for the library to work is within app's own storage (context.getStorageDir or context.getCacheDir) since with Android 11 other apps can't access another app's /dataDir and /obbDir

I guess some implementation detail with fetchMagnet is the problem? I could be wrong, i'm trying to help but I may be pointing towards the wrong direction.

gubatron commented 3 years ago

going back to fetchMagnet I think it has nothing to do with storage permissions, fetchMagnet is supposed to return a byte[] and then you do as you please with that byte[]

/**
     * @param uri     magnet uri
     * @param timeout in seconds
     * @param maxSize in bytes
     * @param extra   if extra data is included
     * @return the bencoded info or null
     */
    public byte[] fetchMagnet(String uri, int timeout, final boolean extra, final int maxSize) {
FunkyMuse commented 3 years ago

going back to fetchMagnet I think it has nothing to do with storage permissions, fetchMagnet is supposed to return a byte[] and then you do as you please with that byte[]

/**
     * @param uri     magnet uri
     * @param timeout in seconds
     * @param maxSize in bytes
     * @param extra   if extra data is included
     * @return the bencoded info or null
     */
    public byte[] fetchMagnet(String uri, int timeout, final boolean extra, final int maxSize) {

The thing is that it always returns null, even tested on FrostWire app itself when you compile it with SDK 30 and run it on Android 11

It works fine when you compile it with 29 and run it on Android 11

Tested using 1.2.10 on pixel XL 2

gubatron commented 3 years ago

that means it couldn't find the magnet and it timed out.

I suggest you place debug stops on the implementation of that method inside jlibtorrent, perhaps you'll be able to see other Alerts coming in that might tell you what's wrong.

Try to debug the shit out of this with an interactive debugger https://github.com/frostwire/frostwire-jlibtorrent/blob/master/src/main/java/com/frostwire/jlibtorrent/SessionManager.java#L613

FunkyMuse commented 3 years ago

that means it couldn't find the magnet and it timed out.

I suggest you place debug stops on the implementation of that method inside jlibtorrent, perhaps you'll be able to see other Alerts coming in that might tell you what's wrong.

Try to debug the shit out of this with an interactive debugger https://github.com/frostwire/frostwire-jlibtorrent/blob/master/src/main/java/com/frostwire/jlibtorrent/SessionManager.java#L613

E/ERROR: SESSION_ERROR - session_error - session error: (13 Permission denied) bind: Permission deniednt_swig_

E/ERROR: SESSION_STATS_HEADER - session_stats_header - session stats header: peer.error_peers, peer.disconnected_peers, peer.eof_peers, peer.connreset_peers, peer.connrefused_peers, peer.connaborted_peers, peer.notconnected_peers, peer.perm_peers, peer.buffer_peers, peer.unreachable_peers, peer.broken_pipe_peers, peer.addrinuse_peers, peer.no_access_peers, peer.invalid_arg_peers, peer.aborted_peers, peer.piece_requests, peer.max_piece_requests, peer.invalid_piece_requests, peer.choked_piece_requests, peer.cancelled_piece_requests, peer.piece_rejects, peer.error_incoming_peers, peer.error_outgoing_peers, peer.error_rc4_peers, peer.error_encrypted_peers, peer.error_tcp_peers, peer.error_utp_peers, picker.reject_piece_picks, picker.unchoke_piece_picks, picker.incoming_redundant_piece_picks, picker.incoming_piece_picks, picker.end_game_piece_picks, picker.snubbed_piece_picks, picker.interesting_piece_picks, picker.hash_fail_piece_picks, picker.piece_picker_partial_loops, picker.piece_picker_suggest_loops, picker.piece_picker_sequential_loops, picker.piece_picker_reverse_rare_loops, picker.piece_picker_rare_loops, picker.piece_picker_rand_start_loops, picker.piece_picker_rand_loops, picker.piece_picker_busy_loops, peer.connect_timeouts, peer.uninteresting_peers, peer.timeout_peers, peer.no_memory_peers, peer.too_many_peers, peer.transport_timeout_peers, peer.num_banned_peers, peer.banned_for_hash_failure, peer.connection_attempts, peer.connection_attempt_loops, peer.boost_connection_attempts, peer.missed_connection_attempts, peer.no_peer_connection_attempts, peer.incoming_connections, net.on_read_counter, net.on_write_counter, net.on_tick_counter, net.on_lsd_counter, net.on_lsd_peer_counter, net.on_udp_counter, net.on_accept_counter, net.on_disk_queue_counter, net.on_disk_counter, ses.num_incoming_choke, ses.num_incoming_unchoke, ses.num_incoming_interested, ses.num_incoming_not_interested, ses.num_incoming_have, ses.num_incoming_bitfield, ses.num_incoming_request, ses.num_incoming_piece, ses.num_incoming_cancel, ses.num_incoming_dht_port, ses.num_incoming_suggest, ses.num_incoming_have_all, ses.num_incoming_have_none, ses.num_incoming_reject, ses.num_incoming_allowed_fast, ses.num_incoming_ext_handshake, ses.num_incoming_pex, ses.num_incoming_metadata, ses.num_incoming_extended, ses.num_outgoing_choke, ses.num_outgoing_unchoke, ses.num_outgoing_interested, ses.num_outgoing_not_interested, ses.num_outgoing_have, ses.num_outgoing_bitfield, ses.num_outgoing_request, ses.num_outgoing_piece, ses.num_outgoing_cancel, ses.num_outgoing_dht_port, ses.num_outgoing_suggest, ses.num_outgoing_have_all, ses.num_outgoing_have_none, ses.num_outgoing_reject, ses.num_outgoing_allowed_fast, ses.num_outgoing_ext_handshake, ses.num_outgoing_pex, ses.num_outgoing_metadata, ses.num_outgoing_extended, ses.num_piece_passed, ses.num_piece_failed, ses.num_have_pieces, ses.num_total_pieces_added, disk.num_blocks_written, disk.num_blocks_read, disk.num_blocks_hashed, disk.num_blocks_cache_hits, disk.num_write_ops, disk.num_read_ops, disk.num_read_back, disk.disk_read_time, disk.disk_write_time, disk.disk_hash_time, disk.disk_job_time, ses.waste_piece_timed_out, ses.waste_piece_cancelled, ses.waste_piece_unknown, ses.waste_piece_seed, ses.waste_piece_end_game, ses.waste_piece_closing, net.sent_payload_bytes, net.sent_bytes, net.sent_ip_overhead_bytes, net.sent_tracker_bytes, net.recv_payload_bytes, net.recv_bytes, net.recv_ip_overhead_bytes, net.recv_tracker_bytes, net.recv_failed_bytes, net.recv_redundant_bytes, dht.dht_messages_in, dht.dht_messages_in_dropped, dht.dht_messages_out, dht.dht_messages_out_dropped, dht.dht_bytes_in, dht.dht_bytes_out, dht.dht_ping_in, dht.dht_ping_out, dht.dht_find_node_in, dht.dht_find_node_out, dht.dht_get_peers_in, dht.dht_get_peers_out, dht.dht_announce_peer_in, dht.dht_announce_peer_out, dht.dht_get_in, dht.dht_get_out, dht.dht_put_in, dht.dht_put_out, dht.dht_sample_infohashes_in, dht.dht_sample_infohashes_out, dht.dht_invalid_announce, dht.dht_invalid_get_peers, dht.dht_invalid_find_node
E/ERROR: SESSION_STATS - session_stats - session stats (299 values): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

Tested on Pixel XL 2 running Android 11 it doesn't work. Tested on Sony Xperia XZ3 running Android 10 and it works.

The error seems kind of a missing permission for something Permission deniednt_swig_

FunkyMuse commented 3 years ago

Running it again I get this kinda error E/ERROR: SESSION_ERROR - session_error - session error: (13 Permission denied) bind: Permission deniedfetch_magnet___magnet:myCustomMagnetVariableHere

Tested on Pixel XL 2 running Android 11 it doesn't work. Tested on Sony Xperia XZ3 running Android 10 and it works.

FunkyMuse commented 3 years ago

Sometimes the error can't be dumped when using SESSION_ERROR and I get this kinda JNI error A/etorrentandroi: runtime.cc:663] JNI DETECTED ERROR IN APPLICATION: input is not valid Modified UTF-8: illegal continuation byte 0xe0 runtime.cc:663] string: 'session error: (13 Permission denied) bind: Permission denied���L��L�' runtime.cc:663] input: '0x73 0x65 0x73 0x73 0x69 0x6f 0x6e 0x20 0x65 0x72 0x72 0x6f 0x72 0x3a 0x20 0x28 0x31 0x33 0x20 0x50 0x65 0x72 0x6d 0x69 0x73 0x73 0x69 0x6f 0x6e 0x20 0x64 0x65 0x6e 0x69 0x65 0x64 0x29 0x20 0x62 0x69 0x6e 0x64 0x3a 0x20 0x50 0x65 0x72 0x6d 0x69 0x73 0x73 0x69 0x6f 0x6e 0x20 0x64 0x65 0x6e 0x69 0x65 0x64 0xf6 <0xe0> 0xf2 0x4c 0xf6 0x1c 0xf3 0x4c 0xf6 0x02 0x01 0x02' runtime.cc:663] in call to NewStringUTF runtime.cc:663] from java.lang.String com.frostwire.jlibtorrent.swig.libtorrent_jni.session_error_alert_message(long, com.frostwire.jlibtorrent.swig.session_error_alert)

gubatron commented 3 years ago

any clue what that byte sequence is or where it's coming from?

it's not liking byte 0xe0 (224), perhaps it's a string encoding issue in your Java layer?

It's trying to pass an invalid UTF-8 string to a JNI function in libtorrent.

FunkyMuse commented 3 years ago

any clue what that byte sequence is or where it's coming from?

it's not liking byte 0xe0 (224), perhaps it's a string encoding issue in your Java layer?

It's trying to pass an invalid UTF-8 string to a JNI function in libtorrent.

My implementation was passing "utf-8", now I changed it to "UTF-8" First run i got this error again 2020-10-13 12:14:30.058 14550-14604/com.masterwok.demosimpletorrentandroid E/ERROR: SESSION_ERROR - session_error - session error: (13 Permission denied) bind: Permission deniedfetch_magnet___magnet:?xt=urn:btih:fa8bcc16683b3c4bf2178cd2e965f06ee23b6f36&dn=The.Food.Guide.to.Love.2013.BRRip.x264-HUD&tr=http%3a%2f%2ftracker.zamunda.net%2fannounce.php%3fpasskey%3d994c30f57886bff77851922c2f0a343e Second run this error 2020-10-13 12:15:32.853 14667-14738/com.masterwok.demosimpletorrentandroid A/etorrentandroi: runtime.cc:663] JNI DETECTED ERROR IN APPLICATION: input is not valid Modified UTF-8: illegal continuation byte 0x30 runtime.cc:663] string: 'session error: (13 Permission denied) bind: Permission denied�0*��P*��' runtime.cc:663] input: '0x73 0x65 0x73 0x73 0x69 0x6f 0x6e 0x20 0x65 0x72 0x72 0x6f 0x72 0x3a 0x20 0x28 0x31 0x33 0x20 0x50 0x65 0x72 0x6d 0x69 0x73 0x73 0x69 0x6f 0x6e 0x20 0x64 0x65 0x6e 0x69 0x65 0x64 0x29 0x20 0x62 0x69 0x6e 0x64 0x3a 0x20 0x50 0x65 0x72 0x6d 0x69 0x73 0x73 0x69 0x6f 0x6e 0x20 0x64 0x65 0x6e 0x69 0x65 0x64 0xd1 <0x30> 0x2a 0xe9 0xd1 0x50 0x2a 0xe9 0xd1 0x02' runtime.cc:663] in call to NewStringUTF runtime.cc:663] from java.lang.String com.frostwire.jlibtorrent.swig.libtorrent_jni.session_error_alert_message(long, com.frostwire.jlibtorrent.swig.session_error_alert) 2020-10-13 12:15:32.853 14667-14738/com.masterwok.demosimpletorrentandroid A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 14738 (SessionManager-), pid 14667 (etorrentandroid)

gubatron commented 3 years ago

For the future, you avoid these errors by using StandardCharsets.UTF_8.name().

Can you share the code causing this?

FunkyMuse commented 3 years ago

For the future, you avoid these errors by using StandardCharsets.UTF_8.name().

Can you share the code causing this?

It doesn't change anything even using this method. 2020-10-13 18:09:48.674 30391-30457/com.masterwok.demosimpletorrentandroid E/ERROR: SESSION_ERROR - session_error - session error: (13 Permission denied) bind: Permission deniedts.

 private fun downloadUsingMagnetUri(magnetUrl: String, downloadLocation: File) = synchronized(dhtLock) {
        shouldDownloadMagnetOnResume = false

        // We must wait for DHT to start
        if (!isDhtReady()) {
            dhtLock.wait()
        }

        // Session was paused while waiting on DHT, defer until resume
        if (sessionManager.isPaused) {
            shouldDownloadMagnetOnResume = true
            return
        }
        sessionManager.download(URLDecoder.decode(magnetUrl, StandardCharsets.UTF_8.name()), downloadLocation)
    }
gubatron commented 3 years ago

Can you share what this does?

sessionManager.download(URLDecoder.decode(magnetUrl, StandardCharsets.UTF_8.name()), downloadLocation)

We don't ever decode the URI, it's an URI, it's supposed to be URI encoded.

This is how we fetch magnets on FrostWire:

public byte[] fetchMagnet(String uri, int timeout, final boolean extra, final int maxSize) {
        if (this.session == null) {
            return null;
        } else {
            error_code ec = new error_code();
            add_torrent_params p = add_torrent_params.parse_magnet_uri(uri, ec);
            if (ec.value() != 0) {
                throw new IllegalArgumentException(ec.message());
            } else {
                p.set_disabled_storage();
                final sha1_hash info_hash = p.getInfo_hash();
                final byte[][] data = new byte[][]{null};
                final CountDownLatch signal = new CountDownLatch(1);
                AlertListener listener = new AlertListener() {
                    public int[] types() {
                        return SessionManager.METADATA_ALERT_TYPES;
                    }

                    public void alert(Alert<?> alert) {
                        torrent_handle th = ((torrent_alert)((TorrentAlert)alert).swig()).getHandle();
                        if (th != null && th.is_valid() && !th.info_hash().op_ne(info_hash)) {
                            AlertType type = alert.type();
                            if (type.equals(AlertType.METADATA_RECEIVED)) {
                                MetadataReceivedAlert a = (MetadataReceivedAlert)alert;
                                int size = a.metadataSize();
                                if (0 < size && size <= maxSize) {
                                    data[0] = a.torrentData(extra);
                                }
                            }

                            signal.countDown();
                        }
                    }
                };
                this.addListener(listener);
                boolean add = false;
                torrent_handle th = null;

                try {
                    this.syncMagnet.lock();

                    try {
                        th = this.session.find_torrent(info_hash);
                        if (th != null && th.is_valid()) {
                            add = false;
                            torrent_info ti = th.torrent_file_ptr();
                            if (ti != null && ti.is_valid()) {
                                create_torrent ct = new create_torrent(ti);
                                entry e = ct.generate();
                                int size = ti.metadata_size();
                                if (0 < size && size <= maxSize) {
                                    data[0] = Vectors.byte_vector2bytes(e.bencode());
                                }

                                signal.countDown();
                            }
                        } else {
                            add = true;
                        }

                        if (add) {
                            p.setName("fetch_magnet___" + uri);
                            p.setSave_path("fetch_magnet___" + uri);
                            torrent_flags_t flags = p.getFlags();
                            flags = flags.and_(TorrentFlags.AUTO_MANAGED.inv());
                            flags = flags.or_(TorrentFlags.UPLOAD_MODE);
                            flags = flags.or_(TorrentFlags.STOP_WHEN_READY);
                            p.setFlags(flags);
                            ec.clear();
                            th = this.session.add_torrent(p, ec);
                            th.resume();
                        }
                    } finally {
                        this.syncMagnet.unlock();
                    }

                    signal.await((long)timeout, TimeUnit.SECONDS);
                } catch (Throwable var26) {
                    LOG.error("Error fetching magnet", var26);
                } finally {
                    this.removeListener(listener);
                    if (this.session != null && add && th != null && th.is_valid()) {
                        this.session.remove_torrent(th);
                    }

                }

                return data[0];
            }
        }
    }

What happens if you pass the magnet URI without decoding it?

FunkyMuse commented 3 years ago

Can you share what this does?

sessionManager.download(URLDecoder.decode(magnetUrl, StandardCharsets.UTF_8.name()), downloadLocation)

We don't ever decode the URI, it's an URI, it's supposed to be URI encoded.

This is how we fetch magnets on FrostWire:

public byte[] fetchMagnet(String uri, int timeout, final boolean extra, final int maxSize) {
        if (this.session == null) {
            return null;
        } else {
            error_code ec = new error_code();
            add_torrent_params p = add_torrent_params.parse_magnet_uri(uri, ec);
            if (ec.value() != 0) {
                throw new IllegalArgumentException(ec.message());
            } else {
                p.set_disabled_storage();
                final sha1_hash info_hash = p.getInfo_hash();
                final byte[][] data = new byte[][]{null};
                final CountDownLatch signal = new CountDownLatch(1);
                AlertListener listener = new AlertListener() {
                    public int[] types() {
                        return SessionManager.METADATA_ALERT_TYPES;
                    }

                    public void alert(Alert<?> alert) {
                        torrent_handle th = ((torrent_alert)((TorrentAlert)alert).swig()).getHandle();
                        if (th != null && th.is_valid() && !th.info_hash().op_ne(info_hash)) {
                            AlertType type = alert.type();
                            if (type.equals(AlertType.METADATA_RECEIVED)) {
                                MetadataReceivedAlert a = (MetadataReceivedAlert)alert;
                                int size = a.metadataSize();
                                if (0 < size && size <= maxSize) {
                                    data[0] = a.torrentData(extra);
                                }
                            }

                            signal.countDown();
                        }
                    }
                };
                this.addListener(listener);
                boolean add = false;
                torrent_handle th = null;

                try {
                    this.syncMagnet.lock();

                    try {
                        th = this.session.find_torrent(info_hash);
                        if (th != null && th.is_valid()) {
                            add = false;
                            torrent_info ti = th.torrent_file_ptr();
                            if (ti != null && ti.is_valid()) {
                                create_torrent ct = new create_torrent(ti);
                                entry e = ct.generate();
                                int size = ti.metadata_size();
                                if (0 < size && size <= maxSize) {
                                    data[0] = Vectors.byte_vector2bytes(e.bencode());
                                }

                                signal.countDown();
                            }
                        } else {
                            add = true;
                        }

                        if (add) {
                            p.setName("fetch_magnet___" + uri);
                            p.setSave_path("fetch_magnet___" + uri);
                            torrent_flags_t flags = p.getFlags();
                            flags = flags.and_(TorrentFlags.AUTO_MANAGED.inv());
                            flags = flags.or_(TorrentFlags.UPLOAD_MODE);
                            flags = flags.or_(TorrentFlags.STOP_WHEN_READY);
                            p.setFlags(flags);
                            ec.clear();
                            th = this.session.add_torrent(p, ec);
                            th.resume();
                        }
                    } finally {
                        this.syncMagnet.unlock();
                    }

                    signal.await((long)timeout, TimeUnit.SECONDS);
                } catch (Throwable var26) {
                    LOG.error("Error fetching magnet", var26);
                } finally {
                    this.removeListener(listener);
                    if (this.session != null && add && th != null && th.is_valid()) {
                        this.session.remove_torrent(th);
                    }

                }

                return data[0];
            }
        }
    }

What happens if you pass the magnet URI without decoding it?

I compiled frostwire from your repo, tried downloading using magnet got the same errors as above, the permission denial something JNI.

Just use target SDK 30 and run it on an emulator or real device, it won't work.

gubatron commented 3 years ago

lower your target then :)

FunkyMuse commented 3 years ago

lower your target then :)

Well that's what I'm doing to work around that issue, but I think this is gonna follow SDK >= 30

gubatron commented 3 years ago

We'll have to test building at some point against SDK 30 or higher the jlibtorrent library itself. Perhaps Android still has more to mature when it comes to storage access. They keep changing everything all the time.

master255 commented 3 years ago

@gubatron There are two suspicious points: p.setSave_path("fetch_magnet___" + uri); - this is a strange save path. Usually, the path starts with: /storage/emulated/0/Folder

                torrent_flags_t flags = p.getFlags(); - why get flags if there is no torrent metadata?
                flags = flags.and_(TorrentFlags.AUTO_MANAGED.inv());
                flags = flags.or_(TorrentFlags.UPLOAD_MODE);
                flags = flags.or_(TorrentFlags.STOP_WHEN_READY);
                p.setFlags(flags);

I only use this command: p.setFlags(TorrentFlags.UPLOADMODE.or(TorrentFlags.APPLY_IP_FILTER)));

gubatron commented 3 years ago

@master255 If I remember correctly, that's just a dummy path, notice how the storage is disabled at the very beginning:

p.set_disabled_storage();

We just need the bytes of the torrent to create the torrent handle, we don't need to save those bytes to a .torrent file on disk (slower IO) to then start the transfer.

FunkyMuse commented 3 years ago

We'll have to test building at some point against SDK 30 or higher the jlibtorrent library itself. Perhaps Android still has more to mature when it comes to storage access. They keep changing everything all the time.

any updates on this?

gubatron commented 3 years ago

On our android Java project we're still building against SDK 29. We want to maintain backwards compatibility for as long as possible.

As for jlibtorrent android binaries, we bumped from api 19 -> 21 on all architectures.

gubatron commented 2 years ago

Update: We'll now be building against API 24, which supports IFADDRS sockets, the alternative to NETLINK sockets which are not supported by Android 11

Making adjustments, fingers crossed.

FunkyMuse commented 2 years ago

Fixed in 1.2.14.0

gubatron commented 2 years ago

Great to hear the new build fixes this