Multiple sessions error

[Morpheusai]

Hi

We use multiple sessions in this way:

    for (int i = 0; i < Sessions_Count; i++) {
            SessionManager session = new SessionManager(true);
            session.addListener(mainListener);
            session.start(SessionSettings.getTauSessionParams());
            sessions.add(session);
    } 

But a fatal error happened:

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fc314d33585, pid=4363, tid=0x00007fc293ef6700
#
# JRE version: Java(TM) SE Runtime Environment (8.0_261-b12) (build 1.8.0_261-b12)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.261-b12 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libjlibtorrent-1.2.7.0.so+0x432585]  aesni_ecb_encrypt+0x1a5
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as: 
# /home/cowtc/Codes/dhtTau/taucoin-linux/hs_err_pid4363.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#

Also with a thread sleep, like 100ms. No error happen but will crash soon when started with the same error.

Thanks for reply.

[Morpheusai]

@arvidn

could you give us some advices ?

[arvidn]

my advise would be to load up the core file in gdb and see what's going on.

[gubatron]

have you tried testing with just 2 sessions? I don't think we ever tested working with more than 1 session at the time (per process)

Are your sessions sharing the same settings? (by the likes of your code it seems they are, that probably leads to all sorts of fucked up behavior when writing/reading settings)

That doesn't sound right.

[master255]

@Morpheusai No one uses many sessions. There is no such scenario.

[wuzhengy]

When we want to fine control the data transmission for the app, we can potentially have multiple session with different setting such as one session with “read only” and “branch factor =1” ... to maintain minimum data traffic, other sessions can scale up when network data is less expensive. So we do not have to load and destroy sessions frequently, this also saves data transmission. We has fixed the multiple session issues with recompile, will publish the compile detail later. Thanks.

[master255]

@wuzhengy We have a "read only" flag and a speed limit for specific torrentHandle. This should be enough for all scenarios.

[wuzhengy]

We are making a chat software using dht data put/get functions. The dht get function sometimes does not respond with data even the data is available somewhere in the swarm which is the collective data store by other peers. We think if using multiple ID will bring more diversity of each routing take to start with. Single session single routing table seems to be too focus on one set of routing due to XOR distance to the node ID. We might be able to use multiple network interface to reflect this nodes ID diversity, but the multiple network interface shares does not give fine control on which interface process which data item get task. The multiple session gives us lots of control flexibility in terms of concurrency get data items that represent chatting messages. Hope this makes sense to use multiple sessions. We did make multiple sessions works to distribute the get tasks so far.

[Morpheusai]

We have figured out the problem, due to the compile options of openssl in build*.sh. (including linux and android platform) export run_openssl_configure="./Configure linux-x86_64 ${OPENSSL_NO_OPTS} -fPIC --prefix=${OPENSSL_ROOT}"; We changed into: export run_openssl_configure="./Configure linux-x86_64 -fPIC --prefix=${OPENSSL_ROOT}";

Also there are some problems in build.sh. We have tested five build.sh (build-android-x86_64.sh, build-android*.sh). In common compile file - build-utils.shinc, we first confirm the tools(unzip, gradle, git etal) are installed in the system. Another important things:

1. Boost install. you must assign different toolset in different system, gcc as default in linux and clang in android system.

2. Serveral envionment variables are better to be assigned before later compile stages, so that you can manually compile by yourself when errors happen.

In build-linux-x86_64.sh and build-android-*.sh, toolset=*** needs to be modified like: export run_bjam... toolset=clang--${os_arch} ..." should be export run_bjam... toolset=clang-${os_arch} ...

Also, some compile options in swig/config/*.jam need to be ignored or modified. these may be caused by different systems.

In build-android-x86_64.sh: export CC=$ANDROID_TOOLCHAIN/bin/i686-linux-android-clang should be export CC=$ANDROID_TOOLCHAIN/bin/x86_64-linux-android-clang and export run_openssl_configure="./Configure linux-elf... shoule be export run_openssl_configure="./Configure linux-x86_64

Hope helpful to developers. I will close the issue.

[arvidn]

It sounds like this issue requires a fix in either the cmake files or the documentation. I have no stake in this repo specifically, but it would seem like a good thing dra to leave the ticket open until a fix has landed.

[gubatron]

Thank you @Morpheusai

Here's the first fix with the extra dash on the toolset=clang-${os_arch} error you spotted https://github.com/frostwire/frostwire-jlibtorrent/commit/7cf2b75153da5155c9f637465a91ea5792425111

As for removing ${OPENSSL_NO_OPTS} when building openssl, I'm not so sure about that. We do this so that we keep the library to a minimum size, perhaps in your case you need more of what openssl has to offer, but not really for FrostWire builds. How big is the jlibtorrent library for you when you build openssl with everything?

[gubatron]

The other settings I'll have to try and test, if everything works alright I'll merge those changes into the repo for everybody.

Thank you so much @Morpheusai

Thank you @arvidn for taking a look at this, your input is most valuable and always welcome.

[gubatron]

Second set of fixes passed on my local build, thanks @Morpheusai https://github.com/frostwire/frostwire-jlibtorrent/commit/fa8fb34783eefad14f425db3629b8c16bc40e25d

[Morpheusai]

Thank you @Morpheusai

Here's the first fix with the extra dash on the toolset=clang-${os_arch} error you spotted 7cf2b75

As for removing ${OPENSSL_NO_OPTS} when building openssl, I'm not so sure about that. We do this so that we keep the library to a minimum size, perhaps in your case you need more of what openssl has to offer, but not really for FrostWire builds. How big is the jlibtorrent library for you when you build openssl with everything? libjlibtorrent.so - 9.2 mb, linux, or android - 1.2.10.jar - 3.4 mb, 3.1 mb jlibtorrent-1.2.10.0.jar is 677 kb, almost the same as before.

[gubatron]

Yeah, so the .so grows about 2Mb worth of unnecessary symbols and attack surfaces.

[gubatron]

so thinking more about this, perhaps there's a flag we can remove from OPENSSL_NO_OPTS so that you don't have the aesni_ecb_encrypt related crash, or perhaps with a simple update to openssl 1.1.1h the problem might go away, and you get to save 2Mb per download (if you're distributing a client like us)

Arvid just bumped the 1.2 branch to 1.2.11, so I'll be making an update here soon, I'll bump openssl to see if your issue goes away.

I'll try also bumping boost to 1.75 if it came out, we couldn't build correctly with 1.74 back in september.

[wuzhengy]

I think saving 2mb per download is more important. The reason we need multi-sessions is for a very specific purpose to fine tune each session's settings to help chatting and imaging in Africa network.