Stuck in Jamtaba server connection on win+osx platform

[fab672000]

Hi Team, anyone having this problem?

Debugging through it and quickly patching the call to be synchronized for isolating the problem, I ended up finding that http://jamtaba2.appspot.com/vs is never connecting... Any idea what is happening ? the command below does not connect anymore: loginService.connectInServer(userName, 0, "", map, version, userEnvironment, getSampleRate()); It happened fairly recently to me with and will 2.1.8, 2.1.7 on both windows and osx platforms.

I also checked do have internet and the site is accessible from a web browser.

[fab672000]

Also is it possible to run the service locally (I noted an alternative local 8080 connection in the commented code) ? if yes how to setup this local web server?

[elieserdejesus]

http://jamtaba2.appspot.com/vs is never connecting...

@fab672000 , I tested here in Windows and Mac (Debug and Release versions) and is working without problems.

[elieserdejesus]

Also is it possible to run the service locally (I noted an alternative local 8080 connection in the commented code) ? if yes how to setup this local web server?

1 - Install NetBeans (download the full package because you need the JEE)

2 - Download GAE - Google App engine SDK

3 - Configure the GAE SDK in NetBeans. Go to Tools -> Servers and Add the Gae Server

4 - Clone the JamTaba-Server (is a GAE NetBeans Project)

5 - Open the Project in NetBeans and Run! Now you have the server running in localhost!

[fab672000]

Thanks Eli will do! It works when I use a VPN connection, is it possible that I got blacklisted somehow after intense debugging repeated sessions ?

[elieserdejesus]

is it possible that I got blacklisted somehow after intense debugging repeated sessions ?

It's possible, but I can't see a blacklist in the GAE admin panels. Please tell me If you see something about "blacklist" in your local server.

[fab672000]

nothing, local server works fine, only the other server gets me rejected but not if I use vpn hence my question, it seems recent toolike not more than 10 days :-/

[fab672000]

they seem to say there is one here but I'm not a web guy so not sure where to look there...

[elieserdejesus]

The link is old, the UI changed in GAE admin panels.

@fab672000 , try to create a new project in QtCretor and just access the jamtaba server URL. Check the http error status you are receiving, maybe this can be an important clue to undestand this issue:

http://doc.qt.io/qt-5/qnetworkaccessmanager.html#details

[fab672000]

Now i reconnect on osx, but the public chat is still dead (still better as i can jam at least :) Here's what i get right now

10.5.4 503 Service Unavailable. The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If known, the length of the delay MAY be indicated in a Retry-After header.

[fab672000]

Also, I am using your latest OSX 64 bit build ATM so it cant be my qt lib ?

[elieserdejesus]

Also, I am using your latest OSX 64 bit build ATM so it cant be my qt lib ?

@fab672000 , I'm 99% shure it's your Qt lib. What Qt version are you using?

In JamTaba release I'm using Qt 5.6. There are changes in Qt network package since 5.6 (it's a bit old version), so this can explain the bugs you are seing.

I'm using Qt 5.6 because is compatible with Mac OSX 10.7+. Newer Qt versions are not compatible with OSX 10.7.

[fab672000]

But I am using the installed version that you built right now, and same thing happens so it cannot be the Qt lib right? is it built statically anyway and I am not currently even using my build but only the your installer so the qt lib is the one you used.

[fab672000]

looking at more logged errors in the chat room: I could get https://www.herokucdn dot com/error-pages/application-error.html Do we use herokucdn dot com? (replace dot by .)

[fab672000]

Also tom reported similar problems with the site and the public chat.

[elieserdejesus]

@fab672000 , the public chat is a very simple NodeJS application runing in Heroku, using a free account. Since is a free account the public chat quota is always exausted in some day of the month (this is the price for free 😄). Right now (12/30/2018) the quota is exausted, but instead a error page I'm just seeing this in the public chat:

I will check for this error page appearing in the chat, this is not expected.

---

The public chat (running in Heroku) is 100% decoupled from JamTaba server running in GAE. This reduce the GAE costs (I'm paying some dolars every month) and keep the "modules" independent. If one server is shutdown the other survive. So, a problem in public chat is not related with problems in JamTaba server, they are separated beasts 😃 living separated lives.

---

Just to clarify: JamTAba server is just feeding JamTaba clients with a json containing the public servers and the connected ninjamers in each server. This JamTaba server is used to avoid overload the Cockos server with every JTBA client doing a new http request every 30 seconds.

---

Previously you said:

Now i reconnect on osx, but the public chat is still dead (still better as i can jam at least :)

So, I will assume you are connecting with JamTaba server, and only the public chat server is broken. Please correct me if I'm wrong.

If your problem is only the public chat server I think the main problem (the title of this issue) is still your Qt lib.

The public chat is using the WebSocket Qt module, and the JamTaba server client/server communication is using QtNetwork module. These are separated modules in Qt and have different bugs fixed or not in each Qt version.

---

You mentioned Tom reported some problems too: Another reason to these crazy issues in servers can be something related with load balance in GAE servers (this not explain the public chat issue). GAE feed users requests using the most close server. For example, when I connect the jamtaba server possibly I'm not connecting in same server as you. This load balanced is managed by GAE, and maybe this is a temporary in these services. I will search if other devs are seing similar issues in GAE services.

[fab672000]

So, I will assume you are connecting with JamTaba server, and only the public chat server is broken. Please correct me if I'm wrong.

Thanks for the clarifications!

So I am experimenting two different problems, one is the public chat connection broken which I originally associated with the server connection problem, and the other is actually the jamtaba server connection, I believe and you are probably right that Qt is the problem, I'll update it.

I think I was confused at the beginning as I was stuck because of the jamtaba server connection never resuming (why don't we have any time-out there if rooms never load?).

Why do we need to mandatorily connect to the Jamtaba Server prior to getting the rooms? This seems to cause a potential bottle neck and that is actually blocking in the case the server is not responding but the other ninjam servers are fine.

Also It could cost you zero dollars if we could remove this bottleneck?

[elieserdejesus]

I think I was confused at the beginning as I was stuck because of the jamtaba server connection never resuming (why don't we have any time-out there if rooms never load?).

I'm guessing you are seeing a QtNetwork bug. I suspect the bug will disappear using Qt 5.6.

---

Why do we need to mandatorily connect to the Jamtaba Server prior to getting the rooms? This seems to cause a potential bottle neck and that is actually blocking in the case the server is not responding but the other ninjam servers are fine. Also It could cost you zero dollars if we could remove this bottleneck?

You are right about the bottleneck, but there are another things to consider:

1 - When you enter in a public ninjam server the last chord progression used is displayed in the chat. The last chord progression is stored in JamTaba server for each public ninjam server.

2 - Using a centralized server is possible offer other services in the future (like the chord progression storage mentioned before).

3 - In these years developing JamTaba I made a lot of decisions and using the server I have some data to make better decisions. This is very important for me because I'm making no money with JamTaba, so is mandatory (for me) build something usefull for real users, not for geeks only, teck savy people only, etc.. For example, seeing the connected users in the server I can see some patterns, and these patterns help me when I need decide something important:

• a) the VST plugin is more used (65% of users) than standalone.
• b) about 60% of VST plugin users are not using Reaper, they are using Cubase, Ableton, FLStudio, Logic, Studio one. This is a game changing in the ninjam world, people are not stucked with Reaper, people can use your preferred DAW. For me this means: is the right way, and I'm not guessing, people data usage is showing me!

4 - But the main reason I build the JTBA server is to avoid overload Cockos server. For example, imagine 30 JTBA requesting directly to Cockos http server. In the worst case scenario JTBA_01 will request, one second after JTBA_02 will request too, one second after JTBA_03 will request, etc. After 30 seconds que cocks server will receive 30 http requests, one request per second (considering the worst scenario).

Using JamTAba server in the same scenario just one http request is done every 30 seconds. If the 30 JamTabas are requesting data the server will return cached data, and will refresh the cache after 30 seconds. In this exagerated worst case example we are reducing the Cockos http server request rate in 30x.

But why? Why not just request a lot from the Cockos server and force the Cockos guys to use more memory and Cpu?

I'm being very conservative about the Cockos server. Ninjam is not a priority in Cockos, Reaper is the priority. I'm supposing the Cockos server is a small virtual machine running with 128/256 RAM, and I'm not interested in put the Cockos server down doing a lot of http requests per minute, because this will put down the entire ninjam community. Even people using the ReaNinjam plugin will not be able to play. My idea always be improve the community, not destroy 😄 .

ReaNinjam is updating the servers list just one time when you open the plugin, no more requests when you are playing. When you quit from a server the public servers list is refreshed. It's a very low request rate in comparison with JamTaba. I think all ninjam clients are doing the same thing (no auto refresh). Because this "default refresh approach" in ninjam clioents I suppose the Cockos server is expecting low request rates.

[fab672000]

You are right about the bottleneck, but there are another things to consider: 1 - When you enter in a public ninjam server the [last chord progression used is displayed in the chat]

Understood but couldn't we parse a chords progression in the chat directly? i.e. when ever we match a regex of \|.*\|we attempt to extract a chord sequence and if correctly parsed we use the chord progression? Would be simple and portable wouldn't it?

2 - Using a centralized server is possible offer other services in the future (like the chord progression storage mentioned before).

We could maybe use more modern approaches like distributed networking (my son loves Minecraft and this game uses serverless user game hosting techniques). I am thinking of this as one metaphor that some people were writing about a while ago :) we could use a much simpler implementation of this concept just using a multicast auxiliary network, note that we could have our public chat working great that way too and now every user is a light server ;-) Just a crazy idea, maybe not for now!

4 - But the main reason I build the JTBA server is to avoid overload Cockos server.

You could also use ninbot servers I guess no? it would not be discovering everything all the time, there is a lot we could cache I suppose ?

3. ../.. This is a game changing in the ninjam world, people are not stuck with Reaper, people can use your preferred DAW. For me this means: is the right way, and I'm not guessing, people data usage is showing me!

Point taken, I understand you like getting the stats and why but maybe we can have both and only a recovery mode in case the server is not accessible? not a priority though as I guess we can make it with Qt5.6.3 (still building after several coffees here as I need 4 builds ATM)

-Fab

[elieserdejesus]

Understood but couldn't we parse a chords progression in the chat directly?

This is already implemented, and parse the chord progression in the chat is different from remember the last chord progression for each public server. To remember the last progression we need some persistence (server side persistence is the simpler scheme).

---

we could use a much simpler implementation of this concept just using a multicast auxiliary network, note that we could have our public chat working great that way too and now every user is a light server ;-) Just a crazy idea, maybe not for now!

The idea sounds like a p2p network. In the previous JamTAba (coded in Java) I implemented realtime connections peer to peer using UDP protocol. In my experience TCP/IP client & server is much simpler.

You could also use ninbot servers I guess no? it would not be discovering everything all the time, there is a lot we could cache I suppose ?

Yes, I'm parsing the ninbot and Cockos because I need information from both. Unfortunally these sources don't have all variables. I asked Tom another day about add a "max_users" property in the ninbot json to each server, this will simplify a lot the parsing. Tom say is possible, but I don't know if he is working on this

maybe we can have both and only a recovery mode in case the server is not accessible?

I think this is a good idea, I'm just not shure how the UI will be affected. In OhmStudio, for example, we have a small refresh button in the bottom right corner. My first ideas are similar, a small refresh button in some corner 😄 .

[fab672000]

The idea sounds like a p2p network. In the previous JamTAba (coded in Java) I implemented realtime connections peer to peer using UDP protocol. In my experience TCP/IP client & server is much simpler.

Not really, using multicast you could send once and receive on each peer of the multicast group, way more efficient. I agree it would not be as simple as a client server implementation, but would be needing no special libs, and a chat system could be actually the less complex distributed design to start with first. Just an idea for now but I think it could be great and would make the system more fault tolerant as well (if one user disappears abruptly, others can still share their local state to allow others to rebuild a more global state.

[fab672000]

But I'm not sure how we would multicast on a public internet, I used it only on LAN's, not sure it would be possible to do at a larger scale without costs, damn :)

[fab672000]

Ok, rebuilt under Qt 5.6.3 static x64 new fresh build and jamtaba server connection works fine -> confirmed. Now closing this one as the public chat is just a limitation not really a bug. Finally I can test my fixes for the other issue in Windows :-)

Thanks for the in-depth support @elieserdejesus , I have learned a lot more about the software in the networking aspects!

[elieserdejesus]

But I'm not sure how we would multicast on a public internet

This is the big problem 😄 !

Ok, rebuilt under Qt 5.6.3 static x64 new fresh build and jamtaba server connection works fine -> confirmed.

Nice news!

Thanks for the in-depth support @elieserdejesus , I have learned a lot more about the software in the networking aspects!

You are welcome, thanks for your contributions!