search

jamulussoftware / jamulus

Jamulus enables musicians to perform real-time jam sessions over the internet.
https://jamulus.io
Other
997 stars 222 forks source link

Changes in Server List management #149

Closed corrados closed 4 years ago

corrados commented 4 years ago

See the following post: https://sourceforge.net/p/llcon/discussion/533517/thread/3487509361/?limit=25#0277

Most of the servers are empty.

Before Corona, most of the server were empty most of the time...

There are many more servers that performers. Some of the empty servers do not work well (badly hosted). Many people still don't know how to set up a private server and are listed inadvertently.

Agreed. But how can you distinguish a badly performing server from very well configured, powerfull servers?

Some people run several servers (perhaps they should run their own central server).

This would be something which can be detected easily. So if the server list is full, we might drop multiple server from the same IP address.

Again, I suggest the default is to NOT register centrally (private server).

This should be the case. It should be "not register" if you start up the Jamulus server for the first time.

I understand it feels nice to see your server name published, but many empty servers are a waste of limited resources and their presence denies access for others.

I have written that in another place: If a server is empty, this might not be an indicator that it is a bad server.

I've tried really hard to consider why more servers are a good thing and failed.

You need a server which is close to your location. The more servers are listed, the higher the probability that you have one which is close to you.

Ok, so these were the my comments on the forum post. But we should define treatments for the server list manager in this Issue which improves the situation. Here is a list I could think of:

pljones commented 4 years ago

If someone has registered multiple servers on the same PC, only one is preserved and the others are removed from the list.

I know of a provider in London who runs several servers and they're regularly in use. So that also needs to be taken into account.

Also, you don't know if it's on one PC - only that it's on the same public IP address...

(In the above case, I think he's running on Amazon AWS or something, so not exactly "on one PC" at all.)

pljones commented 4 years ago

Regarding

Question: If the list is full and a Permanent server tries to register, which other server do we kick out? A non-permanent server which is longest/shortest in the list?

from #140

The thing is, how does the Central Server know to stop or start advertising a server? The Central Server doesn't go out and count the clients on each server currently - that's left to the interested party: the client.

So you'd have to ask each server to tell the Central Server when it sends it's regular "ping", "Hey, and I've got non-zero clients right now" (or zero...). If it keeps saying zero, it'll be listed for losing its spot in the list (but not lose it).

OK, so along comes a server that's not in the Central Server's list. The Central Server, of course, doesn't know if it's previously been listed, so considers it "new", regardless. It checks to see if there's space. If not, the oldest empty server gets dropped off.

Now that dropped server sends its next "Ping, empty". Of course, it's now not in the Central Server's list. So the above gets applied to this server, as if it had never been seen before...

It gets a bit of a juggling act and seems like it's just going to be bouncing one or two entries around forever...

gilgongo commented 4 years ago

I'm unclear what problem this ticket is aiming to solve. Is it to increase the quality (however that might be defined) of the servers listed?

corrados commented 4 years ago

Well, yes. Wasn't it the intention of the initial forum post? Since we had a lot of discussions going on about this topic, I just wanted to create an Issue about it to collect all the ideas and maybe we can conclude some specifications from the discussion to improve the server list manager. But if we decide that we should keep the current implementation, just fine with me :-).

gilgongo commented 4 years ago

Ok, so I guess we're talking about the following issues (perhaps not a complete list)?

  1. Empty servers: do they pose a problem (eg in preventing other servers from joining)? If so, what to do about that?

  2. No servers appearing in the list: is there anything we can do about that?

  3. Few servers appearing, or some people not able to see other people's servers in the list: is there anything we can do?

corrados commented 4 years ago

  1. Not really, as I wrote in other discussions. I think you cannot conclude much from the informationt hat a server is empty.

  2. You mean the problem with blocked UDP ports? We should discuss this issue in a different Issue. This is what this issue should not be about.

  3. Same as 2.

In this Issue, the problem with the "200 servers limit" issue shall be improved (just a bit maybe). Right now the server list manager has no decision implemented in the registration process. We might implement some but have to be very careful about not to discriminate server operators and also not oversee possible usecases.

I think this discussion will be a long discussion... ;-)

pljones commented 4 years ago

I just noticed that Central Server (NA) is basically being used as "over-spill" currently. Whilst there's a fair few NA servers there, there seems to be quite a lot of EU/AsiaPac servers on there, too -- I guess once they get told "Sorry, full", the operators just try the other central server in the drop down.

gilgongo commented 4 years ago

I guess once they get told "Sorry, full", the operators just try the other central server in the drop down

Is that a bad thing?

pljones commented 4 years ago

Is that a bad thing?

If the idea is that one server is for North America and one for Rest of World, having non-NA on the NA server seems somehow wrong, wouldn't you say?

gilgongo commented 4 years ago

Well if it gets them listed, then presumably it does what they want :-) This if of course assuming they understand that if they want their friends to connect to their server, those friends also need to set their clients to the same one.

This was partly why I raised #139. A geographic split might makes sense to us, but it may not make sense to other people if we're not seeing the split occurring as we had hoped. So would the split happen naturally if a server operator wanted to play with jazz musicians, and client operators wanted to do the same? They would both set their central servers (aka "genre servers?) to the "correct" one without thinking, and thereby both avoid the 200 limit and scratch their genre itch.

pljones commented 4 years ago

Same applies with the network split -- if you want to play with people with a decent ping distance, you pick the right server. If there were only two genre servers, say "DubHouseReggae" and "AllOthers", I guess there'd be a few non-dub, etc, on the former...

Ideally there'd be a way of splitting by ping to the central server - anything under 100ms on one, anything over 100ms on the other. That would likely group by similar ping distance for clients. Clients could then automatically be redirected to the "nearer" server based on their ping distance to whichever the "primary" central server was.

gilgongo commented 4 years ago

I guess there'd be a few non-dub, etc, on the former...

Yes, and that wouldn't be perceived as a problem I don't think (although more realistically we'd have Rock, Jazz, Folk, Other).

The idea of choosing by network ping sounds interesting - but again, it doesn't naturally fit into the way musicians think, so we might still get people asking why their friends can't see their servers, for example.

pljones commented 4 years ago

The point with split by ping is that it's transparent. The setting would go back to "manual or default" -- default meaning "use the split by ping method".

Splitting the two servers (remember, that's a current constraint) by genre is always going to leave people wondering "do I put my Jazz / HipHop / Metal Fusion server on Other or RockJazzFolk?" (And yes, there are bands that do Jazz/HipHop/Metal Fusion.) I'd guess most people are going to think "What sounds like 'mainstream'?" and use that one -- it'll then get full. "Other" doesn't sound "mainstream"...

Any of these solutions requires client and server users to upgrade and would need to be backward compatible at the central servers -- so no overnight solution.

gilgongo commented 4 years ago

OK so I wasn't aware that we could only have two central servers. If that's a technical constraint we can't overcome then the genre idea obviously isn't feasible for the reasons you point out (I shall leave aside the question of why we can't have more than two). My idea would require four I think.

As to backward compatibility, I don't think that's necessarily huge big problem if we put the word out early enough.

corrados commented 4 years ago

OK so I wasn't aware that we could only have two central servers.

That is not true. We could have more than two. But of course, it would require code changes and a new Jamulus release to enable it. But right now we have space in the North America Central server so I would not create another one until that list is also filled up.

gilgongo commented 4 years ago

But right now we have space in the North America Central server so I would not create another one until that list is also filled up

Yes. And as I said in #139 it's only been a couple of weeks since the geographic central servers have been in operation so it makes sense to wait.

dszgit commented 4 years ago

OK so I wasn't aware that we could only have two central servers.

That is not true. We could have more than two. But of course, it would require code changes and a new Jamulus release to enable it. But right now we have space in the North America Central server so I would not create another one until that list is also filled up.

Maybe we need a Central central server Server :-)

pljones commented 4 years ago

The existing design is hub and leaf - one or more central hub servers off which some number of leaf servers hang.

I've been thinking about the problem on and off and came up with a more "peer to peer" or "tree" idea for publishing server details.

Client and server and what's currently "central server" would all use pretty much the same server list management in this world.

Any server could offer itself as a "central server" -- but in doing so it would take on certain responsibilities.

When a server wants to publish itself, it can choose "manual" or "default". They work pretty much the same way -- except that "default" doesn't just refer to one server but to a maintainable list (some would need to be "built in"). It would work a bit like DNS (which is what made me think of it).

The first thing the server does is say "Hey, give me the server list" to the manual server or the first default server. If the manual server fails, that's it, bad luck, try again. If the first default server fails, try the next, etc. Gives a bit of fail over.

Once it has the server list, it filters it on a new field -- "I am a central server". This is because, as mentioned any server can offer itself as a central server. It then establishes a steady ping to each and says to the nearest "OK, I'd like to register with you". If that gets a "Sorry mate, I'm full" response, it tries the next nearest until it gets registered. So there's a good chance you'll never fail to publish your server.

But wait! What if one of those new distributed "central" servers needs to go offline? Well, in the same way you currently unregister when you take a server down, a "central" server needs to do two things:

Then any server that was registered with that central server can quickly find an alternative.

And, I skipped this above, when requesting the server list, that new field, "I am a central server", does something else -- it triggers a recursive search. The server list will go to that central server and request all its servers (and if any of them are saying "I am a central server", recurse).

mawess commented 4 years ago

just to get this from a user viewpoint: what is the user experience you want? my suggestion is this: i want to se a list of all public servers over the world listed in order of delay. I do NOT want to connect to different central servers do see different groups of servers. .

pljones commented 4 years ago

And my suggestion does just this. If you pick "default", you'd get everything anywhere. If you pick "manual" it explicitly means you know what you're doing.

corrados commented 4 years ago

more "peer to peer" or "tree" idea for publishing server details.

Hmm... Much too complicated. Think of Corona as a temporary thing. After Corona the number of registered servers will go down significantly.

I also though about how to improve the server list. I will try out to send the server list in small chunks. I'll work on this on the weekend and try some things out.

dszgit commented 4 years ago

Unfortunately, "After Corona" is an aspirational phrase that has no definite meaning right now. All indications are that for at least the next year or two, there will be recurring periods of social distancing at various times throughout the world. And it seems likely there will be similar occurences with new pathogens in the future. I'm not weighing in on the proposal at hand, but I think demand for Jamulus will persist.

corrados commented 4 years ago

Ok, that may or may not be the case. Anyway, we need a solution to the server list issues. As I wrote, this weekend I will look at the problem.

corrados commented 4 years ago

Here is an interesting finding: https://sourceforge.net/p/llcon/discussion/533517/thread/0e9aa52428/?page=1&limit=25#51b4

gilgongo commented 4 years ago

I think this issue is now addressed to an extent it can be closed?

corrados commented 4 years ago

Yes, I agree. This should be closed now -> done.