search

mumble-voip / mumble

Mumble is an open-source, low-latency, high quality voice chat software.
https://www.mumble.info
Other
6.28k stars 1.11k forks source link

Update GeoIP database #4184

Closed peylight closed 3 years ago

peylight commented 4 years ago

Hi mumble.4gamer.ir form Unknown country server list is from Iran Alt text

Krzmbrzl commented 4 years ago

And? What are we supposed to do with this information?

For this issue you should probably contact the admin of that server... Or is this actually a bug report? In that case it'd be important to know which server version you are using. Are there other servers from Iran that show up labeled properly in the server list or are all servers from Iran listed as "Unknown country"? Furthermore it'd probably be important to know where this particular server is hosted from...

peylight commented 4 years ago

@Krzmbrzl I am one of admins of this server. This is server info: Alt text Do you mean this issue is about server config? or it is about GeoIP of client or mumble center server?

toby63 commented 4 years ago

Well as I don't see an option to set the country in murmur.ini (I might be wrong though), I think this is a missing feature on mumble side. Edit: I was wrong, see below.

I guess the provider for the public server list will sort the servers based on their location or url into continents and countrys and for iran there is maybe no entry yet, thus it is packed into unknown.

I checked the IP, it is in Iran. And the URL is also the country domain for Iran. There are no other public servers listed for Iran in Mumble.

Edit: Just in case, the ping log:

PING mumble.4gamer.ir (45.156.185.224) 56(84) bytes of data.
64 bytes from hosted-by.parsvds.com (45.156.185.224): icmp_seq=1 ttl=53 time=90.3 ms
toby63 commented 4 years ago

I guess the main question is: What software does the sorting?

Update: There also seem to be more problems with the public server list.

I just wanted to check something and noticed that a german server is listed in the category for "India":

PING mumble.anoxinon.de(argos.anoxinon.de (2a01:4f8:c2c:6b0a::1)) 56 data bytes
64 bytes from argos.anoxinon.de (2a01:4f8:c2c:6b0a::1): icmp_seq=1 ttl=55 time=13.9 ms

The domain is german and the ip is also from germany.

I looked at: https://github.com/mumble-voip/mumble/blob/master/src/mumble/ConnectDialog.cpp and found:

    qsCountry = pi.qsCountry;
    qsCountryCode = pi.qsCountryCode;
    qsContinentCode = pi.qsContinentCode;

So there seems to be an allocation with codes for country and continent.

Also this is very interesting:

// The continent code is empty when the server's IP address is not in the GeoIP database
        qmContinentNames.insert(QLatin1String(""), tr("Unknown"));
peylight commented 4 years ago

@toby63 Where is the GeoIP database?

toby63 commented 4 years ago

Whoops :sweat_smile:

I made a mistake, there is actually a config for the location in murmur.ini: registerLocation=

So @peylight can you check the murmur.ini of your server whether that is set?

It should be set to: registerLocation=IR

; Location is typically the country of typical users of the server, in ; two-letter TLD style (ISO 3166-1 alpha-2 country code)

https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2

Nonetheless I wonder if thats the only "check" for location :thinking:

peylight commented 4 years ago

@toby63 hmmm... So the problem was that. The default config file have no registerLocation option! Alt text

toby63 commented 4 years ago

Glad we solved it :smiley: .

So the problem was that. The default config file have no registerLocation option!

That really seems to be a problem. Can you tell us what Distribution your server is running on?

I just checked the Debian package for example and it also does not include the Location option :thinking: Time for some bug reports :smile: .

Update: It was already introduced on 3 Dec 2010: https://github.com/mumble-voip/mumble/commit/e8a2bb5

So I guess some distros made a mistake by taking it out.

peylight commented 4 years ago

And Static Linux Server have the same issue.

toby63 commented 4 years ago

So you use the static linux server?

Wiki also does not include it. Well something to do :smile:.

And by the way thank you for reporting :+1:, it wouldn't have been noticed, though it must bother some other users and servers as well :thinking:.

Todo-List: Note: would be useful to be added in first post, so it's more visible.

Update: not necessary because the problem is upstream. - [ ] check murmur.ini in all packages and builds provided by mumble

Update: not necessary because the problem is upstream. - [ ] check & contact Distros (about including the newest murmur.ini)

Suggestions for the future:

peylight commented 4 years ago

Yes we are using Static Linux Server. And should change Static macOS Server too.

Krzmbrzl commented 4 years ago

@toby63 good job finding this option. I'll add it to the wiki ASAP :)

toby63 commented 4 years ago

And should change Static macOS Server too.

Ok, added a general check for all builds and packages provided by mumble to the todo list.

peylight commented 4 years ago

Suggestions for the future:

* maybe build in an automatic option to check the URL/Domain of the Server.

We have many .com domains. so i think it is better to use GeoIP database in public server to locate the exactly location of server. Maybe it is already exist and we just need to update it or something like auto update for GeoIP database?

toby63 commented 4 years ago

@Krzmbrzl

By the way I can do the wiki stuff; thats at least something I can do :wink:. I can also contact some of the distros about it.

But you would sometime need to change the builds and packages that mumble is providing directly.

@peylight:

We have many .com domains. so i think it is better to use GeoIP database in public server to locate the exactly location of server.

My idea would be to have a priority:

  1. If it is set manually than the option is used
  2. If the option is not set, mumble might use the domain instead.
peylight commented 4 years ago

We have many .com domains. so i think it is better to use GeoIP database in public server to locate the exactly location of server.

My idea would be to have a priority:

1. If it is set manually than the option is used

2. If the option is not set, mumble might use the domain instead.

Yes and i agree with your idea. But i think it should be already like this idea. Because we have many servers and all of them working great with no problem. So maybe we just need to check GeoIP database. Maybe the database is from a long time ago...

toby63 commented 4 years ago

Because we have many servers and all of them working great with no problem. So maybe we just need to check GeoIP database. Maybe the database is [from] a long time ago...

Interesting :thinking:. Well the devs will have to check and answer that, I already put it in the todo list. (For understanding: the code is partly very old and the recent devs can not know every part of it; also in this case it is possible or even very likely that a different software or service is involved that would check for the GeoIP) That said I agree with you, there seems to be a Geolocation service involved and it partly does not work.

@Krzmbrzl I added a wiki entry for it: (Note for others: For now the change is only visible to members) https://wiki.mumble.info/wiki/Murmur.ini#registerLocation Still we (or better you the devs) could check sometime how it is working in detail, whether there is also a GeoLocation check and if there is, how the priorities work. I found these older commits, that are related: https://github.com/mumble-voip/mumble/commit/e8a2bb5 https://github.com/mumble-voip/mumble/commit/f065c2b

Krzmbrzl commented 4 years ago

Regarding the ToDo-list: Afaik our builds are using the ini file from the source dir (the one you found the option in) and I don't think that the static server comes with a ini file to begin with. I think that's just the executable and the ini file you have to create yourself :thinking:

And about the other packages: Fell free to contact the maintainers, but I don't think that's a big priority. After all the geo-location isn't that big of a deal if it's not set up properly...

toby63 commented 4 years ago

Just sayin: this is of course not urgent, but it should be investigated sometime. Also because there might be other outdated stuff in murmur.ini and other things.

On Topic:

and I don't think that the static server comes with a ini file to begin with. I think that's just the executable and the ini file you have to create yourself thinking

The user said it is and I just tested it myself: I downloaded the "static linux server" from https://www.mumble.info/downloads/ and it includes a murmur.ini and it does not include "registerLocation". So I guess someone somewhere is using an outdated or different version of murmur.ini for the "ermine"-build as kissaki called it.

The same will apply to the "static MacOS Server" as reported by @peylight.

After all the geo-location isn't that big of a deal if it's not set up properly...

Well, servers want to be found.

Krzmbrzl commented 4 years ago

Well, servers want to be found.

Which is why admins can now refer to the wiki ;)

Okay I'll check on the static server thingy and ask what kind of ini file we're using for that :point_up:

toby63 commented 4 years ago

Just found the root of the problem. You (mumble team) provided the different version of murmur.ini in the 1.3.x branch, see: https://github.com/mumble-voip/mumble/blob/1.3.x/scripts/murmur.ini

(Of course also a problem in 1.3.1-rc1: https://github.com/mumble-voip/mumble/blob/1.3.1-rc1/scripts/murmur.ini)

So probably no variant of mumble-server in any distribution and repo right now will include the "complete" murmur.ini.

I created a side-by-side diff (Note: the diff is a bit cut (the end of lines are missing), so it's just to show the difference, not for use): https://gist.github.com/toby63/ab4084c860681c14c985b184fa9f6c37

Krzmbrzl commented 4 years ago

Ha! Yeah that makes sense then :facepalm: Then I suppose it could be a simple matter of backporting the newer ini file to the 1.3.x branch (though some care has to be taken in order to not backport options that don't exist in 1.3.x) :+1:

Krzmbrzl commented 4 years ago

As #4198 ha sbeen merged I think this issue can be closed. Let me know if you think otherwise.

toby63 commented 4 years ago

Well...you will see this as nitpicking, but: There is still the open question, whether there is a sorting mechanism (IP or Domain check) implemented, besides the config option. Because strictly seen, if there is a mechanism, then it is flawed and the config option is only a workaround.

An important detail: I already found this in https://github.com/mumble-voip/mumble/blob/master/src/mumble/ConnectDialog.cpp : // The continent code is empty when the server's IP address is not in the GeoIP database This indicates that there is (or was) a GeoIP Service.

And I assume that the ip or domain check would be implemented in a service you run, to provide the public server lists. So it is probably not implemented in mumble. But maybe the sentence is only missleading or outdated.

Krzmbrzl commented 4 years ago

I just skimmed through the code again and to my understanding the country code gets set here: https://github.com/mumble-voip/mumble/blob/206a22f53fcc243dd61880ba29836e44b2702a09/src/mumble/ConnectDialog.cpp#L1796 This means that the country code is not determined by Mumble but simply read out from the server information that Mumble is being sent. Thus if you configured your server correctly, you should be fine. I don't see any other mechanism here. No idea what that comment is referring to.

toby63 commented 4 years ago

No offence, but at least someone in the team should know that for sure, right? Because if nothing is in the mumble code itself, it must be a service the mumble-team runs (to be correct, you probably don't run a GeoIP-check yourself, but you run the server that provides the public server lists).

To be clear: I can only rely on what @peylight told me, that they run more servers and that those are sorted correctly. It seems without the mentioned option, so some different mechanism has to sort it.

But i think it should be already like this idea. Because we have many servers and all of them working great with no problem. So maybe we just need to check GeoIP database. Maybe the database is from a long time ago...

Source: https://github.com/mumble-voip/mumble/issues/4184#issuecomment-631450514

@peylight Could you check whether your other servers have registerLocation set in their config? Also could you name them and in which country and continent they are listed (You of course don't have to :slightly_smiling_face:, if you for some reason (maybe privacy etc.) don't want it)?

peylight commented 3 years ago

@toby63 I'm talked to ZER0 that owner of [SGDN] S K Y N 3 T that located at United States mumble server list. He told me i have no option setting for registerLocation and it is automatically detected US based on my server IP. Alt text

davidebeatrici commented 3 years ago

The servers that are in Unknown have an IP address that is not part of the GeoIP database we have on our public list server.

peylight commented 3 years ago

@davidebeatrici Yes we know mumble have GeoIP database. The second request of this issue is about it. Please update GeoIP database.

Krzmbrzl commented 3 years ago

@davidebeatrici where do we even embed such database in our application? I always assumed that we were using some external Db and query it over the net or use something built into Qt... :eyes:

davidebeatrici commented 3 years ago

It's not embedded in the application, it's on our server (which the client queries).

toby63 commented 3 years ago

It's not embedded in the application, it's on our server (which the client queries).

Just as I thought :wink:. Thx @davidebeatrici for viewing at this (slightly old) topic and explaining :+1:.

May I ask what software is used for that?

davidebeatrici commented 3 years ago

It's a Perl script that uses the API provided by MaxMind: https://metacpan.org/release/Geo-IP

We have to switch to GeoIP2: https://metacpan.org/release/GeoIP2

davidebeatrici commented 3 years ago

Done!