[Bug] Error after updating RIRData

lightlike commented 3 years ago

System Data

Nextcloud: 21.0.1 (linuxserver-io Docker) PHP: 7.4.15 SQL: MySQL 10.4.18

Expected Behaviour

Nextcloud should detect my IP as a "DE" IP and not throw any warnings.

Actual Behaviour

Geoblocker recognizes my IP as "AA" and shows this warning every fiew seconds:

[geoblocker] Warning: The user "***" attempt to login with IP address "***" from blocked country "AA". No reaction is activated.

GET /index.php/apps/passwords/api/1.0/session/keepalive
from *** at 2021-05-07T19:51:13+00:00

IP as *** is correct and recognizes as "DE" from other websides.

More Information

this happened after updating yesterday (data was a fiew months old before that)
currently trying to get someone to login from other IP. Will update when I get more information.

HomeITAdmin commented 3 years ago

Do I understand it correctly, that before the update, everything was working as expected? Is it an IPv4 or an IPv6 address? There is a know problem with IPv6 addresses on less then 64 bit systems (#69). But this wouldn't fit if it was working before. What is the reported status of the service on the settings page? Is it '"RIR Data": OK.' ?

lightlike commented 3 years ago

IPv4, 64x, but in docker

RIRData Ignore the visual glitch with the dropdown.

I saw the other issue but that did not really fit as that is IPv6 and less than 64x.

Maybe some error with the database entries? I could try to recreate or compare an old backup but that has to wait a fiew days.

HomeITAdmin commented 3 years ago

Yes, I think the best guess at the moment is, that everything seemed to be ok, but maybe the downloaded lists were not complete so something is missing in the DB. It would be interesting to know, how many entries you have in the "oc_geoblocker_ls_rir" table overall and splited with "version=0" and "version=1". Do you know, how to read this? If the root cause was just some temporarily connection issue, then you could solve it by just updating again, or did you already tried this?

lightlike commented 3 years ago

Already did a another update one day after the first and that did not help.

The database has 152887 rows for geoblocker. Everything is version=0.

Data as Json here: oc_geoblocker_ls_rir.zip

lightlike commented 3 years ago

I had some problems in the past when I deleted the RIR database. That was in issue #47. Could that be the problem?

HomeITAdmin commented 3 years ago

I just tested an update and have similarly amount of entries. In an version end of April it was 100.000 more. That is a hint. I have to investigate this more. Could you send me the IP that should be DE but is not correctly detected?

lightlike commented 3 years ago

Send those to you via E-Mail. Don't want those public in here.

HomeITAdmin commented 3 years ago

Thanks. Ok, what I can say so far is, that when reading the files from the rir services I get a EOF in the middle of the file. Because some entries are already read, this is counted as ok and everything goes on as if everything was read. Do not know yet, why this is happening. Maybe it has to do with some changes in the memory management of the docker image.

First step that I can already do is detecting the problem. At the beginning of the file the number of expected entries is mentioned. If it does not fit, I go out with an error.

Second step is, if there were valid data in the database before, I shouldn't go to a complete error state but keep on using the old data and just inform about the problem.

Nevertheless I keep on investigating how to overcome the problem.

HomeITAdmin commented 3 years ago

I still do not know, why it is happening, but it is working correctly, if I download the file first to disk and the process the entries.

lightlike commented 3 years ago

Is there anything I have to look out for, when clearing the rir-table? I'll try some cleanup and reloading the data to try and fix this. Or I'll restore a backup for that table.

And thanks for looking into this.

HomeITAdmin commented 3 years ago

It should be working out of the box be just hitting the "update" button.

HomeITAdmin / nextcloud_geoblocker