search

kanersps / essentialmode

A FiveM project
https://docs.kanersps.pw/docs/essentialmode
GNU Affero General Public License v3.0
82 stars 98 forks source link

Fails to decode json / Duplicate Users #95

Closed rdelian closed 4 years ago

rdelian commented 4 years ago

First time I thought it was because I used the same database for multiple servers, but I've bought the OneSync Infinity and the same thing happens, especially this one:

SCRIPT ERROR: @essentialmode/server/db.lua:239: bad argument #1 to 'decode' (string expected, got nil)
> callback (@essentialmode/server/db.lua:239)
> userCallback (@essentialmode/server/db.lua:85)
Please don't mind the NUMBER LINE Code its because i've added a few prints into the gamemode

SCRIPT ERROR: @essentialmode/server/db.lua:239: bad argument #1 to 'decode' (string expected, got nil) points to https://github.com/kanersps/essentialmode/blob/master/server/db.lua#L237

As I said on discord when the issue happens the err from requestDB it's also 0

I have provided more details on discord (but no one responded, yet) if you wish you can dm me at any time if you want me to debug that https://discordapp.com/channels/138265073490722817/505439192453611532/742442673377640458

Server config

add_ace resource.essentialmode command.sets allow
add_ace resource.essentialmode command.add_principal allow
add_ace resource.essentialmode command.add_ace allow
set es_couchdb_url "127.0.0.1"
set es_couchdb_port 5984
set es_couchdb_password "my_username:my_password"
set es_defaultDatabase 2

More Info

OS: "Debian GNU/Linux 10 (buster)"
Database: "CouchDB"
CouchDB_Version: "2.3.1"
EssentialMode_Version: "6.4.2"
Server_Build: "2802"
rdelian commented 4 years ago

About the decode problem

1. I have found out that if you add a delay (I used Wait(3000)) for each user in savePlayerMoney() there are a lot fewer errors.

2. I also found from my logs that all these calls runs in parallel due to asynchronous behavior of PerformHttpRequest (if I'm not wrong) and none of them gets a return until the last one gets called (mostly because there is not delayed as I said 1. and the database doesn't have to process & return a response).

After a few ms/sec all of them gets returned at the same time and some them throw the error. because rText it's nil (and err its 0)

My idea about the problem

I suspect that the problem it's because there are too many calls to the database? Because I was able to get a lot fewer errors with a delay but the problems still exist when I have ~70 players, for like ~50 it's fine

I still have no idea what cause the duplicate users, I don't know if it's related

kanersps commented 4 years ago

This is very odd. Because usually database requests don't really happen that often by EssentialMode itself. Only when a player leaves, joins or every few minutes.

Are you sure this is directly an issue related to EssentialMode?

rdelian commented 4 years ago

Well, 90% of the time the errors are starting from savePlayerMoney() but after I've added a delay for each user the errors didn't pop up so often so it can be that the database has too many requests pe X minute?

At this point I'm sure, but as I told you I even stopped the resource and nothing else used EssentialMode and the errors still appear.

EDIT: I will play a little bit with https://docs.couchdb.org/en/stable/maintenance/performance.html and I let you know if the problems still appear but please take into consideration what I said

rdelian commented 4 years ago

No luck...

Looks like nothing works, I tried to tweak CouchDB a little bit https://docs.couchdb.org/en/stable/maintenance/performance.html#max-dbs-open - set max_dbs_open to 1024 https://docs.couchdb.org/en/stable/maintenance/performance.html#disk-and-file-system-performance ERL_FLAGS its set by default to "+A 16"

I also downloaded the last commit just to be sure, stopped the resource that uses the essentialmode, and leave the essentialmode alone

SCRIPT ERROR: @essentialmode/server/db.lua:237: bad argument #1 to 'decode' (string expected, got nil)
> callback (@essentialmode/server/db.lua:237)
> userCallback (@essentialmode/server/db.lua:85)
SCRIPT ERROR: @essentialmode/server/db.lua:237: bad argument #1 to 'decode' (string expected, got nil)
> callback (@essentialmode/server/db.lua:237)
> userCallback (@essentialmode/server/db.lua:85)
-- + same 5 more

Server Log

I even changed to an older server build like 2781 (and I use onesync infinity if that helps) https://pastebin.com/raw/KhpLi6tJ

rdelian commented 4 years ago

Fixed but not really

Somehow the essentialmode table size was 10.5MB for 1500 docs, I exported and reimported using a tool - it was fine and the size was 500kb for the same amount of data

But after a while, I checked the database again and from 1845 docs at 500kb now I have 1866 docs at 3.0MB

The tool: https://github.com/danielebailo/couchdb-dump

rdelian commented 4 years ago

Before using the tool

before

After I used it

after

After 1 hour

after-1h

json.decode error, again

There are also a lot of duplicated users and after a while, I still get the same errors (with 40-60 players) even after the re-import process

kanersps commented 4 years ago

Database size itself shouldn't be directly related to it.

Also sorry for the long response times, I've been fairly busy. Due to that almost no one uses CouchDB at all (max 2-3 servers). It's not really that supported anymore. Do you face the same issues with SQLite?

So I've brainstormed what the issue could be but I haven't been able to decipher it.

rdelian commented 4 years ago

No worries, meanwhile I tried a bunch of things.. but no luck so for, I just manage to reduce them a little bit.

I didn't try SQLite because I have a quite large project that uses CouchDB and I don't really know what do I have to change regarding the code, didn't even try it because I guess this one will be the most time-consuming plan

Now my last shot it's to change a CouchDB setting related to process timeout from 5s to something else.

Unfortunately, I don't really know when I can try it because my server it's under a DDoS attack at this time - after all these errors and user-reports, that's the last thing I want to have 😃

Thanks for the response anyway, at least I know in which direction to go if that's not working

rdelian commented 4 years ago

Hey, a little update on the issue

I've tried everything to config the CouchDB but nothing helps, it only reduced them to a few but I still have errors So after all these years using ES & CouchDB I decided to move to SQL since I have a lot of players playing something else than rp.

Also, you're the guy who helped me start doing scripts for fivem with Basic UI so keep the good work going 😃

kanersps commented 4 years ago

I'm sorry for the late replies, my college started again, anyway let me try to respond:

I think that it's a good idea that you moved to SQL, I've got statistics as to how many people use CouchDB and it was always less than 5 out of the thousands of servers.

That's also the reason why I officially stopped supporting CouchDB, but I still applaud your attempt! I think it's awesome that you tried to solve it, I'm sorry that I couldn't be of more help though.

Friendly regards,

Kane