search

serge-web / serge

Serious Gaming, Evolved - web interface
https://sites.google.com/deepbluec.com/serge/
Apache License 2.0
14 stars 4 forks source link

2922_Message_response_slowly_when_save_wargam #2927

Closed lilitkarapetyan closed 3 months ago

lilitkarapetyan commented 4 months ago

🧰 Issue

🚀 Overview:

🔗 Link to preview

🤔 Reason:

🔨Work carried out:

🖥️ Screenshot

Confirmations

📝 Developer Notes:

lilitkarapetyan commented 4 months ago

Hi @IanMayo , Our CouchDB server is slow because we make too many API calls. For example, when we run the populateWargameList function, it takes a long time because it sends a new API call to the server for each wargame - https://gyazo.com/d7c1ec6083db42c04009a7d732f1d335.

I tested putting the /wargameList API call on CouchDB, sending all the wargame lists at once in a single API call, and it significantly improved speed. https://gyazo.com/bd0d5ddf3cc3c364fe5799848bfbb111 - https://gyazo.com/9c88278eb784e894a8be15aa7ef8736c

I've included sample code in this PR for you to review. Please Note that it only works with CouchDB Server

Tristina1788 commented 4 months ago

Checking on https://serge-inet.herokuapp.com/, I see the response is still slow.

Tristina1788 commented 4 months ago

@lilitkarapetyan , @IanMayo Testing on https://serge-inet.herokuapp.com/ : I still don't see the improve on speed. So that, the action look messy.

https://github.com/serge-web/serge/assets/107697044/e749b604-76d7-4087-a5e0-d070025d0946

IanMayo commented 4 months ago

@lilitkarapetyan , @IanMayo Testing on https://serge-inet.herokuapp.com/ : I still don't see the improve on speed. So that, the action look messy.

Yes @Tristina1788 - that's a serious delay. I tend to focus on the player experience more than the game designer, but that delay makes it almost impossible to use.

Tristina1788 commented 4 months ago

Testing on https://serge-2922-message-resp-svqtsm.herokuapp.com/, I think it has a little improve but it's still slow for playing

https://github.com/serge-web/serge/assets/107697044/acac7af3-9180-46ff-97a3-bbbb303a3860

IanMayo commented 4 months ago

Hey @Tristina1788 - the performance in this preview is certainly quicker now. What do you think?

lilitkarapetyan commented 4 months ago

Hello @IanMayo , I optimized all the bad api's, it made it faster, I think Given the size and complexity of the wargame document our provided, it's not surprising that the PUT and GET operation takes a relatively longer time. The document appears to contain nested structures, arrays, and various properties with different data types, which can contribute to the processing time in couchDB.

IanMayo commented 4 months ago

Hello @IanMayo , I optimized all the bad api's, it made it faster, I think Given the size and complexity of the wargame document our provided, it's not surprising that the PUT and GET operation takes a relatively longer time. The document appears to contain nested structures, arrays, and various properties with different data types, which can contribute to the processing time in couchDB.

Thanks for that @lilitkarapetyan - I'm really pleased we're discussing it. For one wargame, I see the wargame document is about 12Kb. Do you believe CouchDb is doing some processing of the document when it stores it? I thought it just saved the whole document. Hmm, PouchDb matches all CouchDb features, and we've been saving lots of 1Mb files in an instant (when we had lots of templates and assets in theP9 wargame last year).

lilitkarapetyan commented 4 months ago

Hello @IanMayo , I optimized all the bad api's, it made it faster, I think Given the size and complexity of the wargame document our provided, it's not surprising that the PUT and GET operation takes a relatively longer time. The document appears to contain nested structures, arrays, and various properties with different data types, which can contribute to the processing time in couchDB.

Thanks for that @lilitkarapetyan - I'm really pleased we're discussing it. For one wargame, I see the wargame document is about 12Kb. Do you believe CouchDb is doing some processing of the document when it stores it? I thought it just saved the whole document. Hmm, PouchDb matches all CouchDb features, and we've been saving lots of 1Mb files in an instant (when we had lots of templates and assets in theP9 wargame last year).

You're correct that CouchDB typically stores documents in their raw JSON format without much processing. It preserves the original structure, including nested objects and arrays. However, larger documents can indeed slow down network transfers and storage, as they require more data to send and store.

IanMayo commented 4 months ago

You're correct that CouchDB typically stores documents in their raw JSON format without much processing. It preserves the original structure, including nested objects and arrays. However, larger documents can indeed slow down network transfers and storage, as they require more data to send and store.

But, at 12Kb, these don't seem large in Internet scale of things. As I said, in P9 I think the wargame grew to 1Mb, and was still processed instantly - that's almost a hundred times bigger than what we're looking at here. I wonder if there is some other cause of the delay. Could it be the DigitalOcean hosting? Or maybe lots of calls we aren't aware of (maybe via sockets?).

The Network tab only shows calls to Heroku - it doesn't show calls from there to DigitalOcean.

lilitkarapetyan commented 4 months ago

You're correct that CouchDB typically stores documents in their raw JSON format without much processing. It preserves the original structure, including nested objects and arrays. However, larger documents can indeed slow down network transfers and storage, as they require more data to send and store.

But, at 12Kb, these don't seem large in Internet scale of things. As I said, in P9 I think the wargame grew to 1Mb, and was still processed instantly - that's almost a hundred times bigger than what we're looking at here. I wonder if there is some other cause of the delay. Could it be the DigitalOcean hosting? Or maybe lots of calls we aren't aware of (maybe via sockets?).

The Network tab only shows calls to Heroku - it doesn't show calls from there to DigitalOcean.

Was P9 wargame processing quikly on couchDB ?

IanMayo commented 4 months ago

Was P9 wargame processing quikly on couchDB ?

No, it was on PouchdB. But, the node server was effectively acting as a CouchDb server - thousands of large and small Serge documents being sent/retrieved etc. The performance delays were in hugely complex client-side Javascript algorithm code blocks - the network/database stuff was effectively instantaneous.

Tristina1788 commented 4 months ago

Hey @Tristina1788 - the performance in this preview is certainly quicker now. What do you think?

@IanMayo , @lilitkarapetyan It looks much better. But the saving force/ channel when I've created force/Channel still quite slow. It returns the default name after user updated. So it makes user confuse.

https://github.com/serge-web/serge/assets/107697044/038b9676-a586-4ac0-9723-d1281f0d4113

Tristina1788 commented 3 months ago

From admin page, when I open wargame 1, exit wargame 1 and open wargame 2. It shows the name of wargame 1 in about1 second then show name of wargame 2. Another saving (saving name game, saving new channel, saving new force) often slow about 1 second

https://github.com/serge-web/serge/assets/107697044/d277248c-b41d-443e-acb6-fc49f72f0f81

Tristina1788 commented 3 months ago

From admin page, when I open wargame 1, exit wargame 1 and open wargame 2. It shows the name of wargame 1 in about1 second then show name of wargame 2. Another saving (saving name game, saving new channel, saving new force) often slow about 1 second

slow-response.2.mp4

Testing on https://serge-2922-message-resp-svqtsm.herokuapp.com/, I think the performance is the same with yesterday.

IanMayo commented 3 months ago

Merging this, because it contains some useful performance improvements, though we still have some way to go.

lilitkarapetyan commented 2 months ago

Fixes #2922

Tristina1788 commented 2 months ago

Merging this, because it contains some useful performance improvements, though we still have some way to go.

@IanMayo @lilitkarapetyan Do we need to continue to improve it? We closed this PR and the ticket. If we still need to improve, I can create another ticket for it.