Sector loading problem

[Jimx-]

@Nebual I notice that when I start game and connect to a server, the sector_packet stays 0 for about 20~30 seconds and after such a long time, it increases rapidly to 1000+ and the world is loaded in few seconds. I've tried it on several computers but the same problem happened. change_sectors is called as soon as the game connect to the server and begin to request sectors. So is this a server-side problem or something else？

[ronmurphy]

I have noticed that also, it is "Lag" for lack of a better word, and it reduces me from just walking around to taking a few steps, waiting for the world to catch up, repeat over...

[travcunn]

The problem is in the client. In between receiving sectors from the server, the client is trying to render the blocks on the screen. Instead, the client should request the sectors, then receive them and put them into a queue. Simultaneously, the queue should be processed. This should get rid of some lag as well as speed up getting all of the sectors from the server.

It looks like a queuing system has been started in world.py but has yet to be finished.

[Nebual]

travcunn: I believe sector loading is fully queued. Heres the flow, after the world has fully loaded around the player:

1. Player moves.
2. (in update_sector): When the player crosses into a new chunk, run change_sectors, which attempts to find the new chunks we now need and request them from the server. I'm now noticing change_sectors is extremely laggy, and disabling it (after the world has initially loaded) makes my FPS perfectly smooth.
3. Whenever the server's main thread isn't doing anything else, it processes received requests from the clients (such as to send a new chunk to them).
4. The client's packetreceiver thread constantly listens for packets and puts them in a queue (world.sector_packets).
5. Every 1/60th of a second, the client runs world.process_queue, which attempts to process as much of all the queues as possible, stopping once it notices its processing has taken more than 0.3/60 of a second (30% of a frame), to avoid hurting FPS.
6. client.dequeue_packet processes packets, including adding sectors to the world and displaying blocks if the server reports they're visible. The client does not do visibility checks on incoming sectors, thats precalc'd on the server, though it does do them when adding/removing blocks (since the alternative would look laggy).

Under this system, theres two reasons why lag might occur:

1. The task is not in a queue, and takes a long time to complete. Solution: Queue it.
2. A single element of the queue takes longer than 0.3/60 of a second to process. Solution: Split the element into smaller parts. If processing a single 8x8x8 sector of blocks is taking too long on some computers, we'll need to rework how sectors are loaded...

Personally: If I go to Multiplayer -> Launch Server, then connect to it, I start receiving packets about 0.5 seconds after the main menu disappears (2 seconds after I click Connect to Server). If I don't move to a new sector, I'm perfectly lagless while it loads sectors. 11 seconds later the world is fully loaded and my packet queue is empty. If I disable change_sectors then, there is no lag as I run around. Is anyone else experiencing something drastically different?

[travcunn]

I found something interesting about your 5th point. Even though process_queue in world.py stops when it hits the maximum time it is allowed to execute, it seems like something in there is causing the frame rate to suffer. If you take away the while loop that checks if it hits the maximum time limit, you will see your framerate increase, but the rate at which sectors are received and render will suffer.

I'm still looking into it.

[r58Playz]

Personally for me, the world starts loading as soon as the main menu disappears, and it's super laggy until the world fully loads. Then it's laggy when I press W, S, A, D, or space. When I try to break a block, it does nothing. If I try to break the same block again, the server gives me a KeyError.

It's weird.

EDIT That's on Singleplayer.

[ronmurphy]

you do realize that this project has not been updated or touched in YEARS, and as fast as the linux world progresses, updating the whole project to current standard to take advantage of updated apps and such is a huge undertaking...

Feel free to start debugging. just keep in mind, most of these files are 5-6 years old, and that python and such has changed. also keep in mind, this uses python 2, not 3.

[r58Playz]

On multiplayer no problem, but I can't remove blocks. #89

[Nebual]

I may have some fixes for the originally reported issue, which is that the world seems to take a long time to initially load, opening a PR shortly.

There's a separate issue that running "Singleplayer" (which launches the server in the same process, in a separate thread) is considerably laggier (FPS wise) than launching python server.py separately and connecting to it. It feels as if the GIL isn't being properly released back to the GUI thread while the server is generating/loading a chunk. I may investigate subprocesses for that...

89 is a separate issue regarding Pyglet updating their API over the years...