search

PrismarineJS / mineflayer-navigate

mineflayer plugin which gives bots a high level 3d navigating API using A*
59 stars 22 forks source link

findPath async version #2

Open andrewrk opened 11 years ago

andrewrk commented 11 years ago

that way we don't hog the CPU while pathfinding

andrewrk commented 11 years ago

child process fork won't have access to the parent process's memory. But we can still provide an async version.

yocontra commented 11 years ago

Why not use something like asyncjs for this? If you have tests on the pathfinding code I'd be glad to lend a hand

andrewrk commented 11 years ago

Are you talking about this? https://npmjs.org/package/asyncjs

The problem is not how to organize callbacks; it's more fundamental than that. The problem is that we need to run the CPU for a long time, and we have no threads.

yocontra commented 11 years ago

asyncjs lets you do loops without "hogging" the CPU - it puts every iteration in a nextTick so other things have a chance at the CPU (right now it will block it for who knows how long). If you want to just throw it in a thread and forget about it you can always use threads-a-go-go or write the A* stuff as a native module and use libuv threads (that's what I would recommend)

andrewrk commented 11 years ago

neither of these libraries bring anything new to the table; we're left with the same problem that a child process leaves us. the problem is that what's taking a long time is searching memory. the child process, or the thread, whichever you decide to use, still has to communicate with the main thread via IPC or events/messages for block data.

it's not to say that we can't provide an async version. The question is whether to accept the overhead of starting a thread or process or not. the simplest implementation is to call process.nextTick every once in a while to let other events run.

yocontra commented 11 years ago

So why not write the A* stuff as a native module and use the libuv thread pool to do the CPU intensive work?

andrewrk commented 11 years ago

Because the problem is not just a bunch of number-crunching. it's a problem of memory sharing. We could do a native A* module, but then we'd have to store the block data in a different way so that both threads could get access to it.

You're welcome to try to tackle the problem and submit a pull request. If it's clean, maintainable, and solves the problem, I'll merge it. However I think if you do you'll discover it's a bit more tricky than you think.

Here is one new idea I haven't considered until now though: When mineflayer starts it could start a child process and when mineflayer gets new data that the subprocess needs, it will send it via IPC. So the subprocess keeps a copy of mineflayer's state; this will take up twice as much memory, but there will be a free process ready to do navigation or whatever. And when actually called upon, you won't waste all that time initializing a process and sending data back and forth; it will already have the data cached.

No matter how you choose to solve it, it gets messy.

Darthfett commented 11 years ago

I think this is approaching the subject that you, Josh, and I discussed a bit, of having a shared block memory for bots running on the same machine.

yocontra commented 11 years ago

Native modules have access to the same memory though - it's the same process.

andrewrk commented 11 years ago

It might be possible to create an addon, make it be in charge of the raw chunk buffers, and then be able to spawn threads to do things with the chunk buffers.