Pitching an idea: Full blown Box2D ecosystem

Lusito commented 3 years ago

Hi @flyover,

I recently started digging a bit more into typescript game development and wanted to get started by porting some of my existing java/libgdx projects to typescript, which use box2d and box2dlights.

After researching the various box2d implementations that support typescript, I ended up with your port (nice work!). However, I was still missing box2dlights, which doesn't seem to have a typescript or even javascript port yet.

While spending some time on porting it myself, I noticed the performance issues and forked your project to find the issue. This led me to read more into the code and getting distracted by the additions (controllers & particles) as well as the duplicated files (.js files still present in the "first pass at 2.4.0" commit).

Anyway, I ended up playing a bit with the code, restructuring it into subpackages, porting benchmarks, fixing performance issues, etc. and I'm currently at a point, where I'm committed to create a typescript ecosystem for box2d, where users can choose, which subpackages they want and which they don't.

Currently I have the following subpackages in mind:

benchmark: library, cli and webapp to compare different implementations of box2d.
controllers: The controllers part of your code
particles: The particles part of your code
core: The rest of your code, except for the testbed stuff
lights: 2D Lights for WebGL (currently WIP for WebGL 1).
testbed: A separate project in order to avoid dependencies between the other packages.
canvas-debug-renderer?: possibly extract this from testbed to simplify getting started.

Here you can find my current testbed showcasing the box2dlights port, including your existing tests:

Long story short: Would you be interested in a cooperation on this? Or should I just keep this a fork?

Elvynia commented 3 years ago

@Lusito I'm into typescript game development and I would really appreciate that ! Especially for box2dlights which I used with libgdx too x) I'd be happy to contribute too but I don't know what I would be able to do to help.

Lusito commented 3 years ago

It would be good to have multiple contributors, since the code-base is quite big. Just at the top of my mind I see a lot that needs to be done:

Further performance improvements (see below for the current state)
Improve garbage collection
Making the code more idiomatic to TypeScript
Keeping up with the official box2d code (c++ version).
Maintenance/bugfixes
Writing documentation and possibly tutorials

Current benchmark results on my machine:	Name	avg ms/frame	5th %ile	95th %ile
box2d.js	2.74	2	4	1.00
box2d-web	4.30	4	5	1.57
my fork of @flyover/box2d	9.85	10	9	3.60
box2d-html5	11.68	10	13	4.27
@flyover/box2d	18.51	16	21	6.77
plank.js	22.29	18	27	8.15

DanielHZhang commented 3 years ago

@Lusito I would love to help out with this idea. Currently I'm using my own fork of this repo and this implementation of visibility polygons for a basic lighting system.

Lusito commented 3 years ago

I've just gotten arround to publish the current state of my fork on github and reserve the npm org @box2d. Still haven't published anything on npm yet though.

Have fun taking a first look: https://github.com/lusito/box2d.ts

Also take a look at the different performance results depending on browser or node: https://github.com/Lusito/box2d.ts/tree/master/packages/benchmark#readme

Interesting how the results can vary so much.

Lusito commented 3 years ago

Testbed and online benchmarks are now hosted on github: https://lusito.github.io/box2d.ts Feel free to discuss the roadmap to 1.0 here: https://github.com/Lusito/box2d.ts/issues/1

flyover commented 3 years ago

Hey @Lusito, nice work! I've been trying to keep this port resembling the original source. I think as long as the source files and directory structure within the packages remain similar, changes could still be shared between the projects. I'm interested in hearing about any performance improvements you find. I'll revisit the change where the math classes use a typed array. At one point, there wasn't much of a performance hit, and it made it possible to use the position/velocity arrays from a particle system as a vertex buffer.

Lusito commented 3 years ago

I get that vertex buffers accept typed arrays, but having one typed array per vector doesn't really help, since you would either have to:

draw one vertex buffer per particle, which would result in a lot of GPU calls.
for each particle, update the same buffer at a different offset, which would still result in a lot of GPU calls.
or combine the typed arrays of all particles into one large typed array so you can draw your vertexbuffer in one go. But when you do that, you can just as easily do it with the old style.

Not a WebGL pro, maybe I'm missing something. Can you show an example where you use this?

Either way, I think that particles are not going to be used by all users and as such should not add such a huge performance hit to the rest of the code.

Keeping the directory structure similar is certainly a good idea. Though comparing the sources with the original c++ version is still quite difficult. Maybe if there was a way to automatically convert the c++ code into a more typescript-ish code (combine header and source into one file and making sure the attributes and functions are in the same order) would help to do diffs with the original and keep up to date. But I'm not aware of such a project.

flyover commented 3 years ago

I agree. The typed array vectors seem too much of a performance hit on slower hardware. I don't have my WebGL renderer checked in anywhere, but it's similar to the one in the LiquidFun project.

Fxlr8 commented 3 years ago

Here is the benchmark result on my 2,3 GHz 8-Core Intel Core i9 (firefox 82.0b)

Name	avg ms/frame	5th %ile	95th %ile	Ratio
box2d.js	4.80	6	4	1.00
box2d-web	7.63	9	9	1.59
box2d-html5	11.63	12	10	2.42
@box2d/core	11.76	12	10	2.45
@flyover/box2d	19.13	30	21	3.99
plank.js	52.06	45	24	10.85

Lusito commented 3 years ago

I'm assuming, this is the node.js benchmark? As you can see in the benchmark review, the results depend heavily on if you use node.js, chrome or firefox.

The only constant so far seems to be, that plank.js is way behind in performance.

jcyuan commented 3 years ago

very cool!

i7 4790 3.6GHz 8CPUs Chrome

jcyuan commented 3 years ago

i thank typo? plank.js -> planck.js

Lusito commented 3 years ago

You're right. Thanks

jcyuan commented 3 years ago

i have a question, why http://fatidol.com/phy-benchmark/ this test made by @finscn has the different result? box2d.ts wins. but not sure which version he used from flyover's repository.

Lusito commented 3 years ago

Maybe because that benchmark uses 3-4 year old engines? It's also testing things different I imagine. Aside from that, benchmarks are not perfect. Depending on your scenario, your performance may vary. I took a benchmark that was fairly easy to adapt. Feel free to add more tests that resemble the phy-benchmark(s).

jcyuan commented 3 years ago

SHA: a2ed91c3f36d4e7c681110ab7b7a5f307cfde857

@Lusito i believe this commit has the high performance result. (box2d 2.3)

the newest commit is with c++ box2d 2.4

hope you can find something from the high performance version.

SuperRobot998 commented 3 years ago

Browser: Chrome 88	Name	avg ms/frame	5th %ile	95th %ile
box2d.js	6.73	7.999999994353857	6.000000001222361	1.00
box2d-web	10.29	10.999999998603016	10.000000002037268	1.53
@box2d/core	18.75	19.00000000023283	14.000000002852175	2.79
box2d-html5	23.63	23.999999997613486	21.999999997206032	3.51
@flyover/box2d	31.60	34.9999999962165	29.999999998835847	4.70
planck.js	61.51	59.000000001105946	44.99999999825377	9.14

finscn commented 3 years ago

Browser: Chrome 88

Name avg ms/frame 5th %ile 95th %ile Ratio box2d.js 6.73 7.999999994353857 6.000000001222361 1.00 box2d-web 10.29 10.999999998603016 10.000000002037268 1.53 @box2d/core 18.75 19.00000000023283 14.000000002852175 2.79 box2d-html5 23.63 23.999999997613486 21.999999997206032 3.51 @flyover/box2d 31.60 34.9999999962165 29.999999998835847 4.70 planck.js 61.51 59.000000001105946 44.99999999825377 9.14

avg ms/frame ? it's as small as better ?

HypnosNova commented 3 years ago

It would be good to have multiple contributors, since the code-base is quite big. Just at the top of my mind I see a lot that needs to be done:

Further performance improvements (see below for the current state)

Improve garbage collection

Making the code more idiomatic to TypeScript

Keeping up with the official box2d code (c++ version).

Maintenance/bugfixes

Writing documentation and possibly tutorials

Current benchmark results on my machine:

Name avg ms/frame 5th %ile 95th %ile Ratio box2d.js 2.74 2 4 1.00 box2d-web 4.30 4 5 1.57 my fork of @flyover/box2d 9.85 10 9 3.60 box2d-html5 11.68 10 13 4.27 @flyover/box2d 18.51 16 21 6.77 plank.js 22.29 18 27 8.15

Excuse me, what does "5th %ile" and "95th %ile" mean? Is it mean in 5th times test and 95th times test? And what does ratio in this table mean? And does "ms/frame" mean fps or mean render one frame cost several ms?

And could you please give us a test demo link so we can also test on our machine to compare.

flyover / box2d.ts

Pitching an idea: Full blown Box2D ecosystem #74