brianzinn / react-babylonjs

React for Babylon 3D engine
https://brianzinn.github.io/react-babylonjs/
828 stars 105 forks source link

Declarative Entities #6

Closed konsumer closed 6 years ago

konsumer commented 6 years ago

I'm trying to make a simple character-chooser screen, with 3 characters (colored spheres, for now) and I can't seem to load them from imported definitions. I am trying to have all the characters/models in their own files, so I can just load the entities I need for a scene. I imagine that each component should track it's own position and visual state (like which animation to play, etc) via props, and the environment will set positions, lighting, terrain, etc. Here is an example, with just one:

App.js

import React from 'react'

import { Scene, FreeCamera, HemisphericLight } from 'react-babylonjs'
import { Vector3 } from 'babylonjs'

import PlayerFire from './player/PlayerFire'

const App = () => (
  <Scene id='character-choose'>
    <FreeCamera name='camera1' position={new Vector3(0, 5, -10)} target={Vector3.Zero()} />
    <HemisphericLight name='light1' intensity={0.7} direction={Vector3.Up()} />
    <PlayerFire position={Vector3(0, 1, 0)} />
  </Scene>
)

export default App

PlayerFire.js

import React from 'react'

import { Sphere, StandardMaterial } from 'react-babylonjs'
import { Vector3, Color3 } from 'babylonjs'

const PlayerFire = ({ position = Vector3.Zero() }) => (
  <Sphere name='PlayerFire' diameter={2} segments={16} position={position}>
    <StandardMaterial diffuseColor={Color3.Red()} specularColor={Color3.Black()} />
  </Sphere>
)

export default PlayerFire

I get this error and a blank screen:

no onSceneMount() or createCamera() defined.  Require camera declaration.

If I just do all the code inline like this, it works fine:

import React from 'react'

import { Scene, FreeCamera, HemisphericLight, Sphere, StandardMaterial } from 'react-babylonjs'
import { Vector3, Color3 } from 'babylonjs'

const App = () => (
  <Scene id='character-choose'>
    <FreeCamera name='camera1' position={new Vector3(0, 5, -10)} target={Vector3.Zero()} />
    <HemisphericLight name='light1' intensity={0.7} direction={Vector3.Up()} />
    <Sphere name='PlayerFire' diameter={2} segments={16} position={new Vector3(0, 1, 0)}>
      <StandardMaterial diffuseColor={Color3.Red()} specularColor={Color3.Black()} />
    </Sphere>
  </Scene>
)

export default App

Is context not working how I expect, or am I doing something else wrong?

konsumer commented 6 years ago

Ah, looking through source, it would seem that Scene doesn't setup a react context. Is there interest in this? It might be cool for cases like this to be able to get access to engine and current parent scene from context. I wonder if it would also make sense to have a parent engine provider, so people can do multiple scenes, for quick-swapping & multiple scenes at once (like here):

import Scene1 from './scenes/Scene1'
import Scene2 from './scenes/Scene2'
import Scene3 from './scenes/Scene3'

<Engine>
  <Scene1 visible />
  <Scene2 />
  <Scene3 />
</Engine>

If Engine is a Provider[engine], and Scene is a Consumer[engine] and Provider[scene] I think it would allow users to drill into anything with just JSX, and structure the tree however they like (as long as <Engine> is at top, and their stuff is in a <Scene>.)

I think it might help with other things, too, like making it easier to make custom components, for example I want a camera that uses different controls, so I can just do scene.activeCamera. attachControl(engine.canvas, myCustomControl) in some custom component, and it will basically work anywhere.

As a interim solution, I tried getting around it by passing it directly down:

App

import React, { Fragment } from 'react'

import { Scene, FreeCamera, HemisphericLight } from 'react-babylonjs'
import { Vector3 } from 'babylonjs'

import PlayerFire from './player/PlayerFire'

const App = () => (
  <Scene id='character-choose'>
    <FreeCamera name='camera1' position={new Vector3(0, 5, -10)} target={Vector3.Zero()} />
    <HemisphericLight name='light1' intensity={0.7} direction={Vector3.Up()} />
    {scene => (
      <Fragment>
        <PlayerFire position={Vector3(0, 1, 0)} scene={scene} />
      </Fragment>
    )}
  </Scene>
)

export default App

PlayerFire

import React from 'react'

import { Sphere, StandardMaterial } from 'react-babylonjs'
import { Color3 } from 'babylonjs'

const PlayerFire = ({ scene, ...props }) => (
  <Sphere name='PlayerFire' diameter={2} segments={16} scene={scene} {...props}>
    <StandardMaterial diffuseColor={Color3.Red()} specularColor={Color3.Black()} />
  </Sphere>
)

export default PlayerFire

But I got the same error, not sure why.

brianzinn commented 6 years ago

hi @konsumer - you have some great and valid points. This project originally started without any declarative components and also the context was 'experimental', but is now supported obviously. What I am passing around now with props would indeed be better on context. I am not a big fan of drilling props through components and like the new way of composing (producer/consumer). I am happy to accept any PR and work through you with it. I am not concerned with backwards compatibility as I feel there is lots of room for improvement, as you have found. If you want to get it going in a hacky way for now, look at the props that are handed to you by the scene (props.container.scene).

Also, I hadn't properly considered the multiple Scene as was trying to simplify by having the engine auto-created in the , but it does make sense to have the Engine component for that scenario and we can still auto-create if it's not the parent component. Keep in mind that the declarative components are fairly new and this project started out as just a wrapper to get the canvas. Let me know if you wanted to work on that (with me) or if you were just sharing a better way. Thanks.

konsumer commented 6 years ago

I'm happy to help. I started with a POC here, but it doesn't seem to be totally working. I probly missed a step (if you see anything, let me know.) I will take another look when I have time. Great tip with props.container.scene. I will work on a PR, next. I could try to keep it current API compatible by checking for engine in context, so if you don't make an <Engine />, it will make one for you, but I could see this leading to confusion (multiple engines and such.)

konsumer commented 6 years ago

I've been really digging the HOC pattern lately, especially with recompose, and it seems to make the context pretty simple to follow (just add withScene or withEngine.)

brianzinn commented 6 years ago

I got that POC going for you. The main thing you were missing was attaching the canvas. I added to your FreeCamera:

camera.attachControl(scene.getEngine().getRenderingCanvas());

That's the only way to get to it with only scene that I can think of.

setState({canvas})

Above is asynchronous, so you do not have state.canvas available in componentDidMount. There is an overload for setState() that has a callback on completion, which is possibly more appropriate. Also, you don't want to create the Engine in the render() loop of the React component - it can fire if the application is setting state. I have done some unsightly hacks with render() myself :) I am really seeing some potential with some of your ideas!

konsumer commented 6 years ago

Ah! The withEngine HOC (in Babylon.js) also provides canvas & engine. I updated the POC to use that instead, but good to know I could get it from scene, too.

If you are using recompose, you can do this:

export default compose(
  withEngine,
  withScene
)(FreeCamera)

I have been using this pattern to create re-usable logic/state for display-only components in react, lately. It's very fast and easy to follow. If you leave off the component ((FreeCamera) part), you get a reusable HOC.

It translates to the slightly less-readable:

export default withEngine(withScene(FreeCamera))

After trying your suggestion, it still doesn't work for me, though (see current.)

I did this:

camera.attachControl(canvas)

I also console.log'd to double-check that canvas is what I think it is. If you can make it work, feel free to fork the sandbox, and send me the link, and I will incorporate it.

I tried to dig in to react-babylon code, to make a PR for context, but got a bit overwhelmed (I'm not a big typescript guy, and got caught up on a few lil language-things, I think.) I will take another stab when I have a bit more time.

brianzinn commented 6 years ago

hi @konsumer. I like what you have done there with composing. There are two ways to get your latest working. Line 29 in FreeCamera, along with a sanity check target should be lockedTarget or setTarget(...):

if (target) {
    camera.lockedTarget = typeof target === 'string' ? scene.getMeshByName(target).position : pos
}

Without the above change, set your camera to {y=0}. With the above change, set y to +-5 and you will see the shadows changing, so you know that the camera is locked on target and centered. It's OK if you don't want to get into TypeScript - I totally get that. If we can get that example a bit more fleshed out and working with your own components working then it already passes a big hurdle and is a big improvement over the existing implementation. I'm happy to merge the changes in when I get some spare time. I'm working on some HMR stuff this week and then want to revisit here after. Let me know if there's anything else I can look at :)

brianzinn commented 6 years ago

Maybe I can start a new branch and we can see how that goes? Also, I'd like to give you credit for this, so not sure best way to do that :) I was thinking it would be useful to add properties to the HOC. ie:

const withEngine = (antialias, engineOptions, adaptToDeviceRatio) => Component => ...

Just looking at the default exports also and wondering if they should be using the HOC - thought we could use Context API directly and have purer components. I will look into that. Thanks.

konsumer commented 6 years ago

Yeah! that worked. Thanks for your help!

I'd really like to automate as much as possible, like run over the babylon typescript definitions, and just generate a bunch of react components, but maybe this is too ambitious. Without really looking at it, I feel like the cascaded babylon class interfaces might work well, in a few families of component-templates (Camera, Mesh, Light, etc) and make a few special containers, like Engine, Scene.

I think if we can get the render/context stuff in order, the HMR stuff might fall into place. I put my engine-maker in render, because I wasn't swapping props around <Babylon>, but I see how that's bad. In the past, I have used shouldComponentUpdate() to limit re-renders for canvas/context stuff, and it works pretty well. I think with more clever componentDidMount() and componentWillUnmount() it could better manage the current stuff in the engine/scene.

I will try to automate some of the work, and put together a more complete demo of these ideas over here.

You replied before I was done with this message, but a branch is also fine. Maybe we can do both, use mine as the "javascript version" while I prototype ideas, then your branch can be for the final typescript version. I like the idea of engine options, but maybe those should be props to <Engine />? the HOCs are meant to just make it easy to get a Context.Consumer in children, but I could see it working that way, too.

konsumer commented 6 years ago

I wrote a lil script that might be an ok start for auto-generating components. It's very dumb, and just tries to make a simple component for every exported class (so many should be removed), but I think it might help with ensuring all the props are in there.

brianzinn commented 6 years ago

that's very cool as well - despite have written many generators in the past the thought never occurred to me to use a syntax tree, although I should have clued in when I was hacking on TypeDoc... :)

konsumer commented 6 years ago

ts-simple-ast was the first thing I found to analyze typescript, that seemed at all comprehensible, but I feel like I could probly get 100% code generation for most things with smarter analysis, and it's a bit hard to figure out how to get stuff at different levels of the tree. Maybe I should take a look at TypeDoc.

The current output isn't great. For example, FreeCamera has a bunch of things as props that make no sense. I think it might be the right direction, though, like <Scene /> and <Engine /> set up the context (canvas, engine, scene) and the children components update their settings based on props, so you can put logic outside a component and have it refresh the props. I think I need to do more prototyping by hand to get the initial system worked out (balancing constructor, componentWillMount, componentWillUnmount, getDerivedStateFromProps, render, etc), then I can improve the code-generation based on that.

brianzinn commented 6 years ago

Just moving the talk from issue 2 back over here as it's not just reconciler related...

"it's easy enough to add export default <JSON_HERE> to the generator." I had tried that, but must be missing something, as our reconciler was not being triggered and instead the regular react reconciler was being triggered. I thought it was due to the import needing to be a string "FreeCamera" and not an object "FreeCamera": {...}, but I will look into that properly. ie: createInstance was being triggered on the wrong renderer?? I just got it working and there will definitely be a better way! :)

The HOCs with context are, in my opinion, what make this such a huge improvement from a developer experience perspective. What do you think of engineContext having even more functionality. ie: observables on the Engine like when a mesh is added - this is something the shadow generator needs (maybe I misunderstood ie: instead of withEngine, withEngineRender, withEngineRenderMesh, withEngineMesh, etc. -- leads to an explosion of methods). That's why I was considering only withEngine providing the list of observables? Another question is that if we are generating TypeScript - is that a place for object specific code - I am leaning towards not as it will be kind of hidden and then we can put helper classes, when needed, for custom functionality - this is a way to wire-in custom handlers or instantiation logic without messing with the generator. Also, I was thinking that even setXx(...) methods could be available as well with an object with a property for each argument. ie: <Mesh setProperty={arg1: 1, arg2: 2} />. That would open up the entire object API and we can do something like shallow-equal when we build the list of changes reconciler.prepareUpdate(...) -> reconciler.commitUpdate(...).

I'm going to move the 'fiber-hoc' branch to master and update the readme. I think your input has moved the direction into something more like a proper V1 beta release. Anyway, I just have a million ideas now - can't sleep!! :)

brianzinn commented 6 years ago

oh, I just saw your edits on issue 2! I think there is something going on with github "eventual consistency" taking ages. One thing to note is that BeforeRender can be at Scene level or Mesh level. At mesh level you can influence the mesh only rendering (ie: clipPlane). There is a good amount of design coming up to have a clean/stable API. My question is that if we have components like BeforeRender then nesting these types of components (and targets) perhaps becomes a challenge. Quite likely I've not made the mental shift to 'recompose' as you have. Sometimes these paradigm shifts take a bit to settle in with me. Cheers.

brianzinn commented 6 years ago

OK, so I sorted out the components.json and have updated code generation for "tags.json" to "tags.ts". I think that means that everything in your original examples + reconciler should be working with yarn build and npm link. I just pushed everything to master branch as this is obviously the way forward and really anything goes on a v0.x project! Now just to find more time :) I am going to work first on the reconciler to try to bring it closer to current feature parity (creation and prop update diff handlers).

konsumer commented 6 years ago

Ah, yeh, github is acting a bit crazy today.

I am going to work first on the reconciler to try to bring it closer to current feature parity (creation and prop update diff handlers).

That seems like a really good idea. I know it's not at all optimal. I will try to take a stab at adding some shadows & physics in typescript, if I have some time this week. It may take me some time to ramp-up. After we get some basic components worked out, it might also be fun to port some babylonjs demos to this, so people can see why it's a good way to assemble stuff. It might also help us work out bugs or missing features.

There is a good amount of design coming up to have a clean/stable API. My question is that if we have components like BeforeRender then nesting these types of components (and targets) perhaps becomes a challenge. Quite likely I've not made the mental shift to 'recompose' as you have. Sometimes these paradigm shifts take a bit to settle in with me. Cheers.

For me, I think it's ok to put it in just one space for simplicity, like if it's all at the scene-level, it doesn't really change anything, if you can get to the scene & engine (and individual meshes, via scene.getMeshByName, cameras, etc.) if you wrap a mesh with a <BeforeRender> it's at the scene-level, but it's only acting on it's children, so it can be used like it's mesh-level (via props.) I think the real trick is exposing enough props so you can use it at the scene level, and it will work ok (like there is still some work on materials, as they are currently not dynamic.) There doesn't seem to be a big cost to adding a ton of scene beforeRenders for every little thing. I think this is one of the strengths of having a graph of components, but also having access to everything in the current unit (mostly scene.) You can make lots of little things that manage themselves, instead of a big thing and manage everything, top-down.

My question is that if we have components like BeforeRender then nesting these types of components (and targets) perhaps becomes a challenge. Quite likely I've not made the mental shift to 'recompose' as you have. Sometimes these paradigm shifts take a bit to settle in with me.

Well, my favorite thing about recompose is that you don't have to use it at all to work in this way (like it's not a dependency of anything I did.) It's just basic chaining of HOCs, so you can use it however you like. You can build HOCs out of other HOCs easier with recompose, and they provide nice stuff like withState, withHandlers, lifecycle, etc, so you can keep it all functional (no classes, setState, etc) but it's not at all required. I think the value in nesting components under BeforeRender is that you can make a self-contained "this lil bit needs to update all it's children when the scene updates" sorts of components, but you could just as easily not use it that way, I think (I'd have to play with how it re-renders, but we could make it work if not.) I used the target style prop, because I wanted a prop that could random-access look things up, regardless of where it is in the scene. The same kind of thinking could be applied to actions or logic-management. Here is an example of a central manager that could just be dropped into your scene, and it would go and find all the stuff it needs to mess with:

const doStuff = scene => delta => {
    const camera = scene.activeCamera
    const player = scene.getMeshByName('player')
    // do stuff with player & camera here, no return needed
}

const LevelManager = ({ scene }) => ( <BeforeRender handler={doStuff(scene)} /> )

export default withScene(LevelManager)

I call this hoist-inversion, where you have a function that takes a param it needs and it returns another function that does the actual stuff. it's counter-intuitive that this is ok, but it really only calls it on mount, once, so it's not a huge deal, and the function has everything it needs at before-render-time.

It feels more react-ish, to me, to do it the other way (put the things that are controlled by the <BeforeRender> in it's children) but I don't think there's anything stopping you, if you prefer this style, and in situations where you have a ton of interdependent objects, I could see doing it once as being way more efficient (you don't have to lookup player in a billion sibling-component badguys).

I could also see totally self-contained components making up a scene, where all the children look up each other to figure out what to do, like characters in pacman:

const VisualPacman = ({ x, z, frame }) => (
  // draw a pacman, choosing animation with frame, and use x,z for position in scene
)

const pacmanAction = scene => delta => {
  // listen for input
  // check for collisions, especially with ghosts
  // pick frame based on delta
  // play "wokka" sounds, if applicable
  // etc
  return { x, z, frame }
}

export const Pacman = withScene(({ scene }) => {
  const Guy = withBeforeRender( pacmanAction(scene) )( VisualPacman )
  return <Guy />
}

const VisualGhost = ({ x, z, frame }) => (
  // draw a ghost, choosing animation with frame, and use x,z for position in scene
)

const ghostAction = scene => delta => {
  const pacman = scene.getMeshByName('pacman')
  // move towards/away from pacman
  // check for collisions, especially pacman
  // pick frame based on delta
  // play spooky ghost sounds, if applicable
  // etc
  return { x, z, frame }
}

export const Ghost = withScene(({ scene }) => {
  const Guy = withBeforeRender( ghostAction(scene) )( VisualGhost )
  return <Guy />
}

I think of a <FollowCamera /> being sort of like this, as you don't have to tell it where to be, you just say "go find the player and follow it around." so for a player, you could for example use physics to make them move around, and not have to worry about what the camera is doing.

This would also allow for very easy assembly of permutations of games (like react components speed up web development, because they contain their own behaviors, and you can just drop them in when you get them worked out.) I could even imagine following a standard (player is always player, enemies names start with enemy) and mix&matching games from your component library (the digdug guy + pacman ghosts + the background from joust = "our new mega-hit, jousty pacdug".)

Once you're inside the scene, you have access to engine/scene/etc, and you could just ram babylonjs code directly into the scene, in a componentDidMount, if you prefer. This would make it easy to have a mostly component-driven workflow, but just slam together a quick scene from any random demo-code to get it working. I could even see this being useful (even if the user is not a big fan of react) simply as a way to load all your scene stuff in one piece (like a "level" in a game) and have automatic tear-down, without even using anything other than <Engine /> and <Scene />. You could basically just use it as a gamestate-manager, and write your whole game like you normally would with babylon.

I could see all 3 of these workflow styles being pretty badass for different situations, and nothing really stops you from doing any of them, or mixing them, like use a ready-made code scene from another project or babylon demo in componentWIllMount, then add a couple pacman ghosts that are self-contained, and a <LevelManager /> that beams messages out to all your entities. It'd probly get stupid-messy quickly, to mix them all, but totally doable.

konsumer commented 6 years ago

realized that maybe the HOC thing wasn't totally clear (even though I typed a bunch, whew!)

A basic HOC looks like this:

const withCoolStuff = Component => props => {
   // mess with props or whatever
   const myProps = { cool: true }
   return <Component {...props}  {...myprops}  />
}

// usage
const CoolerComponent = withCoolStuff(YourComponent)

so if you wrap your thing, you get whatever it had before, prop-wise, but also cool. Because it adds and returns (functional style) you can chain & compose these. If you need other options, that don't come from props, you can do this:

const withMoreCoolStuff = options => Component => props => {
   // mess with props & options or whatever
   const myProps = { cool: true }
   return <Component {...props}  {...myprops}  />
}

// usage
const EvenCoolerComponent = withMoreCoolStuff(options)(YourComponent)

with recompose, you can quickly chain a few together, to make another HOC:

const withSuperCool = compose(
  withCoolStuff,
  withMoreCoolStuff(options)
)

// usage
const AwesomeComponent = withSuperCool(YourComponent)

But again, recompose just makes it easier to read and offers some nice HOCs to mix into your stuff. Without recompose, it looks like this, and does the same:

const withSuperCool = Component => props => {
  const Thing = withCoolStuff( withMoreCoolStuff(options)(Component) )
  return <Thing {...props} />
}

// usage
const AwesomeComponent = withSuperCool(YourComponent)
konsumer commented 6 years ago

I just updated the demo to include material color animation and smoother movement animation.

konsumer commented 6 years ago

I was thinking about other things that are kind of like shadows, and noticed that WaterMaterial might also need similar care (you add stuff to it's reflection list,) and realized the auto-generator isn't picking it up, so I might need to write that differently.

Also, I tried to follow the typescript, but it's very hard to read for me, so I might have to take more time learning it. I'm dyslexic, and the shape of the language is super-important. For right now, I think I'd rather just prototype ideas in javascript. I'm a lot more excited about react+babylonjs than typescript :) Maybe we should make a feature-list as we go, so we can keep reasonable feature-parity, and benefit from each other's development. Maybe gitter would be better for all the back & forth, not sure.

So far we've got this:

Some other things that need improvement:

see this for ideas on things that need to track other things that aren't children (physics, water, shadows, etc.)

There are probly tons more I am forgetting. Does this seem like an alright strategy?

brianzinn commented 6 years ago

oh yes, that looks like a great strategy! Also, agree maybe something like email/gitter would be good.

I don't know if you saw the updated AST code, I'm generating the code now that exports the types for fiber and now just JSON for 'components.json' (I imagine we will phase out JSON when code generation progresses). The createInstance is the same as the current SceneComponent abstract create(scene: Scene) : T; (we are going to throw that code all away now - I ended up abusing the ReactDOM.render() method to create exactly what is provided by fiber 'out of the box'). We will be able to maintain a full composite list in fiber render to manage everything. Also, instead of attaching properties to BabylonJS object, we can maintain metadata on our nodes - that's my current way of thinking. Even our own parent/child for walking the hierarchy and inspecting entire object graph.

So, react-reconciler createInstance will return something like:

type InstanceObject = {
  babylonJSObject: any,
  parent: InstanceObject | null,
  child: InstanceObject | null,
  metadata: InstanceMetadata
}

Then I think on our babylonJSObject (our real object like a mesh or light), we can link back to our InstanceObject.

I left the code generator in javascript!! :) You are as usual just full of such good ideas -- I don't even know where to start, because there are so many ideas :) Maybe we could make a project/milestone here in github with the features - at least that way we can keep track and prioritize everything? I can make you a collaborator as well, if you want. I think my first target is to get feature parity (so, I want to also generate code from 'babylonjs-gui') and then we will have people using these improvements from NPM and will start to get more feedback from the community (like here!).

One more comment as well is that I want to show in the examples how we can build an "Entity component system" with this paradigm. Like with your HOCs, we can just bolt on functionality to another component. I want to make some fun demos showcasing this as well, because it's nice to have this available declaratively.

konsumer commented 6 years ago

I don't know if you saw the updated AST code, I'm generating the code now that exports the types for fiber and now just JSON for 'components.json' (I imagine we will phase out JSON when code generation progresses).

Yep, I saw that. Very clever solution to the JSON issue, as it just uses the same typescript AST engine for output. My first generator responded to classes in an AST-walker, and seemed to get more stuff, the current one tries to find classes first, based on the JS export, then goes back over the code AST and finds the classes, and doesn't seem to get as many, so I will look at it it, and see if the first way is better, I think it might be.

Maybe we could make a project/milestone here in github with the features - at least that way we can keep track and prioritize everything?

Sounds good.

I can make you a collaborator as well, if you want.

Sure! I can do same on JS ideas repo, so we can both work on stuff faster.

Then I think on our babylonJSObject (our real object like a mesh or light), we can link back to our InstanceObject.

I think babylonJSObject already has parent & child, but I can see how this might be an easier type to track (any is only on babylonJSObject) I will follow this pattern in JS.

I think my first target is to get feature parity (so, I want to also generate code from 'babylonjs-gui')

Definitely. I will look at exporter this week, along with the shadows/physics stuff, and hopefully we should have feature-parity to the original system, as well as lots of new stuff!

One more comment as well is that I want to show in the examples how we can build an "Entity component system" with this paradigm. Like with your HOCs, we can just bolt on functionality to another component. I want to make some fun demos showcasing this as well, because it's nice to have this available declaratively.

Sounds good!

I think after that milestone, it might be a good time for me to focus on a bunch of demos too, as it will help show it off, but also help us find sneakier bugs and help us work out what kinds of react API people will need to do stuff that is in current bablyonjs demos. I think it can help make it easier to understand, too, because a user can just compare it to the original. Maybe a "getting started" article for typescript and ES6, and some auto-generated docs would also go a long way.

Have you used storybook? It's pretty awesome for making componentized demos, and might be a nice way to build & dev a bunch of nice demos. Here is a guide for using it with typescript. If we focus on making each story just show one thing, it can be a really good way to show off how to do stuff, somewhere between API docs and demos, without having to write a tutorial. I use the styleguide in webdev to actually develop, because you get hot-reloading and can quickly piece together a demo of just one thing. It might also be fine to setup a little sandbox. I can do some investigation on this.

Whichever way we go, I think the idea of a big menu with a ton of demos that show their own code is really useful. SDD is a major feature of thinking in components, and react helps so much with that, I think it'd be good to show it off that way. I'm a big fan of SDD + source docs, even over API docs (which I think we should also make, as our API will slightly differ from babylonjs, in things like lookAt), as I seem to be able to get to the gist of how stuff works faster.

I made a codesandbox team for us to put up ideas initially, and it might actually just be fine with some example links in the docs (via embedding), for generating nice live docs. Setup an account, and I will add you to the team. I tried making a Basic Scene but react-babylonjs seems to not work as expected (probly needs to be published.) Maybe initially, I should make a bunch of demos in a seperate repo, since codesandbox can pull directly from there, and then we can update them all at once, in git, instead.

Also, have you seen this project? Maybe we should join forces.

konsumer commented 6 years ago

Also, I had another idea to play with stencil. It allows you to quickly make web-components that can be used in any other lib (preact, react, angular, plain HTML, etc.) There is a concept of state-tunnel that seems super-similar to react's context. It might be a nice way to make all the work more portable to different frameworks. It would look something like this, in any lib:

<bab-engine>
  <bab-scene clearColor="black">
    <bab-hemispheric-light direction='up' />
    <bab-sphere name='player'>
      <bab-material diffuseColor="tomato" />
    </bab-sphere>
    <bab-follow-camera name='camera' lookAt='player' heightOffset='4' cameraAcceleration='0.05' maxCameraSpeed='100' />
  </bab-scene>
</bab-engine>

I feel like it could help us both do what we are good at really fast, and not overlap too much work. We can work on the auto-generation code, and component wrappers together, and you can do typescript + react wrappers (using same components and code-introspection) for react-babylonjs, and if we wanted to, we could do wrappers for preact, etc, but also it'd work without them.

as for those colors, I was think of using a util that can parse more color-types:

import colorString from 'color-string'

// get a color3/color4 from a CSS-color (hex, rgb, name, etc)
export const getColor = (c, alpha) => {
  if (typeof c !== 'string') {
    return c
  }
  let [r, g, b] = colorString.get(c).value
  return typeof alpha !== 'undefined' ? BABYLON.Color4.FromInts(r, g, b, alpha) : BABYLON.Color3.FromInts(r, g, b)
}

This would allow colors like tomato or rgba(0,0,0,0.5) so it looks more like css (which is really good in things like react, as web developers will be more used to it.)

Another utility I had in mind:

// get titlecase of string: "oh COOL, neato" => "Oh Cool, Neato"
const titleCase = s => {
  const str = s.toLowerCase().split(' ')
  for (var i = 0; i < str.length; i++) {
    str[i] = str[i][0].toUpperCase() + str[i].slice(1).toLowerCase()
  }
  return str.join(' ')
}

// get a direction/named-vector for a string
export const getDirection = d => typeof c !== 'string' ? d : BABYLON.Vector3[titleCase(d)]()

This would handle up, down, left, right, zero and others, for direction or other vector-things.

brianzinn commented 6 years ago

Oh, yes - we can definitely bring in utilities that makes writing software easier! Hopefully not too many big utilities to keep size/dependencies down or try to design in a way that plugins can be added for augmenting functionality. ie: textToColorProvider() or something.

I hadn't seen the SSD or Storybook before. I don't know if you have seen the HMR screencast on my sample JavaScript repo: https://brianzinn.github.io/create-react-app-babylonjs/ 100% agree though, it would be great to have working live examples that are easy to edit and code through to learn. codesandbox is great like your examples that kickstarted this - not as good as VS Code, but with a click of a link it's working anywhere and easy to edit! :)

Also, had not seen that other project - It's kind of doing the same thing :) Let me know what you find out with stencil, but the main thing I can think of is that this project is now getting tied (and always was really) with the react renderer. I don't know if those can be split off as reusable.

OK - I just pushed a commit. 1/2 way there to dynamic prop updates from generated code: https://github.com/brianzinn/react-babylonjs/blob/master/src/generatedCode.ts I just need hopefully some minor edits and to wire it up properly to react-reconciler (there is some untested code there now).

Also, yes - it's not on NPM yet. Let me get at least mesh props working and I will publish a 1.0.0-alpha.1 tagged with next. Then we have a fully experimental project!

konsumer commented 6 years ago

I played with stencil for like 2 hours, and just ended up fighting with type-errors ☹️. It's very typescript focused, and I don't see an easy way to just use ES6. I get tons of errors from babylon (first one referenced here, cannon errors, even though I installed it, etc.) I did my best to update all installed versions, etc, and I still couldn't get 1 tag (engine) to work! I am starting to actually not like typescript. Seems like a lot of futzing around with definitions, and not a very big payoff (I already get most of same stuff with my linter, with ES6.) I put it up here if you wanna take a look, I added you as a collaborator.

the main thing I can think of is that this project is now getting tied (and always was really) with the react renderer. I don't know if those can be split off as reusable.

Well, that's the trick with stencil: you can still add more layers on top in react, but it's not tied to any framework, and works really well with react, preact, and angular out of the box. The idea is we could share the part that is "components with props that efficiently render their children adding/removing/updating-props" and you can use those, as-is, in react or anything else (or nothing and use them directly.) It's just a thin wrapper, with utils, around standard web components. I'm so grumpy about typescript, that I'm actually not rooting for it anymore, but the idea of web-components does make sense to me for a general way to share the main chunks of our work, and have it also work in other component-libraries.

Also, yes - it's not on NPM yet. Let me get at least mesh props working and I will publish a 1.0.0-alpha.1 tagged with next. Then we have a fully experimental project!

Yep, I figured. No prob or hurry.

brianzinn commented 6 years ago

Ouch, for me a language needs to accomplish:

  1. Fun to work with
  2. Easy to communicate intent
  3. Good tooling support Looks like TS is failing my tests for you. I found there is a learning curve, but it excels over JS in all except maybe the 2nd codewise, but that pain is what makes the third.
konsumer commented 6 years ago

Yeh, stencil itself makes a lot of sense, and worked pretty well, very quickly. The structure was totally sensible. I had a "hello world" HTML component on the screen in less than 5 minutes. The typescript, as a language is fine, reminds me of actionscript (from flash.) I picked up how to use it pretty quickly, it was all typescript definition junk that was the issue for me. I spent most of the time trying to find compatible definitions in seperate repos, to make the app compile, with babylon, searching through issues queues, etc, and I never figured it out. As a sidenote, it doesn't work to build this project currently, probly for similar reasons:

screen shot 2018-10-24 at 11 56 38 am

Web-components are a standard, so I don't really need a lib to do it at all. Here and here show a setup that isn't really different than skate/stencil to my eyes. I basically just need 2 things: components that understand when their props change, and context that can pass variables to it's children. I mean, I could even use a scene string prop in everything to make it all connect to the correct scene, and a global (to the components) engine, and not even need the second part, but then the code ends up looking like just regular babylonjs.

brianzinn commented 6 years ago

Those errors are because of the version of ts or you can edit the typings file and change thatfrom Float32Array to undefined (in babylon.d.ts)

Nice work on Web Components. I like how so many cool concepts are coming together.

brianzinn commented 6 years ago

OK, so - dynamically generated code is now working with this project to update props with react-reconciler! I just got end-to-end working just for Vector3 and number/string using a proper diff algorithm, so not just updating everything. I think this paves the way for adding Camera and GUI without too much grief. That will add a lot of API support and to be honest, there is no way I would type all this generated code :) Still no NPM, but will push an alpha this weekend. You can use npm link, if you wanted to try:

class DefaultPlayground extends React.Component {

  constructor(props) {
    super(props);
    this.state = {
      position: Vector3.Zero(),
      visibility: 0
    }

    window.setInterval(() => {
      this.setState((prevState) => ({
        ...prevState,
        position: new Vector3(0, prevState.position.y + 0.2, 0),
        visibility: (prevState.visibility + 0.1) % 1
      }))
    }, 1000)
  }

  render() {
    return (
      <Engine>
        <Scene clearColor ={new Color3(250, 0, 0)}>
          <FreeCamera name="camera1" position={new Vector3(0, 5, -10)} target={Vector3.Zero()} />
          <HemisphericLight name="light1" intensity={0.7} direction={Vector3.Up()} />
          <Sphere name="sphere1" diameter={2} segments={16} position={this.state.position} visibility={this.state.visibility} />
          <Ground name="ground1" width={6} height={6} subdivisions={2}/>
        </Scene>
      </Engine>
    )
  }
}

I want to get your HOC goodness in the NPM as well! Been reading about the web components / stencil - looks like we could compile the React and include this in web pages, so that's a really fascinating idea as well.

konsumer commented 6 years ago

Those errors are because of the version of ts or you can edit the typings file and change thatfrom Float32Array to undefined (in babylon.d.ts)

Yeh, I gathered that, but wasn't sure how to fix it. If we can fix it here, I can maybe apply those fixes to stencil and get it working better.

I made a codepen that uses all regular vanilla standards-based stuff to wrap babylon in web-components. I think I did something wrong in babylon-space, because it's not rendering, but I think it communicates the basic strategy. If you have a look at the code, you should be able to see how easily this could be generated from BABYLON. Basically just add a bunch of allowed-props for all the editable properties and arguments, and then on render, work out the particulars like color or direction. It also poses very little abstraction-overhead, if you are on a modern browser (and the lazy-loaded polyfills aren't too bad, on an old browser.)

For me, this seems much better, as it will work for react, but also preact, angular, any other framework, and with plain old HTML. We could make seperate lil distros of it that have a more comfy feel in all the supported libraries, but overall it doesn't need it (I could see myself just using all web-components in plain html to make games.)

I think I could improve the base-class for that to have a nicer context implementation, as well as smarter change resolution.

konsumer commented 6 years ago

OK, so - dynamically generated code is now working with this project to update props with react-reconciler! I just got end-to-end working just for Vector3 and number/string using a proper diff algorithm, so not just updating everything.

I may be misunderstanding, but I think mine does this, too (using a shallow-compare) for every tag that is supported. We might be out of sync, in our 2 projects, or maybe mine doesn't do what you are saying.

That will add a lot of API support and to be honest, there is no way I would type all this generated code :)

Totally, generated + renderer is the way to go. Much less code to manage (work on babylon-parsing instead of individual components.)

Still no NPM, but will push an alpha this weekend. You can use npm link, if you wanted to try

Yep, no prob. I still can't get it to build, though. Is there some trick? I did git pull, then npm i, then npm run build:

node_modules/babylonjs/babylon.d.ts:8386:5 - error TS2717: Subsequent property declarations must have the same type.  Property 'leftBounds' must be of type 'number[] | null | undefined', but here has type 'Float32Array | number[] | null | undefined'.

8386     leftBounds?: number[] | Float32Array | null;
         ~~~~~~~~~~

node_modules/babylonjs/babylon.d.ts:8387:5 - error TS2717: Subsequent property declarations must have the same type.  Property 'rightBounds' must be of type 'number[] | null | undefined', but here has type 'Float32Array | number[] | null | undefined'.

8387     rightBounds?: number[] | Float32Array | null;
konsumer commented 6 years ago

I want to get your HOC goodness in the NPM as well!

Totally, this is a great way to give props to things. It makes it easier to do purely functional components, too (no class)

Been reading about the web components / stencil - looks like we could compile the React and include this in web pages, so that's a really fascinating idea as well.

Yep, that is my thinking. I think we could share all the work of "components with props", which would include the code-parsing, prop-messing (strings to enums/directions/colors/etc) and "efficiently update babylon scene-graph when props change". Then, if people want to use it with react, it's fine, but if not it's fine, too. And if you wanted to add some sugar on top for just react people (context comes to mind) you're free to, but most of the other work will be done. Like here is an example of using a web-component (<ba-engine >) to make the canvas (locked into shadow-dom for extra-awesome containment) but also a provider to give react children context:

import React, { Fragment } from 'react'

// this sets up <ba-engine> web-component
import 'ba-components/BaEngine'

// this is the one I made, for react
import { EngineProvider } from './EngineContext'

// this is <Engine /> complete with context for non-web-component react-use
export default () => (
  <Fragment>
    <ba-engine ref={r => { this.engineComponent = r }} />
    { this.engineComponent && (
      <EngineProvider value={this.engineComponent.context.engine}>
        {this.props.children}
      </EngineProvider>
    )}
  </Fragment>
)

This would work the same as current react <Engine />.

konsumer commented 6 years ago

Another crazy thing is that you remove all the custom-renderer stuff (ba-engine will handle it's own) and export a ton of just string mappings (like tags.json) for things that don't need any special wrapping:

export const HemisphereLight = 'ba-hemisphere-light'

react's renderer will take this to mean "use the regular html tag <ba-hemisphere-light /> when someone imports HemisphereLight" in the default renderer. We could even put them all in seperate files, so that the web-component wrapper doesn't get defined unless you import the react component:

import 'ba-components/BaHemisphereLight'
export const HemisphereLight = 'ba-hemisphere-light'
konsumer commented 6 years ago

Ah, I figured out my codepen, I forgot scene.render() in engine's runRenderLoop

brianzinn commented 6 years ago

mindBlown === true that is a "game" changer. You just keep coming up with the coolest demos! I will have a look at your ba-components/stencil this weekend

For this project, I'll generate all lights and cameras and make sure the "Model" loader is working. Should be able to get ShadowCaster working with minor edits - I added a bunch of comments and console.logs to the react-reconciler and you can follow how the object graph is built and the call orders. I'm going to leave the GUI for another week at least. I will push an NPM this weekend as well and try for the HOCs in the NPM. Then all of your examples should be working! yay :)

About the error

8386     leftBounds?: number[] | Float32Array | null;

I am literally editing the babylon.d.ts in my node_modules/babylonjs/.., because I don't want to waste time with compiler versions. I have to re-edit it everytime I add a module, but I just have tried to solve the problem by working around it... it's more fun to add new stuff :) Sorry, but I never figured out a proper solution either.

brianzinn commented 6 years ago

"I may be misunderstanding, but I think mine does this, too (using a shallow-compare) for every tag that is supported." You are correct. I am maybe trying to optimize too much and not include 3rd party libraries. I'll look at the library better. Thanks for mentioning. At the expense of including a 3rd party, probably this code is growing faster! :)

konsumer commented 6 years ago

I am literally editing the babylon.d.ts in my node_modules/babylonjs/.., because I don't want to waste time with compiler versions. I have to re-edit it everytime I add a module,

Oh dear! We've got to fix this for them. Maybe they will take a PR? I couldn't clearly figure out who's definition it is, like it may be a few projects deep. Is it something we can override/ignore in config? It totally spoiled my fun early foray into typescript. Gave me the immediate "oh this is all broken, how does anyone write any code with this, rather than just play with type-definitions?" sort of impression.

What's weird is that I am working on a new, better parser, using typescript module, and it has no probs reading the definitions. ts-simple-ast seemed ok with it, too. maybe it's just when it tries to actually transpile it?

brianzinn commented 6 years ago

Did a minor update and am generating basic Camera classes. Nothing new to see really, but looks on schedule to work on your examples in 2-3 days.

brianzinn commented 6 years ago

OK. I have published 1.0.0-alpha.1 under the tag next. It's working for me like the original version, but only for shapes, materials and lights (no VR, Shadows, GUI, Model, etc.). I am generating GUI, but they do not work and there are some issues on teardown, but you should be able to get a scene showing at least and it's using the react-reconciler and generated code!! I am going to split the code into individual files-it's 11K lines now and fix when the main component is unmounted. It does work with HMR updates and own state changes. I want to add some HOC goodies still, especially the before render one and see if they all work together :)

konsumer commented 6 years ago

Great work! I will take a look this weekend.

brianzinn commented 6 years ago

Thanks. It's got lots of issues and does excessive logging, but I will try to sort out the majority this weekend, if I can get some computer time. I just added 'target' to match your demos. It has a bug that you need to declare the meshes before the camera, but I will sort that out. Here is working syntax like your other samples:

<Engine>
  <Scene name="mainScene">
    <FreeCamera name="camera1" target={this.state.target} position={new Vector3(0, 5, -10)} />
    <HemisphericLight name="light1" intensity={0.7} direction={Vector3.Up()} />
    <Sphere name="sphere1" diameter={2} segments={16} position={new Vector3(-2, 2, 0)} />
    <Box name="box1" size={2} position={new Vector3(2, 2, 0)} />
    <Ground name="ground1" width={6} height={6} subdivisions={2}  />
  </Scene>
</Engine>
brianzinn commented 6 years ago

I realized that I was missing the export of withScene, which you would need. In doing so what I noticed is that our own components (functional or Component) would not get access to where they were declared in the JSX (ie: in the hierarchy) or the BabylonJS objects - that would be a disaster! What I have added is a component called HostWithEvents. This is picked up by the reconciler and you get access to lifecycle events (like when you are parented, which is the only one so far). Here is a control that when added as a direct child will alter the rotation by a defined RPM/axis:

import React, { Component } from 'react'
import PropTypes from 'prop-types';
import { HostWithEvents, withScene } from 'react-babylonjs'
import { Axis } from 'babylonjs'

class SingleAxisRotateMeshBehavior extends Component {

    componentWillUnmount() {
      this.scene.onBeforeRenderObservable.remove(this.handler);
    }

    render() {
      return (<HostWithEvents {...this.props} onParented={(scene, engine, parent) => {
        this.scene = scene;
        this.handler = scene.onBeforeRenderObservable.add(() => {
          // TODO: if parent.babylonJsObject.rotationQuaternion then .rotate(xxx, axis)
          switch(this.props.axis) {
            case Axis.X:
              this.rotationProperty = 'x';
              break;
            case Axis.Z:
              this.rotationProperty = 'z';
              break;
            default:
              this.rotationProperty = 'y';
              break;
          }
          let deltaTimeInMillis = engine.getDeltaTime() // smooth animation different FPS
          parent.babylonJsObject.rotation[this.rotationProperty] += ((this.props.rpm / 60) * Math.PI * 2 * (deltaTimeInMillis / 1000));
        })
        }      

      }/>)
    }
}

// Specifies the default values for props:
SingleAxisRotateMeshBehavior.defaultProps = {
  rpm: 1,
  axis: Axis.Y
};

SingleAxisRotateMeshBehavior.propTypes = {
  scene: PropTypes.object.isRequired,
  rpm: PropTypes.number,
  axis: PropTypes.object
};

export default withScene(SingleAxisRotateMeshBehavior)

Then you have it in your code like this:

<Engine>
    <Scene touchActionNone={true}>
      <FreeCamera name="camera1" position={new Vector3(0, 5, -12)} target={Vector3.Zero()} />
      <HemisphericLight name="light1" intensity={this.props.lightsDim ? 0.3 : 0.7} direction={Vector3.Up()} />
      <Box name="box" size={4} position={new Vector3(0, 1, 0)}>
        <SingleAxisRotateMeshBehavior rpm={this.props.clockwise ? 12 : -12 } axis={Axis.Y} />
        <StandardMaterial name="mat1" diffuseColor={Color3.Yellow()} specularColor={Color3.Black()} />
      </Box>
    </Scene>
  </Engine>

I am going to experiment if this can be used to allow controls like ShadowGenerator and 3D GUI to have cleaner solutions (I have already made adding materials pluggable). The original controls were tightly coupled to immediate parent and I think we will want to have crawling the parents (ie: search for parent Light that can cast shadows) or some kind of registry or recursive search from root when using names as strings. Anyway, this is one more piece of the puzzle. I think the technical hurdles are behind (hopefully). Sorry for long message - just want to make sure if you are having a look that you are not wasting your time. Thanks for all your valuable input. npm 1.0.0-alpha.5 or tag next :)

brianzinn commented 6 years ago

I have got everything mostly working! It's running on: https://brianzinn.github.io/create-react-app-babylonjs For you, I added a functional component on the Mesh Mashup page for the button: https://github.com/brianzinn/create-react-app-babylonjs/blob/master/src/remixMeshMashup/MashupButton.jsx

Thanks for the great ideas and all of your input and help. You have lots of other good comments in this issue (storybook, web components, etc.) - I am definitely going to read through your comments a couple more times for a to do list. Feel free to open any new issues for your ideas ie: Easing / Skeletons / Particles / Animations. Maybe we can make a CodePen for the github page :)

konsumer commented 6 years ago

That sounds great. Sorry to drop out, got a bit busy with other stuff, but it looks great! Good work.