Figuring out CogBlock's Octree and converting it to work for CogBlock is a task closely related to #6 CogBlock Skeleton and #7 Questions about Octrees, and it must be completed before more work can really be done on #6 or the other issue can be judged to be closed.
However, despite how it's an integral part of CogBlock Skeleton, it really has its own sets of difficulties, tasks, debuggings, and understandings, and requires its own procedures to solve. Therefore, it gets its own issue. I also want to keep #6 clean of non-questions and non-explanations.
Heading quickly to successful rendering.
The important thing to do before rewriting everything is get an identical copy of ColoredCubes rendering using OctreeNodeAlt. Once I know I have all the proper pieces, I can start figuring out how to improve OctreeNodeAlt.
While there are a lot of questions to ask myself and a lot of details to work out, the thing I have to do first is make sure that everything starts actually rendering. And in order to do that, I more or less have to copy-paste until it works. THEN I can break it apart. So no big edits yet, please.
The first step is to create a completely independent OctreeNodeAlt that functions identical to OctreeNode, but references itself. CogBlock Volume must then successfully use OctreeNodeAlt in lieu of OctreeNode without the game hanging or throwing exceptions; regardless of the fact that nothing will change about the rendering.
Done, and it looks like it's working. The Volume is currently using OctreeNodeAlt (which really should be renamed, but there you have it.)
Okay, next step, grab some comments and isolate the code that renders ColoredCubes (Oh, push first, in case we destroy something
Works. Yay! Now I have to figure out what to do next.
I think I will rework the Octree to make it Object-Oriented again.
Reworking the Octree
It is important I first establish what the heck I want to accomplish. And why. I believe what I want to do is clean up the OctreeClass, but in order to do that I have to define some subtasks.
Remove BuildMeshFromNodeHandleForTerrainVolume and its siblings; I want this code to work modularly again.
I want the same mesh to be used both by the renderer and collider, if possible. It should not be built twice, which appears to be what the Cubiquity code is currently doing.
I want the activities of SyncNode to be split up into modular functions, especially at the major if/then control loops which are otherwise very hard to read. I do not exactly expect that these functions will have to be overwritten in the future; but I am leaving them protected because I want future authors to have the ability to rewrite chunks of SyncNode without rewriting the whole thing.
In general make the code more modular, more object oriented, more streamlined, and easier to read.
Get the code to render properly again without errors.
Currently renders again without errors. What shall I do next?
I shall perform some small optimizations.
Small Optimizations
One of the big things which OctreeNode used to do and (to a limited extent) OctreeNodeAlt still does is overuse GetComponent. These accessor functions have a tremendous amount of overhead. Therefore, one of the important steps I took was:
I identified the main paths of SyncNode which would run up to a thousand times or more per Synchronization (which seems to be happening every frame; it happens during EditorUpdate in editor and on a different thread during play). I attempted to remove all GetComponent calls from this pathway. The fastest way to do so ended up being to store volumeRenderer and volumeCollider.
I did not attempt to remove the GetComponent calls from NodeMeshUp, owed to how rarely the function is called. It will not greatly effect the runtime behavior of the game, only the startup time.
Now as a result of having done this, I had to consider memory overheads. Looking at the code, OctreeNode was already storing uints to check when each of the volumeRenderer and Collider were last updated... which it was checking previous to performing simple assignment operations on Booleans. Not exactly the best use of conditionals, time or memory.
Not to mention that lowerCorner is only used when rebuilding meshes, and could honestly be a localVariable that was only called for a short timeframe (and would free up 3 floats). It is important to recall that in Unity, every variable type takes up a full sized chunk of memory, and pointers are not big memory hogs.
Now as a result of the code I've written, we have the following situation:
Volume Renderer and Volume Collider, if edited, do not pass their edits down the OctreeNodeAlt chain until synchronization is needed.
The situation needs to be resolved, and there are a couple ways of going about it. :
volRend (the pointer) is tested against null every sync loop and, assuming there is no problem, its parameters are copied into the meshRenderer on the gameObject. However, this will entail using a getComponent every game loop on every node. We're avoiding that.
When the volumerenderer is edited, it GetComponent's the volume or the octree root node itself, and asks it to pass the changes downward. (note this can be done whether volRend is stored in the octreeNodeAlt or not, as volRend can be passed through the dissemination function. This is the easiest and most straigthforward solution to implement at present.
volRend and volColl probably do not need to be stored in pointer form at all in the octreeNode. Their values can be disseminated entirely through their own individual functions as listed above. The question, then, is how often their dissemination functions should be called after normal synchronization. A simple answer is probably "They should be called whenever SyncNode actually does work. However, this would cause them to update the entire tree as opposed to just the NodeMeshUp items.
Following from the above suggestion, it is possible that if the node does NOT store volRend or volColl but instead meshRenderer and meshCollider. Then it can look up its parent's MeshRenderer and MeshCollider by variable (if it is a subnode, so most of the time) and Volumerenderer and VolumeCollider by GetComponent (if it is a root node, so only once), and it can disseminate the necessary values in THAT manner during NodeMeshUp.
If the above plane were implemented, what value would we gain? (ie: it is a solution, but is it a useful solution? Obviously we could just implement stage 2):
Less connection to the volume renderer, and more connection to the previous node. This supports the metaphor of the tree. Fortunately, it appears that Cubiquity does not render anywhere except at leaf nodes; otherwise we would have an issue with an ocean being the parent of ground, and the collision mesh code having an aneurysm. As things are, neither technique 2 nor 4 will prevent us from tweaking collision when it comes to water/quicksand/leaves/ and other permeable or semi-permeable blocks, or even invisible blocks that serve as scripting triggers.
Fewer GetComponent lookup calls mean faster terrain building and sync time, which means faster loading of levels.
Takes up no more memory than storing volRend and volColl.
ERROR: Aha! Thought up the brilliance that will stop me from foolishness. Parent nodes do not HAVE mesh renderers or mesh colliders. Ergo they cannot disseminate such values.
Therefore it is easy to make this decision:
Due to minimal gain (theological gain and minor build time improvement) I do not believe it is useful to look for a way 'around' the difficulties posed above for technique 4
Values could still be passed through technique 3, however I am currently inclined to implement technique 2. I am also morally opposed to sending gameObject or volRend or volColl down through the sync pipeline.
It is still possible to get the build time boost by saving MeshRenderer and MeshCollider IN ADDITION to VolumeRenderer and VolumeCollider. It would be significantly easier to write out lowerCorner, for example, if more space needed to be gained.
In any event this feels like premature optimization, whereas technique 2 is actually quickly solving a problem we have: disseminating values.
Currently, the OctreeNodeAlt is configured to hold on to volumeRenderer and volumeCollider, and to pass these parameters down the tree in order to grab hold of changes.
Digressing on the way to dynamic declaration
Purpose
I initially started out trying to figure out how to solve a problem that didn't need solving; but on the way I realized that all rendering actually occurs inside the Octree code, and that the renderer/collider attached to the main volume are instead control containers which the octree references. There is a bit of illusion/interface going on.
Remember that C# is static!
I've been trying to use the factory-styled 'type of' constructors that I used in Actionscript, but I forget that C# is a statically typed language. Going about this the long way is probably more stupid than just re-envisualizing the problem such that the decedent calls the method of OctreeNode instead of vise versa. It's not like I won't know these things at compile/code time.
The problem, however, is that the game object wants to be initialized with values from the node... and the node can only be created by AddingComponent after the game object already exists. The flow therefore goes initGameObject > Add Component > Initialization Code.
And don't even get me started on Sync node, which needs to work with the generic Octree. Where should the 'iterate through children' code go? How does it create the new nodes?
The problem is the inability to instantiate the monobehavior independently.
You know what... Removing the GameObject component changes everything...My take is that Octree MAY be a game object so that, in theory, in the future, it's leaves could be non-octree-node gameObjects (ie, blocks) IN UNITY. Right now, all that data is stored in C. I probalby don't want to duplicate it. I could see why it's being used, it's an easy means of navigating children?
The realization the volume doesn't render
I have come to the realization that the volume itself does not render. The volume is the parent of a bunch of octree nodes, which don't show or serialize themselves, and which all have their own meshes.
So we're back to square one, we can't remove the game object component. What we can do is inherit from game object...
**Woops! Tehe***
Ha! I'm a fool. I speant hours trying to solve something for which there was no problem. AddComponent takes in a Type. I'm moronic XD. Turns out I had to change almost NOTHING. I feel like such a goose.
Some Edits on SyncNode return type
It looks like OctreeNode currently stores its children as game objects. However, I suspicion that it might be more sense to store the OctreeNodes themselves. The game objects are not used in the code the children are. (I have done this and altered the code to work more with the Octrees and less with the GameObjects)
Debugging 'Till Renders Journal
Currently I am at the stage where the code runs with no bugs but I don't see anything. I am using debug statements to find information. I am discovering...
Whether the Nodes are being created (Yes)
Whether the volumeRenderer is making it 'till the end (No!)
* Observation: It also appears an infinite number of Nodes are being created, which makes no sense at all... I checked and the culprit code is definitely in SetChild, suggesting that the nodeHandles might not be correctly lining up.
I still haven't figured out the infinite nodes trick, but what I do know is that these new errors come about as a result of not holding on to the primary game volume and making use of a getComponent. Managed to get it to work by saying not to serialize the rootNode (so it's created fresh when needed). However this could create a whole bunch of wonderful errors which I need to keep an eye on.
Alright, so we got the volume renderer to pass correctly. Still not rendering; time to form another hypothesis. Looks like the 'infinite node handles' but has somehow resolved itself as well; most likely what was happening is the node handles were incorrect and got serialized, and after I added the 'do not serialize' property they were all destroyed and then recreated in working order.
* Things are not getting preemptively destroyed.
I've gone and un-hidden the octree nodes so that I can study what's happening to them. I think I'm on to something; it doesn't look like their meshRenderer is getting properly added.
* I've noticed that NodeMeshUp() is never called (this would be adding the renderers). The only thing which could be getting in it's way appears to be updating the sync times. I wonder if it's possible that I deleted a location that would have zeroed out the sync times; altered update times without replacing it.
* I may have reversed the meshUpdate detector without realizing it, meaning the mesh sync code was never called (so meshes were never even built)
HA! I definitely fixed the problem, but now unity is crashing every time it tries to run XD. I'm going to have to debug what's happening. Okey-dokey! Commenting out the whole NodeMeshUp/NodeMeshDown logical split allows me to run Unity again, so it's time to investigate whether some null pointer, infinite loop, or other errors might be happening.
Tracked down the crash. It's happening in BuildMesh();
It's happening as early as some of the very first calls. It is crashing at mesh.hideFlags = HideFlags.DontSave; WOOT
Finding crash reports on Le Googlez. "A wild stab in the dark, but would this occur if you had savable game objects nested within non-savable ones?" POSSIBILITY: It may be that the GameObjects of the OctreeNodes must be hidden if the mesh is to be hidden. We can try and reactivate that code simultaneously if needed. No damn it, it's continuously crashing! Hmm, maybe I'll go re-hide the gameObjects and see what happens, as a test. Okay I'm having trouble right now. That didn't work. What do I do now? Does this have anything to do with not serializing the node?
I do not believe these crashes are making any sense. They appear to be happening at arbitrary times, which is really frustrating because it doesn't give information about where things could be going wrong. However I also don't think the editor is threaded. This perplexes me.
Right now it's happening consistantly around the very first CubiquityDLL.GetIndices(nodeHandle) call. Note, something is REALLY lagging even as I'm trying to edit code... Even while Unity is closed. It's starting to get frustrating. I'm hungry and It's getting that time to leave so I think maybe this is the state to leave it in right now. It's really, really, REALLY not making any sense... I just tried putting the same code in NodeMeshUp, which I should like to remind the jury is in the same place OctreeNode was using it.
I need some kind of idea or hypothesis, but honestly this is at the point where one just isn't coming to me. Why would CubiquityDLL.GetVerties(#) throw a unity-crashing error? UNLESS the native plugin itself is dying horribly... AH! I did I flip NodeMeshUp and NodeMeshDown? When it became clear it was crashing at CubiquityDLL and had a nodehandle, I started looking for any checks I might have skipped. Dammit it crashed again >:/ Maybe I can find out an alternative crashing explanation though. Ha! Aha, it was crashing at the testing for mesh of handle 0 in the debug. It still might work!
SQUEE! It's holding in there! Unwrap it slowly now! (This must have happened because I was reversing a lot of ifs in order to get code in a more sensible order, but then later went and pulled them all out into seperate functions)
Alrighty! Still not rendering properly, but not we're going into the correct functions. Time to uncomment some of those debug.log statements.
I'm gonna try un-hiding the nodes again to see if the renderers are showing up. No they are not. So the question is; where are we going wrong in getting the mesh renderer onto the game object... Alright, I'm seeing four successful MeshRenderer: calls, and a HELL of a lot of NodeMeshDown. That cannot be correct. Er, unless it's... hmm. No, can't be, right? Though it may be the flat ground one. Oh I found them! I just had to go all the way down to the leaves
Aha! It turns out by this point in time everything IS renderering, its just one has to zoom out from the focus. It looks like the problem is that transforms aren't getting parented properly. Everything's been shrunk/drawn in and is overlapping everything else, and I'm supposing there's a possibility that the transforms are getting all messed up (it looks more like transforms than normals). The next thing to do will be to get all of those in place.
Once this is done we will have a properly formed, object oriented octree class with a few small performance enhancers that can work with our new blocks. We will also have access to the code responsible for building the rendering meshes and applying textures, which will hopefully allow Lake to more easily do his thing. Yay!
Figured out the solution. Had to set localTransform = lowerCorner AFTER setting the parent Transform. Good thing I've played with Unity enough to know the order one modifies these things matters XD.
Figuring out CogBlock's Octree and converting it to work for CogBlock is a task closely related to #6 CogBlock Skeleton and #7 Questions about Octrees, and it must be completed before more work can really be done on #6 or the other issue can be judged to be closed.
However, despite how it's an integral part of CogBlock Skeleton, it really has its own sets of difficulties, tasks, debuggings, and understandings, and requires its own procedures to solve. Therefore, it gets its own issue. I also want to keep #6 clean of non-questions and non-explanations.
Heading quickly to successful rendering.
The important thing to do before rewriting everything is get an identical copy of ColoredCubes rendering using OctreeNodeAlt. Once I know I have all the proper pieces, I can start figuring out how to improve OctreeNodeAlt.
Reworking the Octree
It is important I first establish what the heck I want to accomplish. And why. I believe what I want to do is clean up the OctreeClass, but in order to do that I have to define some subtasks.
Small Optimizations
One of the big things which OctreeNode used to do and (to a limited extent) OctreeNodeAlt still does is overuse GetComponent. These accessor functions have a tremendous amount of overhead. Therefore, one of the important steps I took was:
Now as a result of having done this, I had to consider memory overheads. Looking at the code, OctreeNode was already storing uints to check when each of the volumeRenderer and Collider were last updated... which it was checking previous to performing simple assignment operations on Booleans. Not exactly the best use of conditionals, time or memory.
Not to mention that lowerCorner is only used when rebuilding meshes, and could honestly be a localVariable that was only called for a short timeframe (and would free up 3 floats). It is important to recall that in Unity, every variable type takes up a full sized chunk of memory, and pointers are not big memory hogs.
Now as a result of the code I've written, we have the following situation:
The situation needs to be resolved, and there are a couple ways of going about it. :
If the above plane were implemented, what value would we gain? (ie: it is a solution, but is it a useful solution? Obviously we could just implement stage 2):
Therefore it is easy to make this decision:
Currently, the OctreeNodeAlt is configured to hold on to volumeRenderer and volumeCollider, and to pass these parameters down the tree in order to grab hold of changes.
Digressing on the way to dynamic declaration
Purpose I initially started out trying to figure out how to solve a problem that didn't need solving; but on the way I realized that all rendering actually occurs inside the Octree code, and that the renderer/collider attached to the main volume are instead control containers which the octree references. There is a bit of illusion/interface going on.
Remember that C# is static! I've been trying to use the factory-styled 'type of' constructors that I used in Actionscript, but I forget that C# is a statically typed language. Going about this the long way is probably more stupid than just re-envisualizing the problem such that the decedent calls the method of OctreeNode instead of vise versa. It's not like I won't know these things at compile/code time.
The problem, however, is that the game object wants to be initialized with values from the node... and the node can only be created by AddingComponent after the game object already exists. The flow therefore goes initGameObject > Add Component > Initialization Code.
And don't even get me started on Sync node, which needs to work with the generic Octree. Where should the 'iterate through children' code go? How does it create the new nodes?
The problem is the inability to instantiate the monobehavior independently.
You know what... Removing the GameObject component changes everything...My take is that Octree MAY be a game object so that, in theory, in the future, it's leaves could be non-octree-node gameObjects (ie, blocks) IN UNITY. Right now, all that data is stored in C. I probalby don't want to duplicate it. I could see why it's being used, it's an easy means of navigating children?
The realization the volume doesn't render I have come to the realization that the volume itself does not render. The volume is the parent of a bunch of octree nodes, which don't show or serialize themselves, and which all have their own meshes.
So we're back to square one, we can't remove the game object component. What we can do is inherit from game object...
**Woops! Tehe*** Ha! I'm a fool. I speant hours trying to solve something for which there was no problem. AddComponent takes in a Type. I'm moronic XD. Turns out I had to change almost NOTHING. I feel like such a goose.
Some Edits on SyncNode return type
It looks like OctreeNode currently stores its children as game objects. However, I suspicion that it might be more sense to store the OctreeNodes themselves. The game objects are not used in the code the children are. (I have done this and altered the code to work more with the Octrees and less with the GameObjects)
Debugging 'Till Renders Journal
Currently I am at the stage where the code runs with no bugs but I don't see anything. I am using debug statements to find information. I am discovering...
* Observation: It also appears an infinite number of Nodes are being created, which makes no sense at all... I checked and the culprit code is definitely in SetChild, suggesting that the nodeHandles might not be correctly lining up.
* Things are not getting preemptively destroyed.
* I've noticed that NodeMeshUp() is never called (this would be adding the renderers). The only thing which could be getting in it's way appears to be updating the sync times. I wonder if it's possible that I deleted a location that would have zeroed out the sync times; altered update times without replacing it.
* I may have reversed the meshUpdate detector without realizing it, meaning the mesh sync code was never called (so meshes were never even built)