[Task] Cubiquity reads in Minecraft files

[Spydrouge]

The first broad task for this project is that Cubiquity should be able to read in Minecraft files (or Substrate files, as Substrate is the C library that we're working with for the saving/loading of Minecraft style files)

Task List

Initial Look Around

• Take a good look on how substrate is saving things.
  • Find our substrate data files
  • These exist in the StreamingAssets folder, and are .dat files (which Mono isn't currently configured to see...)
    • You can write click on the solution in the solution explorer, navigate to display options, and click on 'show non-source controlled files' or the equivalent option in order to get your workspace suddenly cluttered with files! And this will also show the .dat files.
    • However, given that these files are in binary, it's not really useful to be able to see them.
  • Find cubiquity's sample data files
  • The Cubiquity voxel engine stores a volume as a Voxel Database (as per the documentation). Currently this is stored as a Internally it is actually an SQLite database and so can be opened with a tool such as SQLite Browser; but actual data is compressed. We are not supposed to rely on this data format (author's instructions).
  • A best approach might be to find Cubiquity's functions for saving to a VoxelDatabase and an API for hand-creating a map, and to nab the Substrate-to-(OpenCog-Pixeland) code and copy/paste/edit it in.
• Find out how to simply setVoxel in Cubiquity
  • Cubiquity for Unity3D provides a very simple but powerful API for generating volumes through code. Each volume is essentially just a 3D grid of voxel values, and the API gives you direct access to these through the VolumeData's GetVoxel(...) and SetVoxel(...) methods.
• Determine procedure for moving from start to finish for creating a new scene with a new volume that reads in our data.

  Baby Project

• Create new Cubiquity Seen/"Project" that sets up its own empty volume.

  Custom Editors

I feel the need to write a few notes I've come upon in this area as I work.

Slotting in a File Name

Instead of trying to understand the whole of Cubiquity's code and how unity does editors, let's start by just finding out how Cubiquity slots in its file name at edit time.

I am currently looking at ColoredCubesVolume.cs (which is the box under which Cubiquity's file name info shows up) which is a Script under Cubiquity->Scripts (it is not specifically in 'Editor'). It derives from Volume, which Derives from MonoBehavior

• Every script in Unity derives from MonoBehavior. The checkbox for disabling a MonoBehavior (on the editor) will only prevent Start(), Awake(), Update(), FixedUpdate(), and OnGUI() from executing.
• Monobehaviors receive messages from the system such as Update() or onTriggerStay(). They also inherit members such as the gameObject they are attached to.
• Both Volume and ColoredCubesVolume.cs are prefaced with [ExecuteInEditMode], which I must assume means none of these functions are really specific to runtime.
• The [SerializeField] private VolumeData mData = null; seems to suggest that this is data Unity is going to preserve about the object.

• if UNITY_EDITOR #endif contains a cute sanity check where cubiquity attempts to determine they know how Unity's editor scripts work ;)

• I can't tell if this function is used to get the GameObject > New > ColoredCubeVolume menu interface going: public static GameObject CreateGameObject(ColoredCubesVolumeData data, bool addRenderer, bool addCollider)

NONE of this tells us about the ADD/PAINT/DELETE/SETTINGS tabs however. Let's Search in Files for "Paint". We come upon this class:

• public class ColoredCubesVolumeInspector : Editor
• This editor is prefaced thusly: [CustomEditor (typeof(ColoredCubesVolume))] Bingo! Looks right to me! I have learned something.
• Doh, this is for the VolumeData Object, not the .vdb file itself... Let's see if I can find a built in editor that I can examine...
• No wait, I looked up how Cubiquity loads .vdb files. It turns out they put an option under Assets -> Create -> ColoredCubesVolumeData Let's see if I can find the file for that.
• Found a file under editor called MainMenuEntries.cs that seems to actually put all the options in the editor menus and yes calls the GameObject CreateGameObject function I saw before
• PERFECT: Found two options under MainMenuEntires.cs for creating colored cubes volume data that not only demonstrates how to open a file but also how to SetVoxel for each dimension of the data.

  Independent Edit Window

• Although the idea of just copying Cubiquity's existing method is tempting, I may wish to briefly explore independently windowed interfaces (like the project explorer) which can perform tasks.
  • Why? Because although it might make sense to me to have an independent scene which I use in the editor as a loading bay, I doubt it makes sense to anyone else. It'll be a lot easier and more straightforward with other workflows if I create a 'one time only' loader panel- not some script that's added and removed later on just one object.
  • Therefore, I am going to take a look around and see if I can find code for an independent editor.
  • I have found this resource

Q: Which ColoredCubesVolume object will I be popping this data into?
A: None! I'm creating a vdb file. If I have to create a ColoredCubesVolume object along the way, then I can destroy it. My output is a new VolumeData Unity object and its corresponding .vdb file.

• I have just realized that I could probably avoid the entire independent edit window and instead create a menu item that does it all in one click.
• Currently chosen procedure:
  • I am going to push everything in it's current configuration.
  • Then I am going to save an independent empty script that makes an editor window for later editing of cubiquity files, or using brushes, or somesuch.
  • Then I am going to create a menu item that nabs the data in one go.
• Observation: Cubiquity attempts to save all .vdbs in its own folder. I would really prefer if this was configurable by whomever was doing the editing in Unity...
• I am currently seeing if I can just copy/paste the code for save and load. Most likely! I just have to track down the save/load code itself. Let's look for that before we start looking for how substrate loads dimensions (which will be the next task).
  • Got it; it's EditorUtility.[Open/Save]FilePanel(parasms)
• Going back to the old realization that I need a panel; there are some parameters going from Substrate to our game, such as which chunks to load and how far to offset the sky, which really belong in a panel.
• Woohoo! I've just gotten to the point where I've successfully figured out how to create the interface for the editor window, and to load in the file names we're going to need in order to attach all the converstion code to.
  • I have gotten a little derailed from tracking down the exact code I'll need in order to implement the conversion, but I recall that one of the first things Cubiquity is going to need will be the dimensions
• Currently chosen assumptions:
  • I need to figure out exactly what 'chunks' are stored and which ones to convert. My understanding is that a chunk is a 16 by 16 cube, and that the max size for Voxel Databases is more around 512 by 512. However, minecraft works can be relatively... infinite.
  • I think that my beginning procedure, before I exactly know what's going on, is to nab all of the chunks and place them into the same Voxel Database. Then- and only then, when things begin to get slow- should I delve in and start doing some breaking-up and culling stuff.
  • I need to figure out the dimensions to make the ColoredCubeVolumeData, so that's the first thing I've got to get out of Substrate

[Spydrouge]

I have gotten the conversion to work. Lake is asking me to 'dumb down' the interface. I need to make a plan for how to do that.

Personally, one of the things that really gets me about how all this works is that I cannot repeatedly reload block data and tweak parameters (like chunks to load or vertical offset) to see the effects that has on the vdb. The reason for this is that the vdb has severe trouble unloading or deleting itself. (It is almost impossible to delete a vdb from within Unity, even after its corresponding .asset file has been destroyed. I want to spend a little bit of time figuring out why this is the case, and if there is possibly a way around it. I feel it could eliminate a great deal of confusion.

[Spydrouge]

The problem is entirely that Cubiquity does not want to work with the .vdb because the file already exists. Two things are going to make everything prettier...

1) Test if the file already exists so we don't corrupt the data (which happens if we just run things the way they are now; the vdb/Voxel Database.asset stop working) 2) If the file already exists, determine what to do... Delete it?

• Delete it and replace it forcefully? can we do that, or will unity say 'can't open, in use'? hope the Voxel Database.asset and it line up afterward?
• Delete whatever is accessing the .vdb (ie: the Voxel Database.asset) in an attempt to delete the .vdb and then recreate the asset?

Question I ask myself: What opens the .vdb file and ends up leaving it open?

[Spydrouge]

Closing Issue and opening new one for new problems