New Element: OcTree Sub Block Model

[fwkoch]

In addition to name, description, metadata, axis_u/v/w, corner, and attributes, regular block model will have:

• parent_block_size = [i, j, k] doubles
• parent_block_count = [ni_p, nj_p, nk_p], integers
• some way to store level - this needs further discussion

[rowanc1]

For many of the Sub-Block Models, it could be possible to store efficiently through a CSR-like storage format.

Needs some more thinking, but this would allow the attributes to be stored very efficiently.

[mike-berriman]

I haven't been involved in the conversations about this but I wonder why OMF doesn't store all blocks models as "Arbitrary" models (as per the other issue), with possibly some hints as to a sub blocking scheme. Therefore, those products that don't care can just read all models the same way, and if a product does care about the sub blocking scheme it can use the "hints" and the block position and size to retrieve the model as desired?

[mike-berriman]

I will add that Surpac models (in particular) are not all just plain octrees - they can be variable as they go down the tree. They can stop subdividing in one or two directions, meaning that the octree can change into a quad tree or even a dual (?) tree as the tree gets deeper. This obviously add complexity to your design.

[rowanc1]

Hi @mike-berriman, thanks for your comments and engagement!

There was a lot of discussion around these points at the meeting Feb 28-March 1. I have tried to summarize a bit of the thinking/discussion below. The summarized group notes are in the wiki here: Block Model Summary, but they don't capture the why part of our conversations very well at all - they were the decided outcomes.

I hope the below helps clarify some of the thinking. Happy to expand more as required.

---

Summary

1. Workshop participants chose multiple formats in the storage to reduce ambiguity, storage size and data access time.
2. A client library in C++ was decided to help ensure access in a standardized way (arbitrary sub block model) for the two other types introduced (regular and octree).
3. All vendors present (including Dassault/Surpac) noted that they would, or already do support multiple of these types. Best practice will be documented to write the format out as octree/regular sub block model if that is closer to the internal representation.

---

Detailed Reponse

Where the OMF workshop participants got to was persisting data in different ways if there is more information known. This has the advantage of data access speed and storage size for many of the more regular formats (regular sub and octree). We decided to move away from storing everything as an arbitrary sub block model to ensure that the format helps to ensure that there is no ambiguity. For example, if it is regular or octree there is more than a string hinting you to read it like that, but the format actually ensures that it is stored and accessed that way. Both of these examples improve storage size by not having to store centroids, ijk indices, parent block pointers, etc. due to the topology of the grid being strictly enforced.

To expand on your other point: why not just have everything written/read as arbitrary sub block model. The writing has to do reducing storage space, access time, and general ambiguity, as expanded upon above. The reading is included as part of our work plan moving forward, with introducing a C++ implementation. There are conversions between all sub block models and arbitrary; however, the reverse mapping is harder, even if it is tagged in a hint. This conversion (regular sub --> arbitrary and octree --> arbitrary) is aimed to be supported in the C++ library to reduce the burden on vendors.

In the example you provided, if the example is not an octree, it should be either be (a) converted to octree by filling in the appropriate blocks; or (b) converted to an arbitrary sub block model. It is possible that the increase in storage would be less for splitting a few leaf-blocks, rather than removing the concept of the tree altogether and being forced to choose your parent block size as one of the levels in the tree.

I hope that helps in showing some of the discussions we had behind these decisions. Happy to help expand on any of the points above, and revisit these as we move towards the OMF.v2 spec in the coming three months.

[mike-berriman]

Thanks for the clarification Rowan. I realised there was probably a lot of discussion around these points and was curious about how these decisions were reached. While I don't necessarily agree - at least having the C++ library to do the conversion as required is a good step - particularly as many of the vendors that would be targetting are the players in the market who are much smaller, but need read access - but probably aren't part of the GMG committee.

As for the example - I agree there are several ways to export the variable tree model - I just thought I'd mention it for completeness. I'm not aware of any other vendors using Octree block models, so it seemed a good idea if OMF was having a specific octree definition that it should be able to handle the variety.

One point I've always disagreed with OMF is the need for it to be efficient and fast. This is an interchange format - it should be as easy to decode as possible - size and speed are very secondary to that. Anyway - these are my opinions, fell free to ignore them or use them as required.

FYI - my interest in this is I wrote much of the Surpac block model, but I no longer work for Dassault. I'm still in the industry however, so am keen to see what comes out of OMF to be as useful as possible for as many groups as possible.

Thanks for getting back to me.

[rowanc1]

Thanks @mike-berriman! All solid points, and thanks for the background.

The accessibility piece was highlighted as critical, as you also noted. This was one of the main motivations for the C++ library that is targeted for use in multiple clients (e.g. Python, .NET, etc.) (i.e. it should be as easy to decode as possible!). This library will also support some basic conversions between elements. The steering group also decided to change the container to a zip file (see #36), which also is aimed at improving accessibility.

We are putting together a bit of a proposal for the block-model specific elements for OMF.v2 at the moment. I will tag you on a review when it is ready!

[fwkoch]

For implementation details, please refer to the implementation and accompanying discussion in the example notebook: https://github.com/gmggroup/omf/blob/691420e3761c874139ca4127abae276204045111/notebooks/omf_cbi.ipynb?short_path=1aa1177#L285