Newly added door entities - collision problem

[Const1702]

When a composite entity is added through the Cathode editor with the path AYZ\DOORS\ the door does show and it has its functions, but the collisions are a problem:

1. It doesn't let the player pass through, even when it is open.
2. Bullets also don't go through and flares bounce off the opened part of the door.

There are no problems with the already existing doors on the level, only with the added ones.

Attempted on level: Tech_Hub The door composites are added to AYZ\LEVELS\SEVASTOPOL\TECHNICAL\TRANSITSTATION\TRANSITSTATION_SECTION

The way I entered this level isn't by loading a mission, but I used the "Select Level" from the Debug Checkpoint mod, that way it completely refreshes and the changes from the editor can properly apply, but it still doesn't fix the collision problem with the new doors.

(I am not sure if this is an issue or it requires extra modifications to fix)

OpenCAGE version: 0.9.3.02

[MattFiler]

Unfortunately this is a known issue. I'm not entirely sure on the fix just yet, and think it's something I'm yet to reverse engineer.

In the meantime, you may be able to get around it by modifying the CollisionBarrier entities within the door composite?

[Const1702]

Thank you for reading! I'll definitely try to play around with the CollisionBarrier. Hope it gets resolved soon!

[MattFiler]

Let me know if you have any success. The only other lead I have atm is that doors are based on a Door entity function type, which could be doing collision-related stuff internally. In which case, it would probably be that something collision-related is going through the Mover file, which I don't add entries for new doors within (so would explain the issue).

It's something that has been on the roadmap for a while and will be a focus soon!

[Const1702]

I was looking through the properties of a door composite (DOOR_MED to be exact), but I couldn't find anything related to collisions. Only when I click on "Door_Package_1" and then selecting "Go To". Then I can find 2 collision-related entities: - PhysicsBarrier (CollisionBarrier) - ClosingBarrier (CollisionBarrier) I don't think these are useful, because they are related to door packages that are defining whenever the door has a lever, button, hacking, keycard, passcode or just a blank panel and that is a small portion of the door. I explored the "door" function entity and most of its features are triggers (Example: started_opening, finished_closing...), but the ones that grabbed my eye are: "geometry", "nav_mesh", "invert_nav_mesh_barrier", but they are still difficult for me to understand.

I have also found some small differences between existing and new doors. There are 2 that I know of:

1. New doors don't have a delayed animation. If you don't know what a delayed animation is, it is when both of the status lights (the red and green ones) turn on for 1 second, then the door makes a bloop sound and it opens. This animation is random when opening existing doors, but never appears when opening new doors. If you still don't understand I can post a video showing how the delayed animation looks like.
2. When door packages (Levers, buttons...) are changed through the editor for an existing doors, as another user in a closed issue mentioned, it does update the mechanism, but it also leaves the previous panel's texture floating in the air. This is an issue that you are aware of. In added doors the door package issue doesn't happen. It doesn't leave a floating panel and it changes the mechanism properly.

With those two issues shown, I think that there might be very small differences between the existing and added doors' mechanisms and animations. If you are going to put doors on your priority list, I recommend you check in a detailed way packages and animations.

[MattFiler]

The "delayed animation" you mention is actually an indicator that the game is loading in content behind the door. This loading is done via the scripting system's use of Zone entities, which use TriggerSequence entities to group objects to load in/out. When you approach a door, it can optionally wait for the zone contents to be loaded before opening, which allows for level streaming and optimisations to memory overhead at runtime. If you've just added a new door, you probably haven't hooked any zones up to it, which is why you won't get the delay as nothing is loading.

Door packages lingering are again related to the Mover descriptors - changing the package will break the link to the package listed in the MVR file, which will cause it to take all info from the MVR and become a static renderable in the world. You should be able to fix this in the new update by using the "Edit Movers" button and positioning the mover for the package far off in the level. It's not an ideal solution, but one that works for now while I better figure out the contents of the Mover descriptor so that I can add/remove them.

[MattFiler]

Made a discovery related to this today which I'll post here to keep my thoughts in one place:

• Swapping all internal resources to resource parameters produces the same collision issue with existing doors.
  • Is the issue with new doors perhaps an ID issue when pointing to a resource reference?
  • Alternatively, if it's still an MVR or wider problem - perhaps changing the resource IDs to parameters rather than nodes causes a breaking link somewhere. If this is the case, why does that affect children? Do they need MVR entries? Can we spoof them for now?
• Swapping all resource parameters to internal resources causes a hard crash (perhaps they're ref'd somewhere?).

[Const1702]

I have a feeling that this may be related to NavMeshBarriers located in a door's package (in my case the door is AYZ\DOORS\DOOR_MED, I just searched door_package and click "Go To"). NavMeshBarriers has paramaters and resource references like shown in the pictures below: "COLLISION_MAPPING" and "NAV_MESH_BARRIER_RESOURCE" don't seem to be investigated enough to modify them.

Another thing I discovered is that again in the door_package there are 2 collisionbarriers: Both of these barriers have the resource reference "COLLISION_MAPPING". The ClosingBarrier seems to have a "Door" function entity connecting IN to it:

I have tried moving existing doors around the map before and never had problems. Only when creating new ones. Door_Package seems to be a crucial part of the door, because DataType FLOATs like "request_open", "force_open", "request_close" all seem to be connecting to Door_Package. The ModelReferences in the actual door (AYZ\DOORS\DOOR_MED for me) seem to have links connecting IN to them from the Door_Package and not from the actual "request_open", "force_close"...

Will update if I find something else.

[Const1702]

Something related to the lingering door mechanism problem I discovered is that when I move an existing door to a new position and then change its mechanism it does change it, but it leaves the lingering part not to its new position, but at the old one. Here is an example of moving the Door_SML leading to the transit control lift in the SCI_Hub level. What is happening here is that the broken mechanism is always rendered to the position that was initially assigned to it by the game devs, no matter where the existing door is moved: How I achieved this bizarre discovery: If you are testing this it is very important to move the existing door first and then change the mechanism, so you can see how the mover not only gets detached, but also gets moved to the original position where the door was before.

With newly added doors the floating problem doesn't occur at all for some reason. I'm not sure if this is related to entity creation for doors.

You are saying that the current way to fix this problem is to move the mechanism away from the map. I'm not sure if I am doing it correctly, but here is the way I am trying to achieve this: Open the resource reference editor directly on the door composite -> Add new renderable instance -> Change Model -> This is where I run into a problem. The DOOR_PACKAGE composite is located in ARCHETYPES... and in the model browser's sidebar there isn't an "ARCHETYPES" list like in the main window: Tried looking through the available lists, but found nothing. This is probably the wrong way of doing it and it might not be possible yet. But if it is, could you please let me know how to get control of the mechanisms (Door_Mechanism_Front and Door_Mechanism_Back)? You also mentioned that it is related to mover descriptors, but the movers button for the door is greyed out (like shown in the picture below), so I can't control it from there. It seems to be only available in ModelReferences, LightReferences, FogSpheres (I think) and the only way to get to a door's ModelReference is to click "Go To" and modify it from there, but that will change it globally for all doors.

I was trying to use Overrides to delete door mechanisms with the "deleted" paramater, but that also didn't work, due to the checksum problem with overrides.

[Const1702]

I made a discovery with collisions. When an existing door is being hit with a wrench it makes a sound (can be also a glass sound if you are hitting its glass part with doors that have glass on them), but with new doors it does nothing. Flares and collisions work properly with both doors - they bounce off when closed, but when new doors are open flares keep bouncing off the opened part. Shooting at a closed existing door leaves a bullet hole and shooting a new one doesn't (the bullets might be still getting stopped by the door tho, because if I shoot it at an angle where it would go through the door an hit the ceiling it doesn't. That means it is probably only swallowing the bullet without a trace). Flamethrower fire collides with existing doors and goes through newly added ones when they are closed, like they are not there.

In the video shown below (on level: ENG_TowPlatform) the door in its frame is the existing one to the airlock and the badly positioned one is the newly added one: File size too big so unfortunately it had to be done with YT https://youtu.be/uMTc1oqzMqU

[Const1702]

Question related to collisions: What does the blank collision mean? It is not selectable in the dropdown menu. You can find it in some existing ModelReferences. In this case Level: ENG_TowPlatform, Composite: AYZ\DOORS\DOOR_MED, Entity: Door_R (ModelReference):

Is it an unidentified collision by OpenCAGE so newly added entities are created without it? I've noticed if I add some corridor/interior elements like technical ceilings, walls, floors they have no collision at all. You just fall through them (only with newly added ones). On ENG_TowPlatform -> AYZ\TECHNICAL\CORRIDOR_ELEMENTS\FLOOR (haven't tested this composite in this level to see if it works): Collision01_COL (ModelReference) has a blank collision mapping, but the actual Floor (ModelReference), there is mapping assigned to it.

I remember seeing a twitter post (that I can't find right now) you made about someone making a custom level under a day not so long ago and he showed a door went through it and had no collision problem with it. Also the interior/corridor elements in that level from the twitter post had proper collisions and the person showing it didn't fall through.

Congratulations on making it on the game news (PC Gamer...) by the way!

[MattFiler]

Congratulations on making it on the game news (PC Gamer...) by the way!

Thanks!

RE the collisions: I think these were used at map compile time in order to generate the level's collision mesh in Havok. The IDs usually relate to collision meshes which'd be compiled into the whole collision mesh. Changing them now doesn't do anything as the script tool doesn't re-compile the Havok collision mesh yet (although I hope it to someday). For now, when they're unresolved they just go blank - this could either be a issue with how I'm resolving them or just deleted collisions within the game - it's something I'm still looking into.

On a broader topic of collisions, you should be able to use CollisionBarrier entities to add your own custom collision, which you can tie to models by creating a composite with both the ModelReference and CollisionBarrier and then instancing that instead of just using a ModelReference alone. Hope this makes sense!

You wouldn't get any AI support for the custom collision, but again this is something I'm looking into (generating navmeshes on save of Commands).

[MattFiler]

Ok, I've found some more out about this!

Doors contain a PhysicsSystem entity, which defines a DYNAMIC_PHYSICS_SYSTEM resource reference within the door composite, pointing to a physics system index. This resource reference is seemingly further defined by the PHYSICS.MAP file to define the physics system for the door.

It's a similar system to how the EnvironmentModelReference nodes within doors define a ANIMATED_MODEL resource reference (for the door open/close anims), which point to an environment animation index seemingly further defined by the ENVIRONMENT_ANIMATION.DAT file.

While looking into this I've also figured out that the COLLISION_MAPPING resource references appear to point to the COLLISION.MAP file, which defines a load of info about the collision mapping for each instance of an entity (similar to the PHYSICS.MAP contents).

All of these resource definitions appear to then also be referenced within RESOURCES.BIN which is like a global container/database for all the resources in the level.

I'm not parsing or using any of the above files just yet, which explains why more complex composites that rely on resources are sometimes working incorrectly when created from scratch, and indeed why the doors don't work properly when added via a new composite instance rather than just moving existing ones. Commands is apparently only one piece of the puzzle when it comes to the data read-in by the engine.

I've figured all of this out off of the back of the work done for #194, which was caused by Character entities requiring a separate file to define their instanced data in a similar way to this. Figuring out how that extra file linked to commands has unearthed this plethora of new stuff to reverse engineer - and also is very similar to the issue in #144, in that the MVR file contains instance specific material overrides for instances of entities (which I also now know how to link up to commands thanks to #194).

Unforuntately, I don't think it's gonna be as simple as just reverse engineering the files, as the way I display data in the script editor currently is pretty poor in regards to understanding the actual hierarchies of instanced nodes - for example:

• I create a composite with a ModelReference entity in it positioned at 0,0,0
• I create another composite, and instance that first composite at position 1,1,1
• I add another instance of that first composite to my second composite at position -1,-1,-1
• I set my second composite as the root composite that the level loads to If I was now to load the level, I'd have two ModelReference entities within it, one at 1,1,1 and one at -1,-1,-1. Both are coming from the same composite & are the same entity, but they are two instances within the runtime game.

My script editor is great at showing the composites individually, but it's bad at letting you get an understanding of how those hierarchies will populate at runtime. For example, what if I wanted to colour one instance of the ModelReference blue and another green? I could use overrides - but then understanding that hierarchy, particularly if I begin adding in additional levels of composites there is really confusing. This sort of thing is entirely what MVR is for - adding instance specific information to instanced entities. It's also what a lot of these additional files seem to be for - defining instance specific information for resources.

To display it better in my editor I need to do a lot of reworks, to be able to jump down hierarchies and have a better conceptual understanding of where you are (as well as just editing the actual composite itself to apply changes across the board). It'd be better backed up by a 3D interface too. And here you see my issue - as I try to implement one thing, the list grows and grows 😂

The dream would be a full-on prefab system similar to something you'd find in Unity.

I might be able to get away with doing some simpler auto-calculations for this data in the meantime, but first I need to reverse engineer the file formats & then have a think about how I could automate all of it in the background. I might end up just displaying it similar to how the MVR & Character instance-specific stuff is done currently, while I work on the nicer instance display.

TL;DR: I think there's a whole load of additional files to reverse engineer, notably RESOURCES.BIN and PHYSICS.MAP for the doors to function. It'll make the commands editor considerably more robust at generating levels, but will be a considerable amount of effort to display nicely!

[MattFiler]

In the spirit of keeping this issue updated - #268 has been created to cover the ground work for implementing the above changes.

[MattFiler]

Pinning this issue to mark that it's currently top priority.

[MattFiler]

I've been back looking at this again, and I've definitely narrowed it down to the PhysicsSystem entity. Its DYNAMIC_PHYSICS_SYSTEM resource (which actually gets applied to the entire composite) is defined via the PHYSICS.MAP file, which itself is then linked to commands via a separate RESOURCES.BIN file.

If I clear out the PHYSICS.MAP file, the game doesn't crash, but also none of the doors collisions work, in exactly the same way as if you add a new door. So this is definitely the missing info the game is after.

The file contains a big matrix for each entry, and if you mess with the matrix all the physics in the level goes bizarre - I'm assuming because colliders are flying around the place. Basically, I need to figure out what this matrix is defining, and then we can make new ones.

As always with this reverse engineering, this is now also presenting other issues, as I need to figure out what all the other stuff in RESOURCES.BIN is pointing to in order to add new entries to it as well. It's seemingly connected to MODELS.MVR, which is the bane of my existence :D

[MattFiler]

I'm now correctly writing out to RESOURCES.BIN, COLLISION.MAP, and PHYSICS.MAP - and newly added doors are working!

I'll tidy up the code and push it to staging soon.