microsoft / MixedReality-UXTools-Unreal

UX tools and components for developing Mixed Reality applications in UE4.
https://microsoft.github.io/MixedReality-UXTools-Unreal/
MIT License
316 stars 86 forks source link

Surface highlighting on BoundsControls not working #25

Closed SuperFreqA closed 3 years ago

SuperFreqA commented 3 years ago

Using version 0.11 UE Version 4.26.1 plugin loaded in project directory (not sure if that matters.

The default Uxt Bounds Control Component does not appear to have the proximity surface highlighting I am used to for HL2 controls. Is there a way to turn this on. Corner and edge controls display correctly but surfaces between them don't have that "flashlight" highlighting.

jjimenezg93 commented 3 years ago

Hi @SuperFreqA,

Thanks for your report!

I understand that you're missing this highlighting:

proximity_surface

I'm assuming that you aren't observing the issue in our demo scene, is that right? If so, could you please provide a .zip file with a project where this problem reproduces? Please remember to delete the Binaries/ and Intermediate/ folders to shrink the size of the file.

If that wasn't possible, could you please provide more details about how the actor with the UxtBoundsControlComponent is set up?

Thank you!

SuperFreqA commented 3 years ago

Unfortunately this is a project I can't share (company policy). I can probably try to reproduce the issue with another project. The actor is setup with two meshes under the scene node along with some proprietary deformable mesh components to do free form deformation. The UxtBoundsControl and UxtGenericManipulator are components at the root. MicrosoftTeams-image

Most stuff is just still set to default values. We had initially been using the 0.10 version (or whatever the first 4.26 release was) and updated manually to 0.11 from github as a project level plug-in.

jjimenezg93 commented 3 years ago

In order to get that highlighting behaviour, our default BP_CoffeeCup has a mesh component called BoundControl with SM_BoundsControl as static mesh and MI_BoundsControl as material instance, which dynamically render the highlight when appropriate.

Judging by the names of the components in your example, I think you might be missing this StaticMesh component. Is that the case? If so, you can copy it from the BP_CoffeeCup (note that the Construction Script also includes the BP code to automatically scale it).

Could you please let us know if this solves your issue?

Thanks!

SuperFreqA commented 3 years ago

Where can I get this BP_CoffeeCup? is there a demo project that I could reference for this?\

Edit: Now I see there is example code in the Github repo. Grabbing this now.

SuperFreqA commented 3 years ago

This does seem to fix the problem mostly. I did need to add the BoundControl object and the extra variables DefaultActorTransform and ShowBoundControl along with the construction script. Unfortunately getting the bounds for the two static mesh objects I have does not match the bounds for the UxtBoundsControl object. Our meshes have an origin outside of the object which is accounted for in the uxtBoundsControl but not when "get component bounds" is called in the blueprint. So we get a BoundControl object that is smaller than the uxtBoundsControl object. I assume that uxtBoundsControl is using a different call to set the box size?

jjimenezg93 commented 3 years ago

Indeed, the UxtBoundsControlComponent automatically adds a BoxCollider, which we use to prevent pointers from passing through the box and matches the transforms of the handles.

My understanding of your current problem is that this BoxCollider doesn't match the BoundControl mesh added in the Blueprint:

image

We have a task in our backlog to fix this and create both of them automatically, but I'm afraid that we don't know yet if it's going to make it into the next release.

In the meantime, you could work around this.

The logic used internally basically iterates through all the UPrimitiveComponent inside the actor (starting from the RootComponent) and creates an FBox that surrounds all of them. This changes slightly in 0.12.0, because we modified the API (Release Notes entry and code) and added a BoundsOverride property which allows you to start the search on one specific USceneComponent, instead of the root.

Therefore, you can try to replicate the code inside UUxtMathUtilsFunctionLibrary::CalculateNestedBoundsInGivenSpace in the Blueprint, ignoring the BoundControl itself. You could also try to modify our API by adding the UFUNCTION(BlueprintPure, Category = "UXTools|Math Utils") annotation to that function, so it can be called from the Blueprint. However, there'd be a conflict once you upgrade to 0.12.0, as we have modified the parameters of that function.

Do you think any of those workarounds would work for you? Happy to help further if necessary.

SuperFreqA commented 3 years ago

This works. Thank you for your help.

SuperFreqA commented 3 years ago

Screenshot 2021-03-18 095255 Here was my setup for those that have this issue. I am still seeing some artifacts with this solution that include proximity detection for grabbing corners and sides seems to be off by an inch or two. Also I am not getting triggers on translation (grabbing center of volume) with the flags turned on. Also the surface highlight (blue dot) will sometimes remain on the control after hands are moved away.

I am wondering if there were some issues going from 0.10 to 0.11? One of our UxtGenericManipulators was completely blank of details in the blueprint that was using it. We needed to delete it and replace it.

jjimenezg93 commented 3 years ago

Thank you very much for confirming that this worked for you and sharing your solution @SuperFreqA!

As it works, I'm closing this issue, but please do not hesitate to open new ones for each of the problems in your last comment. Please remember to be as specific as possible and keep in mind that we highly appreciate GIFs or videos that showcase each issue, so we can understand and help more effectively.

I'll do my best to comment on them anyway.

proximity detection for grabbing corners and sides seems to be off by an inch or two

What I understand is that either you expect to grab at a certain distance from your fingers but it doesn't, or it grabs when you are not expecting to do so. In either case, would modifying the GrabRadius of the UxtNearPointerComponent help?

Also I am not getting triggers on translation (grabbing center of volume) with the flags turned on

I'm not sure about what triggers and flags you're referring to. Would you mind elaborating in a new issue?

Also the surface highlight (blue dot) will sometimes remain on the control after hands are moved away.

Do you mind providing specific reproduction steps in a new issue as well?

One of our UxtGenericManipulators was completely blank of details in the blueprint that was using it. We needed to delete it and replace it.

I think that's because of the breaking changes introduced in 0.11 for the UxtGenericManipulator, apologies for the inconvenience. We try to make breaking changes less frequent with every release, but unfortunately they can't be entirely avoided.

Thanks a lot!