search

microsoft / Microsoft-OpenXR-Unreal

An Unreal Engine game plugin providing additional features available on Microsoft's Mixed Reality devices like the HoloLens 2 when using OpenXR.
MIT License
163 stars 52 forks source link

found problem while packaging this project with unreal marketplace's openxr plugin #12

Open Hawkpool opened 3 years ago

Hawkpool commented 3 years ago

Hello there, I'm trying to building a hololens 2 app with this plugin in order to try out QR tracking function, but met problem while package project for hololens. I tried to install lasted winrt nuget in vs2019 , but problem still. I'm new to C++ programming and hope you can help me solve this problem. Thx~~!

console logs here:

UATHelper: Packaging (HoloLens): Setting up ProjectParams for F:\ue4\workspace\MsftOpenXRGame\MsftOpenXRGame.uproject UATHelper: Packaging (HoloLens): ** BUILD COMMAND STARTED ** UATHelper: Packaging (HoloLens): Running: F:\ue4\UE_4.26\Engine\Binaries\DotNET\UnrealBuildTool.exe MsftOpenXRGame HoloLens Development -Project=F:\ue4\workspace\MsftOpenXRGame\MsftOpenXRGame.uproject F:\ue4\workspace\MsftOpenXRGame\MsftOpenXRGame.uproject -NoUBTMakefiles -remoteini="F:\ue4\workspace\MsftOpenXRGame" -Architecture=arm64 -Manifest=F:\ue4\wor kspace\MsftOpenXRGame\Intermediate\Build\Manifest.xml -NoHotReload -log="C:\Users\Hawk\AppData\Roaming\Unreal Engine\AutomationTool\Logs\F+ue4+UE_4.26\UBT-MsftOpenXRGame-HoloLens-Development.txt" UATHelper: Packaging (HoloLens): Using ARM64 architecture for deploying to HoloLens device UATHelper: Packaging (HoloLens): Building using Windows SDK version 10.0.19041.0 for HoloLens UATHelper: Packaging (HoloLens): Your Windows SDK version (10.0.19041.0) for HoloLens is newer than the highest tested with this version of UBT (10.0.18362.2147483647). This is probably fine, but if you encounter issues consider using an earlier SDK. LogSlate: Took 0.000250 seconds to synchronously load lazily loaded font '../../../Engine/Content/Slate/Fonts/DroidSansMono.ttf' (77K) LogSlate: Took 0.001146 seconds to synchronously load lazily loaded font '../../../Engine/Content/Slate/Fonts/DroidSansFallback.ttf' (3848K) UATHelper: Packaging (HoloLens): Writing manifest to F:\ue4\workspace\MsftOpenXRGame\Intermediate\Build\Manifest.xml UATHelper: Packaging (HoloLens): Building MsftOpenXRGame... UATHelper: Packaging (HoloLens): Building 2 actions with 12 processes... UATHelper: Packaging (HoloLens): [1/2] MsftOpenXRGamearm64.exe UATHelper: Packaging (HoloLens): MixedRealityInteropHoloLens.lib(AzureSpatialAnchors.obj) : error LNK2038: mismatch detected for 'C++/WinRT version': value '2.0.201113.7' doesn't match value '2.0.201217.4' in PCH.MicrosoftOpenXR.h.obj UATHelper: Packaging (HoloLens): MixedRealityInteropHoloLens.lib(MixedRealityInterop.obj) : error LNK2038: mismatch detected for 'C++/WinRT version': value '2.0.201113.7' doesn't match value '2.0.201217.4' in PCH.MicrosoftOpenXR.h.obj UATHelper: Packaging (HoloLens): MixedRealityInteropHoloLens.lib(CameraImageCapture.obj) : error LNK2038: mismatch detected for 'C++/WinRT version': value '2.0.201113.7' doesn't match value '2.0.201217.4' in PCH.MicrosoftOpenXR.h.obj UATHelper: Packaging (HoloLens): MixedRealityInteropHoloLens.lib(SpatialAnchorHelper.obj) : error LNK2038: mismatch detected for 'C++/WinRT version': value '2.0.201113.7' doesn't match value '2.0.201217.4' in PCH.MicrosoftOpenXR.h.obj UATHelper: Packaging (HoloLens): MixedRealityInteropHoloLens.lib(HandMeshObserver.obj) : error LNK2038: mismatch detected for 'C++/WinRT version': value '2.0.201113.7' doesn't match value '2.0.201217.4' in PCH.MicrosoftOpenXR.h.obj UATHelper: Packaging (HoloLens): MixedRealityInteropHoloLens.lib(MeshObserver.obj) : error LNK2038: mismatch detected for 'C++/WinRT version': value '2.0.201113.7' doesn't match value '2.0.201217.4' in PCH.MicrosoftOpenXR.h.obj UATHelper: Packaging (HoloLens): MixedRealityInteropHoloLens.lib(SceneUnderstandingObserver.obj) : error LNK2038: mismatch detected for 'C++/WinRT version': value '2.0.201113.7' doesn't match value '2.0.201217.4' in PCH.MicrosoftOpenXR.h.obj UATHelper: Packaging (HoloLens): MixedRealityInteropHoloLens.lib(QRCodeObserver.obj) : error LNK2038: mismatch detected for 'C++/WinRT version': value '2.0.201113.7' doesn't match value '2.0.201217.4' in PCH.MicrosoftOpenXR.h.obj UATHelper: Packaging (HoloLens): Creating library F:\ue4\workspace\MsftOpenXRGame\Binaries\HoloLens\MsftOpenXRGamearm64.lib and object F:\ue4\workspace\MsftOpenXRGame\Binaries\HoloLens\MsftOpenXRGamearm64.exp UATHelper: Packaging (HoloLens): F:\ue4\workspace\MsftOpenXRGame\Binaries\HoloLens\MsftOpenXRGamearm64.exe : fatal error LNK1319: 8 mismatches detected UATHelper: Packaging (HoloLens): Took 18.1333085s to run UnrealBuildTool.exe, ExitCode=6 UATHelper: Packaging (HoloLens): UnrealBuildTool failed. See log for more details. (C:\Users\Hawk\AppData\Roaming\Unreal Engine\AutomationTool\Logs\F+ue4+UE_4.26\UBT-MsftOpenXRGame-HoloLens-Development.txt) UATHelper: Packaging (HoloLens): AutomationTool exiting with ExitCode=6 (6) UATHelper: Packaging (HoloLens): BUILD FAILED

Hawkpool commented 3 years ago

MaketPlace plugins versions here : Microsoft OpenXR Plugin v1.1.2 Microsoft Windows Mixed Reality 1.0 Azure Spatial Anchors 1.0 Azure Spatial Anchors for WMR 1.0 Datasmith Content 1.0 and some other default Plugins

Hawkpool commented 3 years ago

I tried to alter the packages.config in NuGetModule folder and rebuild solution in order to fix the version problem , if I turn off the WindowsMixedReality pulgin , package for hololens successed, but do not work well on hololens , able to start up but not work normually. And when I package for hololens with WindowsMixedReality pulgin on, another exception comes. console logs here:

UATHelper: Packaging (HoloLens): LogHttp: Display: Http module shutting down, but needs to wait on 1 outstanding Http requests: UATHelper: Packaging (HoloLens): LogHttp: Display: verb=[GET] url=[https://169.254.232.222:443/] refs=[1] status=Processing UATHelper: Packaging (HoloLens): LogHttp: Display: cleaning up 1 outstanding Http requests. UATHelper: Packaging (HoloLens): LogContentStreaming: Display: There are 1 unreleased StreamingManagers UATHelper: Packaging (HoloLens): Took 13.816607s to run UE4Editor-Cmd.exe, ExitCode=0 UATHelper: Packaging (HoloLens): ** COOK COMMAND COMPLETED ** UATHelper: Packaging (HoloLens): ** STAGE COMMAND STARTED ** UATHelper: Packaging (HoloLens): Creating UE4CommandLine.txt UATHelper: Packaging (HoloLens): Creating Staging Manifest... UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\BaseEditor.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\BaseEditorKeyBindings.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\BaseEditorPerProjectUserSettings.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\BaseEditorSettings.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\BaseLightmass.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\BasePakFileRules.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\Localization\Category.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\Localization\Editor.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\Localization\EditorTutorials.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\Localization\Engine.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\Localization\Keywords.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\Localization\PortableObjectExport.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\Localization\PortableObjectImport.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\Localization\PropertyNames.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\Localization\RepairData.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\Localization\ToolTips.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\UE_4.26\Engine\Config\Localization\WordCount.ini UATHelper: Packaging (HoloLens): Excluding config file F:\ue4\workspace\MsftOpenXRGame\Config\DefaultEditor.ini UATHelper: Packaging (HoloLens): Cleaning Stage Directory: F:\ue4\workspace\MsftOpenXRGame\Saved\StagedBuilds\HoloLens UATHelper: Packaging (HoloLens): Creating pak using staging manifest. UATHelper: Packaging (HoloLens): Executing 1 UnrealPak command... UATHelper: Packaging (HoloLens): Waiting for child processes to complete (1/1) UATHelper: Packaging (HoloLens): Output from: F:\ue4\workspace\MsftOpenXRGame\MsftOpenXRGame.uproject F:\ue4\workspace\MsftOpenXRGame\Saved\StagedBuilds\HoloLens\MsftOpenXRGame\Content\Paks\MsftOpenXRGame-HoloLens.pak -create="C:\Users\Hawk5\AppData\Roaming\Unreal Engine\AutomationTool\Logs\F+ue4+UE_4.26\PakList_MsftOpenXRGame-HoloLens.txt" -cryptokeys=F:\ue 4\workspace\MsftOpenXRGame\Saved\Cooked\HoloLens\MsftOpenXRGame\Metadata\Crypto.json -order=F:\ue4\workspace\MsftOpenXRGame\Build\HoloLens\FileOpenOrder\CookerOpenOrder.log -platform=HoloLens -multiprocess -abslog="C:\Users\Hawk5\AppData\Roaming\Unreal Engine\AutomationTool\Logs\F+ue4+UE_4.26\UnrealPak-MsftOpenXRGame-HoloLens-2021.03.09-18.37.06.txt" UATHelper: Packaging (HoloLens): LogPakFile: Display: Parsing crypto keys from a crypto key cache file UATHelper: Packaging (HoloLens): LogPakFile: Display: Loading response file C:\Users\Hawk5\AppData\Roaming\Unreal Engine\AutomationTool\Logs\F+ue4+UE_4.26\PakList_MsftOpenXRGame-HoloLens.txt UATHelper: Packaging (HoloLens): LogPakFile: Display: Added 1405 entries to add to pak file. UATHelper: Packaging (HoloLens): LogPakFile: Display: Loading pak order file F:\ue4\workspace\MsftOpenXRGame\Build\HoloLens\FileOpenOrder\CookerOpenOrder.log... UATHelper: Packaging (HoloLens): LogPakFile: Display: Finished loading pak order file F:\ue4\workspace\MsftOpenXRGame\Build\HoloLens\FileOpenOrder\CookerOpenOrder.log. UATHelper: Packaging (HoloLens): LogPakFile: Display: Collecting files to add to pak file... UATHelper: Packaging (HoloLens): LogPakFile: Display: Collected 1405 files in 0.01s. UATHelper: Packaging (HoloLens): LogPakFile: Display: Creating pak F:\ue4\workspace\MsftOpenXRGame\Saved\StagedBuilds\HoloLens\MsftOpenXRGame\Content\Paks\MsftOpenXRGame-HoloLens.pak. UATHelper: Packaging (HoloLens): LogDerivedDataCache: Display: Pak cache opened for reading ../../../Engine/DerivedDataCache/Compressed.ddp. UATHelper: Packaging (HoloLens): LogDerivedDataCache: Display: Performance to C:/Users/Hawk5/AppData/Local/UnrealEngine/Common/DerivedDataCache: Latency=0.00ms. RandomReadSpeed=999.00MBs, RandomWriteSpeed=999.00MBs. Assigned SpeedClass 'Local' UATHelper: Packaging (HoloLens): LogPakFile: Display: Added 1405 files, 83844894 bytes total, time 0.48s. UATHelper: Packaging (HoloLens): LogPakFile: Display: PrimaryIndex size: 16974 bytes UATHelper: Packaging (HoloLens): LogPakFile: Display: PathHashIndex size: 36102 bytes UATHelper: Packaging (HoloLens): LogPakFile: Display: FullDirectoryIndex size: 47175 bytes UATHelper: Packaging (HoloLens): LogPakFile: Display: Encryption - DISABLED UATHelper: Packaging (HoloLens): LogPakFile: Display: Unreal pak executed in 0.516516 seconds UATHelper: Packaging (HoloLens): UnrealPak terminated with exit code 0 UATHelper: Packaging (HoloLens): Copying NonUFSFiles to staging directory: F:\ue4\workspace\MsftOpenXRGame\Saved\StagedBuilds\HoloLens UATHelper: Packaging (HoloLens): ** STAGE COMMAND COMPLETED ** UATHelper: Packaging (HoloLens): ** PACKAGE COMMAND STARTED ** UATHelper: Packaging (HoloLens): bStartInVR = True UATHelper: Packaging (HoloLens): Writing cmd line to: F:\ue4\workspace\MsftOpenXRGame\Saved\StagedBuilds\HoloLens\UE4CommandLine.txt UATHelper: Packaging (HoloLens): Creating UE4CommandLine.txt UATHelper: Packaging (HoloLens): Running: C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\makeappx.exe pack /o /f "F:\ue4\workspace\MsftOpenXRGame\Intermediate\Deploy\arm64\MsftOpenXRGame_HoloLens_Development_arm64.pkgmap" /p "F:\ue4\workspace\MsftOpenXRGame\Saved\StagedBuilds\HoloLens\MsftOpenXRGame_HoloLens_Development_arm64.appx" UATHelper: Packaging (HoloLens): Took 0.089001s to run makeappx.exe, ExitCode=1 UATHelper: Packaging (HoloLens): Command failed (Result:1): C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\makeappx.exe pack /o /f "F:\ue4\workspace\MsftOpenXRGame\Intermediate\Deploy\arm64\MsftOpenXRGame_HoloLens_Development_arm64.pkgmap" /p "F:\ue4\workspace\MsftOpenXRGame\Saved\StagedBuilds\HoloLens\MsftOpenXRGame_HoloLens_Development_arm64.a ppx". See logfile for details: 'makeappx-2021.03.09-18.37.08.txt' UATHelper: Packaging (HoloLens): AutomationTool exiting with ExitCode=1 (Error_Unknown) UATHelper: Packaging (HoloLens): BUILD FAILED

fieldsJacksonG commented 3 years ago

A few issues you might be hitting here:

  1. WindowsMixedReality and OpenXR are incompatible with each other - looks like you were successfully able to work around the compile by changing OpenXR's WinRT version to match WMR.
  2. If you have both OpenXR and WMR plugins enabled, the highest priority HMD in BaseEngine.ini's [HMDPluginPriority] list will be selected - Currently WMR will win and your app will not use OpenXR.
  3. The AzureSpatialAnchorsForWMR plugin requires your HMD plugin to be the WMR plugin rather than OpenXR. Multiple AzureSpatialAnchors plugins enabled at the same time will also cause underlying Unreal issues. Instead, you should disable the WindowsMixedReality and AzureSpatialAnchorsForWMR plugins and use the AzureSpatialAnchorsForOpenXR module that exists in this plugin.
Hawkpool commented 3 years ago

First of all, thanks for your answering, now I know where the problem is. I have some other questions.

  1. How could I use the QR tracking function with out OpenXR Plugin?
  2. Whether QR tracking function able to read and print the information from QR Code Image, like url or something?
  3. How could I package this project in a perfect way to make the app work well , (last time I build this app with out WindowsMixedReality and AzureSpatialAnchorsForWMR plugins, the information window was twinkling like crazy, and seems like two cubes stucked in my hand...)
sw5813 commented 3 years ago

Hi @Hawkpool, responding to your questions:

  1. Can you explain more about why you want to use QR tracking without OpenXR? Your two options to get QR tracking with HoloLens 2 in UE 4.26 are to use the Windows Mixed Reality or OpenXR plugins, but OpenXR is the path that all of Mixed Reality at Microsoft is moving towards.
  2. Yes, you can get the url from a QR code. See our documentation for a Blueprint example of how to print the text contained in a QR code as a debug string.
  3. Have you taken a look at the settings for the MsftOpenXRGame sample app in this repo? Try cloning the project and packaging the app as is to see how it runs. I'd also advise checking out the documentation for Unreal development for HoloLens 2 for additional performance recommendations.
Hawkpool commented 3 years ago

Hi @sw5813, Thanks for helping me ,

  1. Base on your response,I think that I might confused the relationship between wmr and openxr. Can I consider that OpenXR is a updated plugin or a replacement of wmr? Which mains that I can replacing wmr with OpenXR in my demo project by add some blueprints and functions, if that's good, I think I'm getting closer to it .
  2. I gonna try agin with that Blueprint example , may be I have lost some detail before.
  3. And yes , I have watched the settings for the MsftOpenXRGame sample app in this repo and packaged the app .I forgot to turn webcam on might be the reason that app doesnt work well ,I gonna double check and repackage.

Some further questions when I using hololens2 :

  1. I'm building an app with a ton of models , its more like a scene rather than just controlling an object. I want to bind the scene with local Spatial anchors but it seems not possible , because I have too much models and lots of them need to add interaction(can not group them all). Because those models have real gps info,so I wander whether I could use them by send in user's real-time gps info by socket or somthing, I tried to manually update player controller's location, which works on pc but not work on headset.

Also I tried to switch the level map in order to update objects location by clicking a button. I add start ARSisson and stop ARSission in every map' bp, sometimes they don's show up normally. But if I remove stop ARSission fuction, maps show up and objects location all base on app starting place instead of now user place. I want the map and the scene controll all rebuild and update form user's real-time location by click a button. Is that possible?

I wander whether I'm going the right direction, please give me some suggestions.

Thank you very much. Have good day~

zidekkamil commented 3 years ago

Hello all,

Two weeks of trying to get some demo in Hololens 2 with OpenXR old mixed reality plugin partially works (camera PV part not working):

The first video is official demo Microsoft OpenXR app: Used microsoft mixed reality tool application to get all preview updates this is without unreal maybee in c#, flickering of object is terrible....

https://tukesk-my.sharepoint.com/:v:/g/personal/kamil_zidek_tuke_sk/ETodoyTfXwFAooEi50OSoIUBe0NL5LkPqcrTvhvjxoEfEQ?e=HxtRBR

The second video is Unreal official demo with OpenXR, this compiled in unreal, horrible flickering and position mess.....

https://tukesk-my.sharepoint.com/:v:/g/personal/kamil_zidek_tuke_sk/EQDVAj5wF7BLgRcVinn1TrMBgMVB1Gm5r_LluED-z562EA?e=JTqo6b

Not implemented any line of my code official demo used in video.

Please if someone get acceptable result with demos provides in official pages please reply, how? :-)

BRG Kamil

brycehutchings commented 3 years ago

Hello all,

Two weeks of trying to get some demo in Hololens 2 with OpenXR old mixed reality plugin partially works (camera PV part not working):

The first video is official demo Microsoft OpenXR app: Used microsoft mixed reality tool application to get all preview updates this is without unreal maybee in c#, flickering of object is terrible....

https://tukesk-my.sharepoint.com/:v:/g/personal/kamil_zidek_tuke_sk/ETodoyTfXwFAooEi50OSoIUBe0NL5LkPqcrTvhvjxoEfEQ?e=HxtRBR

The second video is Unreal official demo with OpenXR, this compiled in unreal, horrible flickering and position mess.....

https://tukesk-my.sharepoint.com/:v:/g/personal/kamil_zidek_tuke_sk/EQDVAj5wF7BLgRcVinn1TrMBgMVB1Gm5r_LluED-z562EA?e=JTqo6b

Not implemented any line of my code official demo used in video.

Please if someone get acceptable result with demos provides in official pages please reply, how? :-)

BRG Kamil

Could you open a separate issue for this? It looks like you may have enabled the "Jiggle view rotations" feature in the OpenXR Developer Tools app for the second video.

sw5813 commented 3 years ago

Hi @sw5813, Thanks for helping me ,

  1. Base on your response,I think that I might confused the relationship between wmr and openxr. Can I consider that OpenXR is a updated plugin or a replacement of wmr? Which mains that I can replacing wmr with OpenXR in my demo project by add some blueprints and functions, if that's good, I think I'm getting closer to it .
  2. I gonna try agin with that Blueprint example , may be I have lost some detail before.
  3. And yes , I have watched the settings for the MsftOpenXRGame sample app in this repo and packaged the app .I forgot to turn webcam on might be the reason that app doesnt work well ,I gonna double check and repackage.

Some further questions when I using hololens2 :

  1. I'm building an app with a ton of models , its more like a scene rather than just controlling an object. I want to bind the scene with local Spatial anchors but it seems not possible , because I have too much models and lots of them need to add interaction(can not group them all). Because those models have real gps info,so I wander whether I could use them by send in user's real-time gps info by socket or somthing, I tried to manually update player controller's location, which works on pc but not work on headset.

Also I tried to switch the level map in order to update objects location by clicking a button. I add start ARSisson and stop ARSission in every map' bp, sometimes they don's show up normally. But if I remove stop ARSission fuction, maps show up and objects location all base on app starting place instead of now user place. I want the map and the scene controll all rebuild and update form user's real-time location by click a button. Is that possible?

I wander whether I'm going the right direction, please give me some suggestions.

Thank you very much. Have good day~

Hi @Hawkpool, regarding your first question- yes, you can consider the Microsoft OpenXR plugin to be an updated replacement for the Windows Mixed Reality plugin.

Regarding your second question, can you please open a new issue for the question? To make issues more searchable, we'd like to limit each issue to addressing one question.

Hawkpool commented 3 years ago

Hi @brycehutchings , Thanks for trying to figure out the problems, It seems I have met the same problems.But I still don't know how to fix it.I tried to reset all settings in OpenXR Developer Tools app, the app seems better. That really works on Unreal official demo with OpenXR. If I want to update my old app by replacing wmr with OpenXr, what should I do? Open this and turn on that? What else do I need to pay attention to?

And I have opened a new issue for other question,take a check. Thanks~

sw5813 commented 3 years ago

Hi @Hawkpool, see this doc for a list of new APIs in UE 4.26+ that replace WMR-specific APIs: https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unreal/unreal-porting

crowbar27 commented 2 years ago

official demo with OpenXR. If I want to update my old app by replacing wmr with OpenXr, what should I do? Open this and turn on that? What else do I need to pay attention to?

Did you ever figure out a viable upgrade path to OpenXR? I have the same problem and although I removed the WindowsMixedReality plugin, rebuilt everything, even cleaned all of the intermediate directories, it still wants to link to "MixedRealityInteropHoloLens.lib" when packaging for HoloLens.