Open argallegos opened 3 years ago
According to my issue in https://community.cesium.com/t/does-this-plugin-support-hololens-platform-packaging/14095/3, I post my packaging record here. Although there is no error, there are some warnings.
One more bug report regarding the HoloLens: https://community.cesium.com/t/no-terrain-shown-on-hololens-2/14250
I'm going to try to run a simple Cesium for Unreal level on the HoloLens 2, and I'll post my findings in this issue.
Documenting what I've tried and what the results were, even though I haven't been able to figure out the problem:
There are at least four ways of testing HoloLens 2 applications:
I was able to run the application with 1 and 2. More on that in a moment.
For 3, I get this error in Unreal:
LogPlayLevel: Warning: WARNING: Project will be packaged to support deployment to remote HoloLens device http://169.254.220.62/.
LogPlayLevel: Error: ERROR: Device connection failed: Requesting operating system information,
I know Unreal can talk to the device, because I can turn it off or restart it through Unreal. It fails to deploy the application, though.
For 4, I can't launch Holographic Remoting. I get this error:
Initialization Failure
Failed to initialize. Please make sure that Windows Holographic is installed on your system. Windows Holographic will be installed automatically when you attach your Head-mounted Display.
I haven't been able to figure out why Windows thinks Windows Holographic isn't installed. I'll start by trying to update Windows to get version 21H1, but I don't know how long that update will take, so I'm not doing it right now.
Back to 1 and 2 (testing on the HoloLens and Emulator):
Just as forum users described, I can see non-Cesium Actors in the level just fine, but I can't see any tilesets. I know from printing messages to the screen that the constructors and BeginPlay()
functions of ACesium3DTileset
and ACesiumGeoreference
aren't being called on the HoloLens or the Emulator.
The process of retrieving logs can be mildly finicky, but it eventually works. Here are the errors I got on the HoloLens:
[2021.07.01-19.29.00:238][ 0]LogStreaming: Error: Couldn't find file for package /Script/CesiumRuntime requested by async loading code. NameToLoad: /Script/CesiumRuntime
[2021.07.01-19.29.00:238][ 0]LogStreaming: Error: Found 1 dependent packages...
[2021.07.01-19.29.00:238][ 0]LogStreaming: Error: /Game/Main
[2021.07.01-19.29.00:238][ 0]LogPackageName: Error: DoesPackageExist: DoesPackageExist FAILED: '/CesiumForUnreal/CesiumCreditSystemBP' is not a standard unreal filename or a long path name. Reason: Path does not start with a valid root. Path must begin with: '/Engine/', '/Game/', '/WindowsMixedReality/', '/Takes/', '/Paper2D/', '/AnimationSharing/', '/GeometryMode/', '/SpeedTreeImporter/', '/DatasmithContent/', '/ChaosClothEditor/', '/GeometryProcessing/', '/GeometryCollectionPlugin/', '/ChaosSolverPlugin/', '/ChaosNiagara/', '/Niagara/', '/MotoSynth/', '/MediaCompositing/', '/Synthesis/', '/AudioSynesthesia/', '/WindowsMixedReality/', '/Takes/', '/Paper2D/', '/AnimationSharing/', '/GeometryMode/', '/SpeedTreeImporter/', '/DatasmithContent/', '/ChaosClothEditor/', '/GeometryProcessing/', '/GeometryCollectionPlugin/', '/ChaosSolverPlugin/', '/ChaosNiagara/', '/Niagara/', '/MotoSynth/', '/MediaCompositing/', '/Synthesis/', '/AudioSynesthesia/', '/WindowsMixedReality/', '/Takes/', '/Paper2D/', '/AnimationSharing/', '/GeometryMode/', '/SpeedTreeImporter/', '/DatasmithContent/', '/ChaosClothEditor/', '/GeometryProcessing/', '/GeometryCollectionPlugin/', '/ChaosSolverPlugin/', '/ChaosNiagara/', '/Niagara/', '/MotoSynth/', '/MediaCompositing/', '/Synthesis/', '/AudioSynesthesia/', '/Config/', '/Script/', '/Extra/', '/Memory/', or '/Temp/'
[2021.07.01-19.29.00:238][ 0]LogStreaming: Error: Couldn't find file for package /CesiumForUnreal/CesiumCreditSystemBP requested by async loading code. NameToLoad: /CesiumForUnreal/CesiumCreditSystemBP
[2021.07.01-19.29.00:238][ 0]LogStreaming: Error: Found 1 dependent packages...
[2021.07.01-19.29.00:238][ 0]LogStreaming: Error: /Game/Main
[2021.07.01-19.29.00:248][ 0]LogStreaming: Error: ****DumpDependencies [Dependencies]:
[2021.07.01-19.29.00:248][ 0]LogStreaming: Error: Export 6 /Game/Main.Main:PersistentLevel.Cube
[2021.07.01-19.29.00:248][ 0]LogStreaming: Error: Linker is ../../../CesiumHoloLens/Content/Main.umap
[2021.07.01-19.29.00:248][ 0]LogStreaming: Error: Dep C_BEFORE_S Export 8 /Game/Main.Main:PersistentLevel.CesiumGeoreferenceDefault (class CesiumGeoreference)
[2021.07.01-19.29.00:248][ 0]LogStreaming: Error: Dep C_BEFORE_S Export 7 /Game/Main.Main:PersistentLevel.CesiumCreditSystemDefault (class CesiumCreditSystemBP_C)
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep C_BEFORE_S Export 6 /Game/Main.Main:PersistentLevel.Cube.TileSet (class Cesium3DTilesetRoot)
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep S_BEFORE_C Import 12 /Script/CesiumRuntime.Cesium3DTileset
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep S_BEFORE_C Import 8 /Script/CesiumRuntime.Default__Cesium3DTileset
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep S_BEFORE_C Import 9 /Script/CesiumRuntime.Default__Cesium3DTileset:TileSet
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep C_BEFORE_C Export 14 /Game/Main.Main:PersistentLevel (class Level)
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Missing Dependency, request for /Script/CesiumRuntime.Cesium3DTileset but it hasn't been created yet.
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Could not find class Cesium3DTileset to create Cube
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: ****DumpDependencies [Dependencies]:
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Export 7 /Game/Main.Main:PersistentLevel.Cube.TileSet
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Linker is ../../../CesiumHoloLens/Content/Main.umap
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep C_BEFORE_S Import 8 /Script/CesiumRuntime.Default__Cesium3DTileset
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep S_BEFORE_C Import 13 /Script/CesiumRuntime.Cesium3DTilesetRoot
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep S_BEFORE_C Import 9 /Script/CesiumRuntime.Default__Cesium3DTileset:TileSet
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep C_BEFORE_C Export 5 /Game/Main.Main:PersistentLevel.Cube (class Cesium3DTileset)
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Missing Dependency, request for /Script/CesiumRuntime.Cesium3DTilesetRoot but it hasn't been created yet.
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Could not find class Cesium3DTilesetRoot to create TileSet
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: ****DumpDependencies [Dependencies]:
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Export 8 /Game/Main.Main:PersistentLevel.CesiumCreditSystemDefault
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Linker is ../../../CesiumHoloLens/Content/Main.umap
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep C_BEFORE_S Export 20 /Game/Main.Main:PersistentLevel.CesiumCreditSystemDefault.DefaultSceneRoot (class SceneComponent)
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep S_BEFORE_C Import 2 /CesiumForUnreal/CesiumCreditSystemBP.CesiumCreditSystemBP_C
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep S_BEFORE_C Import 10 /CesiumForUnreal/CesiumCreditSystemBP.Default__CesiumCreditSystemBP_C
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep C_BEFORE_C Export 14 /Game/Main.Main:PersistentLevel (class Level)
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Missing Dependency, request for /CesiumForUnreal/CesiumCreditSystemBP.CesiumCreditSystemBP_C but it hasn't been created yet.
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Could not find class CesiumCreditSystemBP_C to create CesiumCreditSystemDefault
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: ****DumpDependencies [Dependencies]:
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Export 9 /Game/Main.Main:PersistentLevel.CesiumGeoreferenceDefault
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Linker is ../../../CesiumHoloLens/Content/Main.umap
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep S_BEFORE_C Import 14 /Script/CesiumRuntime.CesiumGeoreference
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep S_BEFORE_C Import 11 /Script/CesiumRuntime.Default__CesiumGeoreference
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Dep C_BEFORE_C Export 14 /Game/Main.Main:PersistentLevel (class Level)
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Missing Dependency, request for /Script/CesiumRuntime.CesiumGeoreference but it hasn't been created yet.
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Could not find class CesiumGeoreference to create CesiumGeoreferenceDefault
[2021.07.01-19.29.00:253][ 0]LogStreaming: Error: Could not find outer CesiumCreditSystemDefault to create DefaultSceneRoot
Also worth noting that when packaging the application for HoloLens, I get the following warnings (similar to the log posted here):
UATHelper: Packaging (HoloLens): LogInit: Display: LogSavePackage: Warning: /Game/Main imported Serialize:/CesiumForUnreal/CesiumCreditSystemBP.CesiumCreditSystemBP_C, but it was never saved as an export.
UATHelper: Packaging (HoloLens): LogInit: Display: LogSavePackage: Warning: /Game/Main imported Serialize:/CesiumForUnreal/CesiumCreditSystemBP.Default__CesiumCreditSystemBP_C, but it was never saved as an export.
UATHelper: Packaging (HoloLens): LogInit: Display: LogSavePackage: Warning: /Game/Main imported Serialize:/CesiumForUnreal/CesiumCreditSystemBP, but it was never saved as an export.
UATHelper: Packaging (HoloLens): LogInit: Display: LogSavePackage: Warning: /Game/Main imported Serialize:/CesiumForUnreal/CesiumCreditSystemBP.CesiumCreditSystemBP_C:DefaultSceneRoot_GEN_VARIABLE, but it was never saved as an export.
I didn't attempt to change cesium-unreal/Source/CesiumRuntime/CesiumRuntime.Build.cs
or cesium-unreal/extern/CMakeLists
. That might be the obvious problem (HoloLens has an ARM architecture), but I would need help understanding how to fix it.
If anyone has a better grasp of the errors and warnings in the logs above, let me know.
Thanks for the writeup @getinthedamnbox. I suspect if you scroll up further in the log, you'll see the Cesium for Unreal plugin isn't being loaded at all. Can you confirm that? I'm not really sure why that would be, though. You said the HoloLens is ARM. It's 64-bit ARM, right? Is it Android? Does it use the same toolchain as regular Android builds?
I suspect if you scroll up further in the log, you'll see the Cesium for Unreal plugin isn't being loaded at all. Can you confirm that?
You're right, I don't see it listed as one of the mounted plugins. Here's the full log: CesiumHoloLens-backup-2021.07.01-19.29.14.log
And here's the log from packaging the application for HoloLens: CesiumHoloLens.log
You said the HoloLens is ARM. It's 64-bit ARM, right? Is it Android? Does it use the same toolchain as regular Android builds?
It's 64-bit ARM, but to be honest I don't know enough about toolchains to answer your other questions. What I know is that in order to package and deploy the application, I had to ensure Visual Studio Build Tools had the component MSVC v142 - VS 2019 C++ ARM64 build tools (v14.26)
installed.
Thanks @getinthedamnbox. Yeah that definitely doesn't sound like "normal" Android, at least. So we probably need a special build of cesium-native.
@kring Alban had some insights:
If you plan to run on the device, you need to have an ARM64 build of Cesium.
When you run the application using holoremoting, it basically runs on your computer, and streams the 3d render in realtime like a video. On the other hand, the HL2 sends its interactions to the device, and you close the loop... In that mode (Tethered or not - I prefer to use Wifi), the app has to be packaged in Win64, so it's very easy to do so using the current Cesium plugin. (I did an app for GTC showcasing that).
But it you plan to deploy and run it "on" the device (meaning without any computer to generate the 3D content), your app needs to be packaged for Hololens2, that expects an ARM64 build. It's fine with most UE libraries, but any included external lib needs an ARM64 version.
I had to do that to port the GeoReferencing plugin I wrote for 4.27. It relies on PROJ, so I had to build PROJ for ARM64 to make it run. Fortunately, it was not that hard when using vcpkg toolchain.
Then, as usual, you need to edit your 3rd party libs build.cs to point to the right libs, with a distinction on architecture. See this sample file for the proj.build.cs
else if (Target.Platform == UnrealTargetPlatform.HoloLens) { if (Target.WindowsPlatform.Architecture == WindowsArchitecture.x64) // emulation target, bBuildForEmulation { PublicSystemIncludePaths.Add(Path.Combine(ModuleDirectory, VcPkgInstalled, "overlay-x64-uwp", "include"));
string LibPath = Path.Combine(ModuleDirectory, VcPkgInstalled, "overlay-x64-uwp", "lib");
PublicAdditionalLibraries.Add(Path.Combine(LibPath, "proj.lib"));
PublicAdditionalLibraries.Add(Path.Combine(LibPath, "sqlite3.lib"));
}
else if (Target.WindowsPlatform.Architecture == WindowsArchitecture.ARM64) // device target, bBuildForDevice
{
PublicSystemIncludePaths.Add(Path.Combine(ModuleDirectory, VcPkgInstalled, "overlay-arm64-uwp", "include"));
string LibPath = Path.Combine(ModuleDirectory, VcPkgInstalled, "overlay-arm64-uwp", "lib");
PublicAdditionalLibraries.Add(Path.Combine(LibPath, "proj.lib"));
PublicAdditionalLibraries.Add(Path.Combine(LibPath, "sqlite3.lib"));
}
else
{
throw new System.Exception("Unknown architecture for HoloLens platform!");
}
}
> Don't be afraid by the "uwp". This is specific to PROJ to allow it to access its data file on a sandboxed device.
> Proj.Build.cs
// Copyright Epic Games, Inc. All Rights Reserved.
using System.IO; using UnrealBuildTool;
> If you want to pick up some code, it's on UE Github there : https://github.com/EpicGames/UnrealEngine/tree/4.27/Engine/Plugins/Runtime/GeoReferencing/Source/GeoReferencing
> Having a build of cesium for ARM64 might be complicated, because you are using lots of external libs.
> But once done, it will be great. I hope the HL2 GPU will be able to deal with this geometry, without raising too much the screen space error value.
any update regarding the issue
No update. If someone wants to attempt this, follow the Developer Setup instructions, adjusting them to build cesium-native for the HoloLens platform, probably by defining a cmake toolchain. It is not currently on our roadmap, but we would welcome a pull request from the community.
Sure will try it. Will this work remotely without packaging?
After running camke files for Cesium native. what steps i have to take for Hololens 2 plugin. please update me
Got this html file running after following the process for cesium native. Now what to do?
If we knew the exact steps, it would already be done. I'm afraid I don't know, and you'll have to figure it out, or wait for someone else to figure it out. From what I can see in your VS Code window, you're just building cesium-native for Windows, so that's not going to help. You need to build it for HoloLens by configuring a toolchain appropriately, but I don't know any more details than that.
I'm not sure what that Eigen3 page is. It's not part of cesium-native, but maybe one of our third-party libraries uses it indirectly.
Great. Just want to confirm is this for desktop hologram or android hologram?
Although i think its for hologram (doesn't matter platform) but want to clear ambiguity of my senior.
Can I access cesium for unreal plugin while connecting HoloLens remotely?
Yes, Cesium for Unreal should already working with HoloLens remoting.
Great
Possibly related to the android build issues (https://github.com/CesiumGS/cesium-unreal/issues/190 https://github.com/CesiumGS/cesium-unreal/issues/472 ), a user on the forum reported issues packaging for HoloLens. Another forum user reported similar issues with launching a packaged project, indicating that the problem arose after adding Cesium World Terrain.
I don't think Hololens has been extensively tested with the plugin, but as far as I know it should work in general. Please add any relevant logs or errors to this issue.