Support for iOS13Beta - Githubissues

wave-electron commented 5 years ago

@kcoley I have found issues with my existing workflow using gltf2usd and iOS13Beta. Many working usdz files were no longer working in iOS13Beta. One solution I found worked was merging the multiple mesh into a single mesh. This fixes many of the usdz files not rendering correctly. I also used Apple’s fixedOpacity to fix some where transparency layers were no longer rendering correctly. However, I have some complex usdz with animations and multiple mesh that I can no longer convert using my current workflow. Apple apparently does have a bug with buffers which is fixed internally but not yet released.

kcoley commented 5 years ago

@wave-electron do you have a sample glTF that works differently on iOS 13 Beta vs iOS 12?

wave-electron commented 5 years ago

@kcoley Yes. multiple meshes do not render correctly. I’ll make you a very simple glTF file. Do you want usdz, usda etc?

wave-electron commented 5 years ago

@ Examples like this that work in iOS12 https://usdzshare.com/?ug-gallery=photo-detail&photo_id=938 don’t work in iOS13... and I haven’t worked out how to fix examples like these because merging mesh is not an option.

kcoley commented 5 years ago

@wave-electron thanks, I'll see if there is any other possibility here

wave-electron commented 5 years ago

@kcoley iOS13 now supports nested usdz. This is good way to package family of usdz that go together like a lounge setting... as each item can be individually moved around inside AR Quick Look & Reality Composer. What would be required for gltf2usd to support such a structure?

wave-electron commented 5 years ago

@kcoley I've attached a glTF that is working correctly in iOS12 but not iOS13Beta. not-working-iOS13.zip

kcoley commented 5 years ago

@wave-electron thanks for the sample glTF. This looks like a bug in iOS 13, since the generated usdz file is valid from what I can tell. It may be good to raise this as an issue to Apple.

As for nested usdz files, does this mean zipping several generated usdz files into a zip archive? Or are you thinking of generated nested usdz files from a glTF file? This might be outside the scope of gltf2usd, though if it is the former, you can probably run a script which is able to call the converter multiple times on several generated usdz files and then zip them together.

wave-electron commented 5 years ago

@kcoley I found this in pixar usdz docs

(Nesting can occur when one usdz file embeds another, e.g. @set.usdz[areas/shire.usdz[architecture/BilboHouse/Table.

It could be a bunch of separate usdz files bundled inside a single parent usdz file. I’ll do some more digging and let you know.

wave-electron commented 5 years ago

@kcoley

I found out how nested usdz works. It does require a separate tool to package input the usdz files into a new usdz archive. This is what the usda file looks like. Apple provides a tool which is bundled with the latest standalone usdpython. usdzcreateassetlib nested.usdz cube.usdz sphere.usdz

would generate a nested.usda file like below.


(
    defaultPrim = "Object"
    upAxis = "Y"
    customLayerData = {
        asset[] assetLibrary = [@cube.usdz@, @sphere.usdz@]
    }
)

def Xform "Object" (
    variants = {
        string Assets = "cube"
    }
    prepend variantSets = "Assets"
)
{
    variantSet "Assets" = {
        "cube" (
            prepend references = @cube.usdz@
        ) {

        }
        "sphere" (
            prepend references = @sphere.usdz@
        ) {

        }
    }
}

kcoley commented 5 years ago

@wave-electron thanks for looking into this!

wave-electron commented 5 years ago

@kcoley Apple’s latest release of usdpython & usdzconvert python command line tool ( version 058 released today) is performing much better than the original release. I did some tests and it handles transparency now, plus glTF conversions with animations are working better. I tested the same file above and it converted without the same animation error. I did notice it may have an issue with metallic image assignment for PBR materials - didn’t seem to work on a couple of tests I did but was working with standard metallic = 1 assignment. Probably worth looking at.

wave-electron commented 5 years ago

@kcoley I also found out using the Pixar usdzip -- asset or --arkitAsset with the same usda structure above would also create nested usdz files

silvainSayduck commented 5 years ago

Hi, just jumping here to say that we face the same issue. Our USDZ files are now not working with iOS 13 (or MacOS Catalina). Here is what we have found out so far:

USDZs converted with gltf2usd do not work anymore
When using Apple's USDZ Tools from last year, USDZs created with gtlf2usd cannot even be unzipped at all, an error happens
When using Apple's USDZ Tools from this month, although USDZs created with gtlf2usd do not work, they can be unpacked, and their .usdc files seem fine
When using Apple's offcial usdzconvert command line tool, everything is fine.

See attached an example gltf, converted in the three different options: 2019.06.26.painting_test.zip

wave-electron commented 5 years ago

@silvainSayduck

Some other things I’ve figured out.

Skinned animations work using gltf2usd but not using usdzconvert on iOS13. I’ve filed a radar with Apple on this.
Usdzconvert is now best for doing animations with simple translation or transforms. However, orientation will be in some cases be different on iOS12 whilst correct in iOS13.
I think iOS13 has a bug with handing metallic image uv. Parameter settings for metallic seem to work. But still need to verify again.

silvainSayduck commented 5 years ago

Nice, thanks for sharing @wave-electron! A couple more things we've seen/heard that might be worth sharing too:

In the USDZ tools from Apple's site, one of the tools is called fixOpacity. From one of the WWDC talks, any objects that had transparency that were made for iOS 12 will need to be run through there. [I've actually tried running the paintings above through fixOpacity, but that didn't actually do anything, so I'm guessing it's not the issue in this case]
We've been notified of a bug with usdzconvert where if you have a gltf that uses a single texture for occlusion/roughness/metalness, then it will not appear correctly on iOS 12 devices when converted. However it looks fine in AR Quick Look on Desktop. That bug has been reported to Apple too.

ephb commented 4 years ago

I looked into iOS 13 some more and it seems to wrongly switch to transparent rendering when PNG textures are used even without transparency. A workaround that worked for me is is to use jpg Textures for now. Transparency z-sorting problems can by worked around by using multiple Materials.

The one thing I haven't found a solution for so far is that meshes with skinned animation seem to loose their normal information. They appear faceted like they do not have any normal information at all. Also only animations that are 10 second or less do automatically loop now. Others display a progress bar and play only once. Meshes that have regular animations do not seem to be affected by this. On the bright side though shadows are now rendered every frame and we are no stuck with the static one form the first frame anymore.

kcoley / gltf2usd

Support for iOS13Beta #162