Creating large usdz files produces invalid files without error

[ShaddyDC]

Description of Issue

Trying to create a usdz file from a large usd folder with UsdUtilsCreateNewUsdzPackage creates an invalid file. Opening it gives this error, both in omniverse and usdview.

The file is not a valid zip file anymore either. We suspect that the issue is that the file size is over the 32 bit limit and the ZIP64 extension is not supported, which is fine, but it would be nice if we then received an error code. Actually being able to create those large archives would be even better, of course.

I can provide some concrete reproduction files, but given the file size we're talking about and the suspected cause, I figured it's probably not worth it. I can also give a concrete code snippet to reproduce if needed.

Steps to Reproduce

1. Create a large, valid folder that can be turned into an usdz archive
2. Call UsdUtilsCreateNewUsdzPackage(path, outPath) on the folder, which will return true
3. Ensure the created usdz is over 4 GB. If not, go back to step 1
4. Try to open the created usdz file with usdview or unzip, both of which fail

System Information (OS, Hardware)

Replicated both on debian-bookworm in docker and on NixOS 24.05.20240515.33d1e75.

Package Versions

23.05, 23.11

Build Flags

    "-DPXR_BUILD_ALEMBIC_PLUGIN=ON"
    "-DPXR_BUILD_DRACO_PLUGIN=ON"
    "-DPXR_BUILD_EMBREE_PLUGIN=ON"
    "-DPXR_BUILD_EXAMPLES=OFF"
    "-DPXR_BUILD_IMAGING=ON"
    "-DPXR_BUILD_MONOLITHIC=ON"
    "-DPXR_BUILD_TESTS=OFF"
    "-DPXR_BUILD_TUTORIALS=OFF"
    "-DPXR_BUILD_USD_IMAGING=ON"
    "-DPXR_BUILD_DOCUMENTATION=OFF"
    "-DPXR_BUILD_PYTHON_DOCUMENTATION=OFF"
    "-DPXR_BUILD_USDVIEW=OFF"
    "-DPXR_BUILD_USD_TOOLS=OFF"
    "-DPXR_ENABLE_MATERIALX_SUPPORT=OFF"
    "-DPXR_ENABLE_OSL_SUPPORT=OFF"

[jesschimein]

Filed as internal issue #USD-9711

[dgovil]

Yeah this is a good callout. I don't think there are any plans to extend USDZ to support zip64, but perhaps someone can add a warning like you suggested if any individual file or the total file size is going to be more than the 32 bit range.

[ShaddyDC]

Thanks for looking into the issue! It would be very useful for us if there were support in USDZ for large files like this in the future, as we regularly work with 3D data sets on that scale and want to offer them to users in the USD ecosystem. Having a warning or preferably have the function fail would help a lot in working around it already, however.

[asluk]

Having a warning or preferably have the function fail would help a lot in working around it already, however.

From my naive perspective-- could you add a file size check in your pipeline after the call to write out the USDZ?

[ShaddyDC]

Yes, that's probably the best option at the moment, but it doesn't feel like that should be the place to address this problem, more like a band-aid. Thank you for the suggestion though!

[asluk]

for sure-- not suggesting it as any more of a bandaid, but wanted to get a sense of how blocking of an issue this is-- thanks!