search

Azure / azure-batch-maya

Cloud rendering from Maya using Azure Batch
Other
27 stars 12 forks source link

Asset path length on linux limited to 260 chars (~144 usable) #29

Open davidkydd opened 6 years ago

davidkydd commented 6 years ago

There is a limitation in the current version of AzCopy (6.0.0) which we use for downloading of assets in the job prep task, in which paths are limited to 260 chars.

Due to the path prefix which batch prepends to assets as they are downloaded to the rendering nodes, the available max path length for assets when they are added to a job / scene is significantly shorter than this at ~144 characters.

The error is surfaced in the stderr of the jobprep task associated with a job, which will contain the error message "The specified path, file name, or both are too long for file"

The AzCopy team are aware of this and are increasing the limit up to 1024 characters in their next release, at which point the link used in the jobprep task to download azcopy (aka.ms/downloadazcopyprlinux) should automatically redirect to the new version

psyrus commented 6 years ago

Azcopy is now at v7.1.0, can this issue be considered solved at this point?

annatisch commented 6 years ago

No quite - while azcopy has been updated - we have not yet removed the "safety net" logic from the plugin that prevents paths getting too long on the servers. So even though azcopy now supports long paths, it's not yet being exercised. Once long paths have been tested to be fully supported on both the Windows and Linux images, we'll send out a new release at which time we can close this off.