Mysterious boost build error

[jcowles]

Description of Issue

After running a clean v0.8.1 build, it failed on boost with the following error:

ERROR: [Error 32] The process cannot access the file because it is being used by another process: 'boost_1_61_0.tar.gz'

The directory was empty and the system was left idle while building.

Steps to Reproduce

python build_scripts\build_usd.py --build-monolithic --no-tests --no-docs --no-python --no-imaging --no-ptex --no-embree --alembic --no-hdf5 build_monolithic

System Information (OS, Hardware)

Windows

Package Versions

Build Flags

python build_scripts\build_usd.py --build-monolithic --no-tests --no-docs --no-python --no-imaging --no-ptex --no-embree --alembic --no-hdf5 build_monolithic

Building with settings: USD source directory C:\src\usd\USD-v0.8 USD install directory C:\src\usd\USD-v0.8\build_monolithic 3rd-party source directory C:\src\usd\USD-v0.8\build_monolithic\src 3rd-party install directory C:\src\usd\USD-v0.8\build_monolithic Build directory C:\src\usd\USD-v0.8\build_monolithic\build

Building Monolithic shared library Imaging Off Ptex support: Off UsdImaging Off Python support Off Documentation Off Tests Off Alembic Plugin On HDF5 support: Off Maya Plugin Off Katana Plugin Off Houdini Plugin Off

Dependencies                zlib, boost, TBB, OpenEXR, Alembic

STATUS: Installing zlib... STATUS: Installing boost... ERROR: [Error 32] The process cannot access the file because it is being used by another process: 'boost_1_61_0.tar.gz'

[jcowles]

Seems to be reproducible, I just hit it again -- so far it's 2 for 2

[jcowles]

3 for 3

[jcowles]

The archive seems to extract correctly, here's the output from a verbose build:

STATUS: Installing boost... INFO: C:\src\usd\USD-v0.8\build_monolithic\src\boost_1_61_0.tar.gz already exists, skipping download INFO: Extracting archive to C:\src\usd\USD-v0.8\build_monolithic\src\boost_1_61_0 ERROR: [Error 32] The process cannot access the file because it is being used by another process: 'boost_1_61_0.tar.gz'

[jcowles]

I added some extra debug info:

STATUS: Installing boost... INFO: C:\src\usd\USD-v0.8\build_monolithic\src\boost_1_61_0.tar.gz already exists, skipping download rootDir: boost_1_61_0 extractedPath: C:\src\usd\USD-v0.8\build_monolithic\src\boost_1_61_0 INFO: Extracting archive to C:\src\usd\USD-v0.8\build_monolithic\src\boost_1_61_0 tmpExtractedPath: C:\src\usd\USD-v0.8\build_monolithic\src\extract_dir rmtree tmp before ERROR: [Errno 2] No such file or directory: 'C:\src\usd\USD-v0.8\build_monolithic\src\extract_dir\boost_1_61_0\libs\geometry\doc\html\geometry\reference\spatial_indexes\boostgeometryindexrtree\rtree_parameters_type_constindexable_getter_constvalue_equal_const____allocator_typeconst.html'

Looks like maybe a bad archive?

If archives from SourceForge are such a problem, how about getting it from a different location?

[jcowles]

Actually... the archive seems fine. I'm not sure what's going on.

[c64kernal]

Hey Jeremy, we've hit this problem here too -- try building to a location outside of your source tree...

[jcowles]

Thanks! I unblocked myself by manually extracting boost to the expected directory.

The build is running now, so I'm going to let it keep going and see how it goes.

[meshula]

Bet you could solve the issue by logging off and on again. "The process cannot access the file because it is being used by another process" ---- something terminated and managed to not release file handles.

[jcowles]

Hmm, but I was able to delete the files manually and repro again...

[sunyab]

We've hit this in some cases in our local testing even after deleting the directory and rebooting, and I'm not sure what's going on yet. This only seems to happen if you build to a location inside the USD source tree; if you build somewhere outside the tree the problem seems to go away.

[jtran56]

Filed as internal issue #152499.

[jcowles]

This seems to be fixed in v0.8.2 -- at least my first build successfully made it past Boost.

[sunyab]

I ran into this again last night and did some more digging. I think the real problem here is that we're running into a 260 character path limit on NTFS. In @jcowles' earlier post, the path to that .html file beneath the boost root directory is 207 characters long!

So, if the build script is extracting to some path that is >= 53 characters long it will fail and because of a bug in the build script we'll get that "cannot access file" error instead of seeing the real error from the extraction.

A bug report for this was filed with boost here: https://svn.boost.org/trac10/ticket/11677 but the last activity was 2 years ago. Since the problematic file is in a component that we don't use, one possible fix might be to exclude it from the extraction.

[asluk]

Yikes, I run into this problem a lot when dealing with auto-generated files for protobuf language bindings on Windows. I work around it by mapping the root of my tree to a drive letter to shorten the paths.