search

shaka-project / shaka-packager

A media packaging and development framework for VOD and Live DASH and HLS applications, supporting Common Encryption for Widevine and other DRM Systems.
https://shaka-project.github.io/shaka-packager/
Other
1.9k stars 496 forks source link

Flaky build failure on Windows (shared library configuration) #1341

Closed joeyparrish closed 4 months ago

joeyparrish commented 4 months ago

The windows shared build is failing on occasion with:

LINK : fatal error LNK1104: cannot open file 'D:\a\shaka-packager\shaka-packager\build\packager\Release\libpackager.lib' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_static.vcxproj]

It then passes when re-run. I think this indicates a race condition somewhere in the parallel build, but I'm not sure of the nature of it.

Here is a failed build: https://github.com/shaka-project/shaka-packager/actions/runs/7906044243/job/21580055244?pr=1340 Followed by a successful rerun: https://github.com/shaka-project/shaka-packager/actions/runs/7906044243/job/21585322587?pr=1340

Failed build

Here are log snippets from the failed build:

Building libpackager_shared

Wed, 14 Feb 2024 19:24:15 GMT
     Creating library D:/a/shaka-packager/shaka-packager/build/packager/Release/libpackager.lib and object D:/a/shaka-packager/shaka-packager/build/packager/Release/libpackager.exp
Wed, 14 Feb 2024 19:24:15 GMT
LINK : warning LNK4217: symbol '??0Status@shaka@@QEAA@XZ (public: __cdecl shaka::Status::Status(void))' defined in 'job_manager.obj' is imported by 'media_replicator.lib(replicator.obj)' in function '"private: virtual class shaka::Status __cdecl shaka::media::Replicator::Process(class std::unique_ptr<struct shaka::media::StreamData,struct std::default_delete<struct shaka::media::StreamData> >)" (?Process@Replicator@media@shaka@@EEAA?AVStatus@3@V?$unique_ptr@UStreamData@media@shaka@@U?$default_delete@UStreamData@media@shaka@@@std@@@std@@@Z)' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_shared.vcxproj]
Wed, 14 Feb 2024 19:24:15 GMT
LINK : warning LNK4217: symbol '?Update@Status@shaka@@QEAAXV12@@Z (public: void __cdecl shaka::Status::Update(class shaka::Status))' defined in 'status.lib(status.obj)' is imported by 'media_replicator.lib(replicator.obj)' in function '"private: virtual class shaka::Status __cdecl shaka::media::Replicator::Process(class std::unique_ptr<struct shaka::media::StreamData,struct std::default_delete<struct shaka::media::StreamData> >)" (?Process@Replicator@media@shaka@@EEAA?AVStatus@3@V?$unique_ptr@UStreamData@media@shaka@@U?$default_delete@UStreamData@media@shaka@@@std@@@std@@@Z)' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_shared.vcxproj]
Wed, 14 Feb 2024 19:24:15 GMT
LINK : warning LNK4217: symbol '??1Status@shaka@@QEAA@XZ (public: __cdecl shaka::Status::~Status(void))' defined in 'job_manager.obj' is imported by 'media_replicator.lib(replicator.obj)' in function '"int `private: virtual class media::Status __cdecl shaka::media::Replicator::Process(class std::unique_ptr<struct shaka::media::StreamData,struct std::default_delete<struct shaka::media::StreamData> >)'::`1'::dtor$1" (?dtor$1@?0??Process@Replicator@media@shaka@@EEAA?AVStatus@3@V?$unique_ptr@UStreamData@media@shaka@@U?$default_delete@UStreamData@media@shaka@@@std@@@std@@@Z@4HA)' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_shared.vcxproj]
Wed, 14 Feb 2024 19:24:15 GMT
LINK : warning LNK4217: symbol '??0Status@shaka@@QEAA@AEBV01@@Z (public: __cdecl shaka::Status::Status(class shaka::Status const &))' defined in 'job_manager.obj' is imported by 'media_replicator.lib(replicator.obj)' in function '"private: virtual class shaka::Status __cdecl shaka::media::Replicator::InitializeInternal(void)" (?InitializeInternal@Replicator@media@shaka@@EEAA?AVStatus@3@XZ)' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_shared.vcxproj]
Wed, 14 Feb 2024 19:24:15 GMT
LINK : warning LNK4217: symbol '?OK@Status@shaka@@2V12@B (public: static class shaka::Status const shaka::Status::OK)' defined in 'status.lib(status.obj)' is imported by 'media_replicator.lib(replicator.obj)' in function '"private: virtual class shaka::Status __cdecl shaka::media::Replicator::InitializeInternal(void)" (?InitializeInternal@Replicator@media@shaka@@EEAA?AVStatus@3@XZ)' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_shared.vcxproj]
Wed, 14 Feb 2024 19:24:15 GMT
  libpackager_shared.vcxproj -> D:\a\shaka-packager\shaka-packager\build\packager\Release\libpackager.dll

Building libpackager_static (fails)

Wed, 14 Feb 2024 19:24:56 GMT
LINK : fatal error LNK1104: cannot open file 'D:\a\shaka-packager\shaka-packager\build\packager\Release\libpackager.lib' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_static.vcxproj]

Successful build (rerun)

Here are log snippets from the successful build (rerun):

Building libpackager_shared

Wed, 14 Feb 2024 21:50:07 GMT
     Creating library D:/a/shaka-packager/shaka-packager/build/packager/Release/libpackager.lib and object D:/a/shaka-packager/shaka-packager/build/packager/Release/libpackager.exp
Wed, 14 Feb 2024 21:50:07 GMT
LINK : warning LNK4217: symbol '??0Status@shaka@@QEAA@XZ (public: __cdecl shaka::Status::Status(void))' defined in 'job_manager.obj' is imported by 'media_replicator.lib(replicator.obj)' in function '"private: virtual class shaka::Status __cdecl shaka::media::Replicator::Process(class std::unique_ptr<struct shaka::media::StreamData,struct std::default_delete<struct shaka::media::StreamData> >)" (?Process@Replicator@media@shaka@@EEAA?AVStatus@3@V?$unique_ptr@UStreamData@media@shaka@@U?$default_delete@UStreamData@media@shaka@@@std@@@std@@@Z)' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_shared.vcxproj]
Wed, 14 Feb 2024 21:50:07 GMT
LINK : warning LNK4217: symbol '?Update@Status@shaka@@QEAAXV12@@Z (public: void __cdecl shaka::Status::Update(class shaka::Status))' defined in 'status.lib(status.obj)' is imported by 'media_replicator.lib(replicator.obj)' in function '"private: virtual class shaka::Status __cdecl shaka::media::Replicator::Process(class std::unique_ptr<struct shaka::media::StreamData,struct std::default_delete<struct shaka::media::StreamData> >)" (?Process@Replicator@media@shaka@@EEAA?AVStatus@3@V?$unique_ptr@UStreamData@media@shaka@@U?$default_delete@UStreamData@media@shaka@@@std@@@std@@@Z)' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_shared.vcxproj]
Wed, 14 Feb 2024 21:50:07 GMT
LINK : warning LNK4217: symbol '??1Status@shaka@@QEAA@XZ (public: __cdecl shaka::Status::~Status(void))' defined in 'job_manager.obj' is imported by 'media_replicator.lib(replicator.obj)' in function '"int `private: virtual class media::Status __cdecl shaka::media::Replicator::Process(class std::unique_ptr<struct shaka::media::StreamData,struct std::default_delete<struct shaka::media::StreamData> >)'::`1'::dtor$1" (?dtor$1@?0??Process@Replicator@media@shaka@@EEAA?AVStatus@3@V?$unique_ptr@UStreamData@media@shaka@@U?$default_delete@UStreamData@media@shaka@@@std@@@std@@@Z@4HA)' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_shared.vcxproj]
Wed, 14 Feb 2024 21:50:07 GMT
LINK : warning LNK4217: symbol '??0Status@shaka@@QEAA@AEBV01@@Z (public: __cdecl shaka::Status::Status(class shaka::Status const &))' defined in 'job_manager.obj' is imported by 'media_replicator.lib(replicator.obj)' in function '"private: virtual class shaka::Status __cdecl shaka::media::Replicator::InitializeInternal(void)" (?InitializeInternal@Replicator@media@shaka@@EEAA?AVStatus@3@XZ)' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_shared.vcxproj]
Wed, 14 Feb 2024 21:50:07 GMT
LINK : warning LNK4217: symbol '?OK@Status@shaka@@2V12@B (public: static class shaka::Status const shaka::Status::OK)' defined in 'status.lib(status.obj)' is imported by 'media_replicator.lib(replicator.obj)' in function '"private: virtual class shaka::Status __cdecl shaka::media::Replicator::InitializeInternal(void)" (?InitializeInternal@Replicator@media@shaka@@EEAA?AVStatus@3@XZ)' [D:\a\shaka-packager\shaka-packager\build\packager\libpackager_shared.vcxproj]
Wed, 14 Feb 2024 21:50:08 GMT
  libpackager_shared.vcxproj -> D:\a\shaka-packager\shaka-packager\build\packager\Release\libpackager.dll

Building libpackager_static (works)

Wed, 14 Feb 2024 21:50:45 GMT
  libpackager_static.vcxproj -> D:\a\shaka-packager\shaka-packager\build\packager\Release\libpackager.lib

Possible solutions

We could find the root of this MSVC failure (timing?) and resolve it. @cosmin also suggests that we shouldn't be building two library configurations at once, so maybe we could restructure the targets so that we only ever build libpackager_static or libpackager_shared, but not both.

joeyparrish commented 4 months ago

Looking at this again, it's possible that both targets (libpackager_shared and libpackager_static) are trying to create an import lib under the same name (libpackager.lib). Whether this works or not may merely depend on the timing of whether the old one (shared, according to log timestamps) is still open and in use when the new one (static) is being written.

I'll upload a draft PR that will build them both under different names (no aliases) on Windows and see if .lib files are being generated for both targets. If so, that would point strongly at both a cause (the aliases) and a solution (only build one or the other).

joeyparrish commented 4 months ago

My PR showed that .lib files were generated for both static and shared targets:

-rw-r--r-- 1 runneradmin 197121  251324 Feb 14 23:19 libpackager_shared.lib
-rw-r--r-- 1 runneradmin 197121 1058494 Feb 14 23:19 libpackager_static.lib

So having both alias to libpackager causes a collision that explains this issue.

So I think we should just build one target or the other.