[libtorch] mimalloc build as dynamic library

[xuhancn]

Operating system

Windows

Compiler

MSVC

Steps to reproduce the behavior

Original issue from pytorch project: https://github.com/pytorch/pytorch/issues/125840

Failure logs

mimalloc-redirect: error: mimalloc-redirect.dll seems to be initialized after ucrtbased.dll
  (hint: try to link with the mimalloc library earlier on the command line?)
mimalloc-redirect: error: mimalloc-redirect.dll seems to be initialized after ucrtbase.dll
  (hint: try to link with the mimalloc library earlier on the command line?)
mimalloc-redirect: warning: standard malloc is _not_ redirected! -- using regular malloc/free. (v1.1c)

Additional context

pytorch use mimalloc as static library only in c10, original cmake code is: https://github.com/pytorch/pytorch/blob/51b22d9cf2f120a52719cb1c59dcccdf450fcbeb/c10/CMakeLists.txt#L123-L124

Please update vcpkg port to fix the issue.

[Neumann-A]

https://github.com/microsoft/mimalloc/issues/668

[Neumann-A]

From https://microsoft.github.io/mimalloc/overrides.html

To ensure the mimalloc DLL is loaded at run-time it is easiest to insert some call to the mimalloc API in the main function, like mi_version() (or use the /INCLUDE:mi_version switch on the linker). See the mimalloc-override-test project for an example on how to use this. For best performance on Windows with C++, it is also recommended to also override the new/delete operations (by including mimalloc-new-delete.h a single(!) source file in your project without linking to the mimalloc library).

[xuhancn]

From https://microsoft.github.io/mimalloc/overrides.html

To ensure the mimalloc DLL is loaded at run-time it is easiest to insert some call to the mimalloc API in the main function, like mi_version() (or use the /INCLUDE:mi_version switch on the linker). See the mimalloc-override-test project for an example on how to use this. For best performance on Windows with C++, it is also recommended to also override the new/delete operations (by including mimalloc-new-delete.h a single(!) source file in your project without linking to the mimalloc library).

PyTorch is not turn on mimalloc overwrite: https://github.com/pytorch/pytorch/blob/0910429d7262daf67dc3aa1d4e4aa939752ae675/CMakeLists.txt#L1203

python modules can't overwrite new/delete operation, it will crash on across module class object sharing.

PyTorch only call the mimalloc API directly. https://github.com/pytorch/pytorch/blob/51b22d9cf2f120a52719cb1c59dcccdf450fcbeb/c10/core/impl/alloc_cpu.cpp#L106 https://github.com/pytorch/pytorch/blob/51b22d9cf2f120a52719cb1c59dcccdf450fcbeb/c10/core/impl/alloc_cpu.cpp#L158

[Neumann-A]

But can you then not ignore the message? It can probably be suppressed with setting MIMALLOC_DISABLE_REDIRECT=1 in the environment.

The hard error in the linked post is actually about not being able to find nvfuser_codegen.dll and not about mimalloc. So putyour cuda bin path somewhere where windows can actually find it?

[xuhancn]

But can you then not ignore the message? It can probably be suppressed with setting MIMALLOC_DISABLE_REDIRECT=1 in the environment.

The hard error in the linked post is actually about not being able to find nvfuser_codegen.dll and not about mimalloc. So putyour cuda bin path somewhere where windows can actually find it?

I just forward issue, please reply to the origin issue submitter: https://github.com/pytorch/pytorch/issues/125840

[xuhancn]

But can you then not ignore the message? It can probably be suppressed with setting MIMALLOC_DISABLE_REDIRECT=1 in the environment.

The hard error in the linked post is actually about not being able to find nvfuser_codegen.dll and not about mimalloc. So putyour cuda bin path somewhere where windows can actually find it?

BTW, disable mimalloc will impact on performance, I still suggest to static link mimalloc. Thanks.

[Neumann-A]

BTW, disable mimalloc will impact on performance,

This does not disable mimalloc since you said that pytorch is not using the redirection but calling mimalloc directly. So it will always call out to mimalloc.

[dg0yt]

In the context of vcpkg, linkage is controlled by triplets (files) for a whole install prefix, not for a single dependency relationship. x64-windows chooses dynamic linkage. Via custom triplet files, users might choose to have a static mimalloc even when the default is dynamic.

The mimalloc DLL initialization hint is even printed as usage notification when installing mimallloc. The key messages: The user can do something in the main function.

So nothing really broken here IMO. Just inconvenient transitive usage requirements, advanced level.

[github-actions[bot]]

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 180 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.