Correctly handle dllimport on Windows

[alexcrichton]

Currently the compiler makes basically no attempt to correctly use dllimport. As a bit of a refresher, the Windows linker requires that if you're importing symbols from a DLL that they're tagged with dllimport. This helps wire things up correctly at runtime and link-time. To help us out, though, the linker will patch up a few cases where dllimport is missing where it would otherwise be required. If a function in another DLL is linked to without dllimport then the linker will inject a local shim which adds a bit of indirection and runtime overhead but allows the crate to link correctly. For importing constants from other DLLs, however, MSVC linker requires that dllimport is annotated correctly. MinGW linkers can sometimes workaround it (see this commit description.

If we're targeting windows, then the compiler currently puts dllimport on all imported constants from external crates, regardless of whether it's actually being imported from another crate. We rely on the linker fixing up all imports of functions. This ends up meaning that some crates don't link correctly, however (see this comment: https://github.com/rust-lang/rust/issues/26591#issuecomment-123513631).

We should fix the compiler's handling of dllimport in a few ways:

• Functions should be tagged with dllimport where appropriate
• FFI functions should also be tagged with dllimport where appropriate
• Constants should not always be tagged with dllimport if they're not actually being imported from a DLL.

I currently have a few thoughts running around in my head for fixing this, but nothing seems plausible enough to push on.

EDIT: Updated as @mati865 requested here.

[angelsl]

I will kill myself for suggesting this, but a possible workaround is to

1. Extract the .rlibs that we need to .objs and pass them to LINK
2. Omit dllimport from everything, since LINK will now fix it all up for us

The problem with this is that our resulting binary will export everything those .objs export. Not good.

[Diggsey]

@angelsl I think everyone's in agreement that this should be done "properly", the problem is in discerning what "properly" means in this context.

Per-item dllexport/dllimport attributes would solve this properly, simply because that mirrors how C++ does it exactly. The problem @alexcrichton described can be overcome the same way it is in C++: by controlling their use via cfg options (macros in C++'s case).

The way this would work is when compiling, you would pass in cfg options which determine how each set of external items used by the library are imported (either dllimport or not). Then either cargo can automatically figure out what cfg options to pass, or you can pass them manually. These cfg options are controlled by the program crate, and will be the same for all linked crates.

This would mirror how dllimport works in VC++ exactly. The problem is that you end up with an explosion of cfg variables (one for each library, the same as the number of macros in C++). However, it's possible to use this as a model: if a proposed solution can map onto this model without restricting useful cases, then we can be sure it's "proper".

I'm not sure enough about exactly what @alexcrichton's latest proposal is to be able to usefully compare it to the model, but it does seem to be more restrictive simply because it's much more automatic.

@alexcrichton Maybe you could explain precisely what scenarios can be expressed in this model but not in your proposal, and why you think they won't occur or are not useful? (bear in mind, when convert ing existing header files to rust as @retep998 is doing, the header file does have this level of flexibility about how how dllimport is applied)

[brson]

Nominating this to escalate. This seems to be something of a showstopper for MSVC support and may require language changes.

[brson]

cc @rust-lang/lang @rust-lang/compiler

[eddyb]

I helped @retep998 investigate the issue and it looked like a trans bug: extern { static FOO: T; } referenced from a different crate is treated like a Rust static FOO: T = ...; which is wrong, but only differs right now in dllimport AFAICT, so nobody found it.

You can tell that the entry point in trans::foreign for ForeignFn is designed to work cross-crate, but the one for ForeignStatic isn't (the latter takes a HIR node, which doesn't exist in the cross-crate case).

[alexcrichton]

@angelsl

Wouldn't that still end up having two copies of std?

In this example the DLL (C + B) is being dynamically loaded into the compiler, whereas the executable (A + B) has no interaction with C whatesover. So in that case I don't believe there will be two copies in one process at any one point in time.

Thanks for doing all the investigation as well, it's quite useful!

---

@briansmith

That's not really what I meant. What are the negative side effects of the proposed patch?

Oh sorry, didn't mean to put words in your mouth! To expand on what @vadimcn's patch, it will iterate over the list of all globals (e.g. statics) which are defined and exported in the current compilation unit, creating a sibling static __imp_<name> = &<name>. The reason this fixes the linkage problem in this issue is that when we mass apply dllimport the linker will probe for symbols called __imp_<name> instead of <name>, and this guarantees that the linker will find it if we import it from a DLL or if we import it from a Rust static library. This does not solve importing statics from external DLLs, however, which is where #[linked_from] comes into the picture.

In talking about overhead, it's basically exactly what we do today. Accesses to all statics from other crates will go through an extra layer of indirection (one more pointer jump) than they need to if the crate is linked statically. Some space will also be required to store these addresses, but that's likely negligible. I believe it will not be an improvement over today, however, except that linking will probably work in more cases than before, performance will neither improve nor regress.

---

@vadimcn

Thanks for also collecting those numbers! At this point I'm inclined to merge your patch as it seems like the best way forward in the immediate future.

---

@Diggsey

Maybe you could explain precisely what scenarios can be expressed in this model but not in your proposal, and why you think they won't occur or are not useful?

Most of what I was talking about had to deal with Rust crates actually, not with external libraries. In that case everything's actually really easy because the compiler already knows if libraries are linked via DLLs, so it just needs #[linked_from] to connect the symbols to the library. This basically means that if you have a header file + a DLL, all symbols marked dllimport in the header are in a block with #[linked_from] and all the other symbols are in their own extern block.

The Rust compiler has in general been pretty successful in hiding linkage details of Rust crates, and most of my prior comment was detailing how to continue doing this without having to do something like #[dllimport] extern crate foo.

[briansmith]

. Accesses to all statics from other crates will go through an extra layer of indirection (one more pointer jump) than they need to if the crate is linked statically.

Could we please file an issue to track into improving this in the future. In ring [1] and other projects I am working on, I expose a lot of statics that are to be referenced across modules that represent algorithms and other cryptographic parameters. And, in particular, users of these libraries will often define their own statics that contain references to those statics, cross-module. It sounds like the current model will force the use of static initializers in the case of static data referencing static data in another crate. Obviously, nobody wants static initializers (see all the work that went into Gecko to avoid static initializers).

[alexcrichton]

This issue is basically the one you want, it represents correctly applying dllimport. @vadimcn's patch does not do that and would not close this issue, it would simply make more situations link whereas today there's a link failure.

Unless there's link errors related to statics referencing other statics I don't think that there's anything new which could generate an error beyond what's already happening today (also no extra layers of indirection beyond the perhaps-unnecessary one we're already applying)

[angelsl]

@Diggsey Properly means decorating the correct imports with dllimport and leaving it off the imports that don't need it. How that is done I'm not partial to anything, as long as it is done correctly.

I am OK with dllimport/dllexport attributes. The thing is most of the rest here seem to prefer something that requires intervention from non-Win32 crate writers, and rightly so because why would they care about something that doesn't affect them? So we need rustc to figure this out automatically on its own, and preferably have some control in case it gets it wrong.

What you said about having Cargo pass in new arguments (which would mean also that non-Win32 authors don't need to care since Cargo figures it out) is something I suggested earlier. @alexcrichton did not really like the idea though. Fair enough.

@alexcrichton Ah ok. Thanks for the explanation!

[Diggsey]

@alexcrichton

Most of what I was talking about had to deal with Rust crates actually, not with external libraries.

Yes, but you said you wanted a single solution that would cover both cases.

This basically means that if you have a header file + a DLL, all symbols marked dllimport in the header are in a block with #[linked_from] and all the other symbols are in their own extern block.

OK, so "linked_from" is essentially equivalent a macro that evaluates to either dllimport or nothing depending on the configuration.

@angelsl

I am OK with dllimport/dllexport attributes. The thing is most of the rest here seem to prefer something that requires intervention from non-Win32 crate writers.

AIUI even non-win32 crate writers who link to native dylibs will have to make sure to apply the linked_from attribute, even though their code would run just fine on non-windows without it. (Just as how those native libraries would have to add dllimport/dllexport annotations when ported to msvc or mingw, although the situation is of course slightly more complicated for mingw).

Furthermore, it doesn't seem like it will be possible for rust to check that you've done this correctly at compile time?

[vadimcn]

AIUI even non-win32 crate writers who link to native dylibs will have to make sure to apply the linked_from attribute, even though their code would run just fine on non-windows without it.

@alexcrichton: I also think this is the weak point of #[linked_from]. IMO, if we started associating symbols with the #[link] attribute on the surrounding extern block, the right thing would be much more likely to happen.

Regarding libs, which are provided via the command line by cargo: if name of the lib isn't known ahead of time, linked_from cannot be used either, because, well, you don't know the lib name...

[vadimcn]

Also, I would agree with @retep998 regarding the necessity of some kind of explicit control over dllimport's. We can try to make things as automatic as possible, but there will always be corner-cases. If nothing else, the use of such attribute in *-sys libraries should be uncontroversial, since those are platform-specific.

[angelsl]

@Diggsey

AIUI even non-win32 crate writers who link to native dylibs will have to make sure to apply the linked_from attribute, even though their code would run just fine on non-windows without it.

True. But code that links to native dylibs that aren't compiled through cargo/rustc would need to have thought about building the native dylibs on Windows, so they can go the extra step and annotate it too. And code that links to native (I assume you mean non-Rust) things compiled through cargo/rustc e.g. the gcc crate, the build system would have the information, we just need to pass it to trans.

(Just as how those native libraries would have to add dllimport/dllexport annotations when ported to msvc or mingw, although the situation is of course slightly more complicated for mingw).

How is it more complicated for MinGW? MinGW's ld is happy to link directly to a DLL, unlike MSVC's link, although if you link to a DLL without dllimport in MinGW I believe you end up with an indirection.

Furthermore, it doesn't seem like it will be possible for rust to check that you've done this correctly at compile time?

True. It can only know at link time. And even then the only way it would know for sure is to open up the libraries and check (hacky), or wait for link to barf (even hackier! But if link barfs then compilation fails anyway).

[retep998]

@alexcrichton If we use #[linked_from] to determine whether to apply dllimport rather than some sort of #[dllimport] attribute then we still run into the same case of linking to some native library, say foo, and not knowing whether the library the user has provides static or dynamic symbols. We certainly can't rely on kind=static vs kind=dylib because that is used to distinguish between whether to bundle the library into the .rlib or defer it to link time, which is entirely orthogonal to whether a library provides static or dynamic symbols. So that leaves only one other option and that is to pass a cfg thing through Cargo such that the #[linked_from] is conditionally emitted, but that can also be done with a cfg conditionally emitting a #[dllimport].

[Diggsey]

How is it more complicated for MinGW? MinGW's ld is happy to link directly to a DLL, unlike MSVC's link, although if you link to a DLL without dllimport in MinGW I believe you end up with an indirection.

The documentation seems to indicate otherwise: unless you compile with some additional flags then MinGW also requires that dllimport/dllexport be applied correctly: http://www.mingw.org/wiki/sampledll

Failure to do so for functions will just result in indirection, but failing to do it for data would cause crashes as we see here. In this case ld behaves no differently from link. The extra complication comes from the existence of the additional compiler flags, which seem to do some kind of unix emulation (however, these are not enabled by default).

True. It can only know at link time. And even then the only way it would know for sure is to open up the libraries and check (hacky), or wait for link to barf (even hackier! But if link barfs then compilation fails anyway).

This doesn't help on non-windows platforms: if the advantage of [linked_from] is that it is not windows-specific, then ideally, using it incorrectly should cause compilation to fail across all platforms (so that you can compile on linux, and as long as you don't use any platform specific features, be reasonably confident that the code will also compile on windows, or vice versa)

[angelsl]

@Diggsey

The documentation seems to indicate otherwise: unless you compile with some additional flags then MinGW also requires that dllimport/dllexport be applied correctly: http://www.mingw.org/wiki/sampledll

Failure to do so for functions will just result in indirection, but failing to do it for data would cause crashes as we see here. In this case ld behaves no differently from link. The extra complication comes from the existence of the additional compiler flags, which seem to do some kind of unix emulation (however, these are not enabled by default).

// exe.c
#include <stdio.h>

char *ext();

int main() {
    puts(ext());
    return 0;
}

// dll.c
char *ext() {
    return "Hello!";
}

> gcc -c -O3 dll.c
> gcc -c -O3 exe.c
> gcc -shared -o dll.dll dll.o
> del dll.o
> dir
 Volume in drive C has no label.
 Volume Serial Number is C2CB-8813

 Directory of C:\Users\angelsl\Root\Development\rust\test

20150923  22:26    <DIR>          .
20150923  22:26    <DIR>          ..
20150923  22:23                38 dll.c
20150923  22:26            45,786 dll.dll
20150923  22:23                86 exe.c
20150923  22:26             1,023 exe.o
               4 File(s)         46,933 bytes
               2 Dir(s)  868,320,825,344 bytes free
> gcc -o exe.exe exe.o -L. -ldll
> exe
Hello!

// exe.c
#include <stdio.h>

extern char* ext2;

int main() {
    puts(ext2);
    return 0;
}

// dll.c
char *ext2 = "Hello from data!";

> gcc -c -O3 dll.c
> gcc -c -O3 exe.c
> gcc -shared -o dll.dll dll.o
> del dll.o
> gcc -o exe.exe exe.o -L. -ldll
> exe
Hello from data!

QED.

This doesn't help on non-windows platforms: if the advantage of [linked_from] is that it is not windows-specific, then ideally, using it incorrectly should cause compilation to fail across all platforms (so that you can compile on linux, and as long as you don't use any platform specific features, be reasonably confident that the code will also compile on windows, or vice versa)

Fair enough.

But I would rather have a solution that works correctly on Windows (again, correctly means applying dllimport where needed and only where needed) than a worse solution (or none at all!) that consistently reports errors on other platforms even though it doesn't affect those other platforms.

[alexcrichton]

@angelsl, @Diggsey

AIUI even non-win32 crate writers who link to native dylibs will have to make sure to apply the linked_from attribute

This and the other conclusions you reach are correct, but I think it's just a fact of the way things are. For example any manual attribute (such as #[dllimport]) suffers the same problems. The only way we could not have a situation like this is if we auto-inferred everything, which is currently impossible for the compiler to do in all cases.

---

@vadimcn

I also think this is the weak point of #[linked_from]

I feel, however, that it impossible for this to not be a weak point of any solution. I'd be fine saying that #[link] implied #[linked_from] on a block of functions, but that doesn't obsolete the need for #[linked_from]. Linkage provided via the command line is not always just because the name of the library is not known, it is frequently because the builder of the code determines whether the linkage is static or dynamic, not the author of the code.

regarding the necessity of some kind of explicit control over dllimport's

I agree in principle, but I would prefer to not take that route at this time to avoid stabilizing more than we need to. If we can cover all existing use cases with #[linked_from] then it brings with it the benefits of:

• More ergonomic, don't need to attach it to all native declarations.
• More ergonomic in that no #[cfg] is needed
• Robust in the face of swapping out the linkage at build time
• Easy to add retroactively to crates to compile on Windows.

---

@retep998

So that leaves only one other option and that is to pass a cfg thing through Cargo such that the #[linked_from] is conditionally emitted, but that can also be done with a cfg conditionally emitting a #[dllimport].

Is there a problem with this? Having one stable attribute which is ergonomic to use 99% of the time and in the weird corner cases is slightly less ergonomic seems much better than the alternative of tagging all functions and statics with an attribute which is also wrapping a #[cfg].

I don't understand where you're going with these comments, do you not want #[linked_from]? Do you only want #[dllimport]?

---

@Diggsey

if the advantage of [linked_from] is that it is not windows-specific,

While not necessarily Windows specific, I don't personally have any plans to make compilation fail if it is not or if it's incorrectly applied on Unix platforms. (not sure what this would mean?)

[retep998]

@alexcrichton I find it weird that by simply telling Rust which library a group of functions/statics comes from, it emits dllimport. Also the fact that you even have to tell Rust at all which library those functions/statics came from, which is useless information since the linker will resolve the symbols to whichever library it feels like. It all just feels very unrelated to the point of whether to apply dllimport or not. Making the distinction based on kind=static vs kind=dylib is entirely orthogonal, since a bundled library could still have dynamic symbols that need dllimport and a non-bundled library can have static symbols that don't need dllimport. If there is some way to tell #[linked_from] whether to apply dllimport or not, that does not involve deciding whether to bundle the library, then that would be enough. But as it is, I don't see how it makes sense.

tl;dr Give me a way to specify whether to apply dllimport without forcing my hand with regards to bundling.

[vadimcn]

Linkage provided via the command line is not always just because the name of the library is not known, it is frequently because the builder of the code determines whether the linkage is static or dynamic, not the author of the code.

We could have the command line option override "kinds" set via #[link].

I agree in principle, but I would prefer to not take that route at this time to avoid stabilizing more than we need to. If we can cover all existing use cases with #[linked_from]...

I am afraid it will take a lot of effort to cover all corner cases. dllimport would solve the problem at hand more directly. We already have the #[linkage] attribute for low-level control. IMO, dllimport falls into the same category.

[alexcrichton]

@retep998

tl;dr Give me a way to specify whether to apply dllimport without forcing my hand with regards to bundling.

#[linked_from] does this. If you want dllimport, then you must have some dylib somewhere, so you just connect the two. Your concerns about bundling are only related to the fact that we emit dllimport for all external statics today by default and there's no way around that.

---

@vadimcn

I am afraid it will take a lot of effort to cover all corner cases.

Could you elaborate on these corner cases? I can't actually think of any that #[linked_from] doesn't cover. While we do have #[linkage] it's unstable (and for good reason) and may be difficult to stabilize.

---

Over lunch I also remembered one of the critical aspects of #[linked_from] which is why it exists today in the first place. We haven't been talking about dllexport a lot, but it's a thing, and currently the compiler leverages #[linked_from] to ensure that what Rust DLLs export is actually correct.

When having a Rust dylib as an intermediate dependency, it's possible to link a C library statically to the dylib which then must also be exported from the DLL to have future Rust dependencies linking against it to work. For example rustc_llvm statically links LLVM, but LLVM is called from rustc_trans so the symbols in LLVM need to be exported. Currently the compiler detects this situation by understanding that symbols in a #[linked_from] block which are statically included and reachable need to be exported from a DLL as well.

This, paired with the ergonomic benefits, makes me think that it obsoletes any form of manual dllimport/dllexport. It continues Rust's trend of in general hiding platform-specific linkage details in favor of higher level intentions which end up covering basically all use cases anyway.

[retep998]

So basically if I want dllimport I have to specify #[linked_from] and if I don't want dllimport then I don't specify #[linked_from]? So if some crate wants to link to some native library, and that native library might be a static library or it might be a DLL, does that mean the crate has to do #[cfg_attr(is_dll, linked_from = "...")]?

[alexcrichton]

Kinda, for any library in question I would expect the set of symbols exported via a DLL of that library to be relatively constant. This is typically the entire API surface but it looks like from your examples that's not always true. For the set of symbols that are exported via the DLL you would wrap them in a #[linked_from] unconditionally.

If you expect the native library to them also be sometimes linked statically, only then would you have to use cfg_attr to annotate the set of symbols which are linked statically even when linked via a DLL.

This means that #[cfg_attr] is not common nor needed most of the time (as these obscure DLLs probably always show up as a DLL as opposed to sometimes being linked statically).

[arielb1]

@angelsl

I don't believe Linux goes through the GOT if you statically link things together. That would be quite terrible. (And worse if it's Rust specifically that does that!)

Linux goes through the GOT when you have a non-C-static global variable on a library. Executables (-fPIE) instead steal the global to their own section.

Also, the "dummy function" is just to convince LINK.EXE to create the __imp_ symbols, it shouldn't actually get called.

@alexcrichton

On Windows, linking is always done statically - i.e. against a .lib. The .lib can contain both static symbols that exist only in the .lib, and dynamic symbols that refer to some .dll.

[brson]

I'm going to express some vague opinions, but this topic is pretty vast and I don't understand it near as well as others here.

I see there are two separate problems here - one with rust crates and one with native libs.

For the former, I'm in favor of any intermediate workaround solution that does not surface any new language features while we figure out more performant solutions.

For the latter, my inclination is that #[link = "..."] would have been the right solution but the existing semantics are already baked in and the link attribute has no real correlation to the extern block its attached to. Whether link_from as stands is the right sol'n idk.

[vadimcn]

[I don't have particularly strong feelings about linked_from, it just seems like an extra level of cruft, which makes me slightly sad.]

Can't we extend the existing semantics? I believe this would be backwards-compatible: currently none of the platforms except -windows-msvc care about the symbol<->library associations, so they will continue to work even if markup is wrong (but in many cases it is actually correct, because people usually don't create empty extern blocks just for the heck of it)

[angelsl]

@arielb1

Also, the "dummy function" is just to convince LINK.EXE to create the __imp_ symbols, it shouldn't actually get called.

That would be a workaround to coerce LINK to fix up our broken linkages.

Again if we're going to support MSVC, we might as well do it properly.

[alexcrichton]

@vadimcn

Could you elaborate on how you see extending the existing semantics? I'm totally on board with #[link] implying #[linked_from], but having an extern block without #[link] isn't something that can be ignored, it happens very commonly.

[vadimcn]

@alexcrichton:

• If name of the library is known, just add a #[link] attribute. The kind can be later overridden from the command line.
• If name is not known, #[linked_from] would not have helped anyway, as it needs the name of the library. One way of dealing with this might be to invent an 'alias' kind, e.g. #[link(name="foo" kind="alias"], as well as command line syntax to specify the actual library name it maps to, for example, -l alias=foo=libbar

It would still be fine to have unadorned extern block, those symbols just wont be associated with a library. Existing code will continue to work since only -windows-msvc needs these associations.

[alexcrichton]

That sounds pretty plausible to me! I'll look to draft up an RFC soon.

[retep998]

Here's a nicer table showing the dllimport failure cases.

Library type	Static	Function	Result
Dynamic	No	No	Success
Dynamic	Plain	No	Error
Dynamic	Dllimport	No	Success
Dynamic	No	Plain	Success
Dynamic	Plain	Plain	Error
Dynamic	Dllimport	Plain	Success
Dynamic	No	Dllimport	Success
Dynamic	Plain	Dllimport	Error
Dynamic	Dllimport	Dllimport	Success
Static	No	No	Success
Static	Plain	No	Success
Static	Dllimport	No	Error
Static	No	Plain	Success
Static	Plain	Plain	Success
Static	Dllimport	Plain	Warning
Static	No	Dllimport	Error
Static	Plain	Dllimport	Warning
Static	Dllimport	Dllimport	Error

[alexcrichton]

I've opened an RFC for how to deal with native libraries, which in concert with @vadimcn's patch should eliminate all link errors related to the application of dllimport/dllexport.

[angelsl]

If we merge that patch, are we ever going to properly decorate imports between Rust crates?

[alexcrichton]

Certainly! This issue will not be closed by that PR.

[nikomatsakis]

Compiler subteam: not assigned a priority yet as full implications are not understood. @alexcrichton maybe you and I can chat and you can catch me up here, there's a lot of backscroll. :)

[nikomatsakis]

triage: P-low

After discussion with @alexcrichton, my conclusion was that the remaining problems are low priority, with the exception of the issues discussed (and hopefully addressed) in RFC https://github.com/rust-lang/rfcs/pull/1296.

My understanding of the situation is roughly as follows (feel free to correct me if you feel I am mistaken):

1. Windows generally requires that imports/exports from DLLs be specially declared.
2. But Rust crates do not, typically, know whether they are being compiled for use in a DLL or not
   • rlibs can be used in either way
   • but static linking is the default
3. Currently, we use some set of declarations that are incorrect but the Windows linker nonetheless patches them up. The semantics are correct in all cases but the "fixup" results in suboptimal performance under the following scenarios:
   • when using dynamic linking in Windows, function calls suffer from an extra level of indirection
   • when using static linking in Windows, accessing static variables requires an extra load
4. The performance impact of the above measures in real-world applications has not been measured. @alexcrichton expects it is low, particularly as static linking is the default for Rust (where fn calls are fast), and static variables are unusual.
5. If we wanted to improve the situation, language changes are unnecessary, though of course they might be one route. At worst, we could fix this problem by having rlibs not contain the full .o on Windows, but rather LLVM IR (or something similar). We could then patch up the declarations at the last minute, when we know how a crate is to be used. Another option might be having cargo propagate the information about whether a crate will be used for dynamic or static linking more aggressively.

[retep998]

@nikomatsakis While I can accept linking between Rust crates to be low priority since it does work, albeit poorly, getting dllimport correct when linking to native libraries is very much a concern still. uuid-sys does not work on msvc at all and it's a fairly important library since it provides a ton of GUIDs that are necessary for things like working with COM.

[alexcrichton]

@retep998 note that @nikomatsakis indicated that the issues are low priority with the exception of rust-lang/rfcs#1296 which is where the native library case should be handled.

[retep998]

So currently we already have some cases involving mixing and matching of static/dynamic linking between Rust crates that results in errors. For example this results in an error due to duplicate std and friends: rustc --crate-type=dylib a.rs rustc b.rs --extern a=a.dll

Because of this precedent that you already have to be really careful about whether you're linking to things statically or dynamically, plus the fact that the common use case with Cargo is simply linking everything statically, I believe it should be entirely feasible to be more strict about making sure you link to the same dependency as you built against. So if b.rlib is compiled against a.rlib then you cannot later change your mind and link to a.dll instead.

If a PR can be made which imposes this restriction and takes advantage of it to apply dllimport correctly, would it be accepted, or would you insist on some alternative which either doesn't exist or is a poor idea (like delaying all codegen to link time, which would severely impair edit-compile times)? If this should go through an RFC, tell me so I can write an RFC.

[alexcrichton]

@retep998 have you read my response above? I outlined how I thought we could apply dllimport and why it won't work, do you have an idea of how to fix the problems associated with that strategy?

[angelsl]

@alexcrichton

His solution is the same as your solution, except it also prevents linking an rlib dynamically when it was built to be used statically or vice versa i.e. fixing your failure condition.

Honestly, I'm waiting on you Rust people to approve a solution before I even bother writing it, because it seems like anything we suggest is going to be shot down, and any PRs rejected, even when there is nothing wrong with the solution.

[alexcrichton]

Ah so my response was indeed a little long winded, so let me reiterate the last portion to emphasize the problem that needs fixing.

In the case today where a dependency is shared by a plugin and an executable, Cargo will only compile the dependency once. If we were to "correctly apply dllimport", however, this is not correct because the dependency needs to be compiled twice (once to statically link into the executable and once to dynamically link into the plugin). A solution where an rlib ties its upstream dependencies to a particular format will break this model, unfortunately. In other words, I don't think I've seen anything which solves this problem.

I think, however, that this is the only case where we'd run into problems today. There are perhaps a number of ways to handle it, but I haven't found one quite palatable yet.

Can you elaborate on why you think that suggestions are being shot down or PRs are being rejected? I'd also love to fix this, but we have to be careful to not break existing situations today as well, so the various use-cases just need to be thought through. I don't think anyone's intentionally trying to hinder efforts here!

[retep998]

In the case of cross-compiling wouldn't crates that are used in plugins already be compiled twice? Once for the native triple to be used in a plugin and once for the target triple to be used in the final product? Shouldn't doing something similar in the non-cross case also be okay in order to correctly apply dllimport?

[angelsl]

In the case today where a dependency is shared by a plugin and an executable, Cargo will only compile the dependency once. If we were to "correctly apply dllimport", however, this is not correct because the dependency needs to be compiled twice (once to statically link into the executable and once to dynamically link into the plugin). A solution where an rlib ties its upstream dependencies to a particular format will break this model, unfortunately. In other words, I don't think I've seen anything which solves this problem.

There aren't very many solutions to this. You could just compile the dependency twice. or you disallow such a situation and say that in this case, the executable and plugin should really be crates that are compiled with separate invocations of Cargo instead of the plugin being a dependency of the executable. (Of course, I think the ideal way to solve it is if you're going to have dynamically-linked plugins, then link everything dynamically, but that isn't always feasible, so at the very least have the common dependency/ies built as a dynamic library.)

Edit: Actually, making Cargo compile the dependency twice in the same run seems like the only proper solution here.

I suggested the first solution here, but you rejected it on the grounds that it required too invasive changes to Cargo and that it may cause some problems in future.

The second solution is what @retep998 is suggesting. Make such a combination illegal (IMO, we should not have more than one copy of each dependency anyway).

Maybe there is a solution that 1. doesn't have the problem you mentioned and 2. penalises neither dynamic nor static linking, but I highly doubt it. If Rust wants to support MSVC, it should be willing to actually support MSVC, instead of leaving a half-fix that works, but doesn't work as well as it could.

I feel very discouraged working on this problem, because I really want to use Rust, but I cannot do so comfortably (at least on MSVC) when I know that Rust is relying on the linker to fix its poorly-done decorations (and then chomping the linker output so we don't see any of the warnings!). And what makes it worse is that you don't seem to want to make the necessary changes, no matter how large they may be, to properly support MSVC. I'm ready to go ahead and delve into rustc even though I haven't actually written any Rust before so that I can get this fixed, but there doesn't seem to be any point if ideas are rejected before they're even tried out.

If Windows/MSVC is a platform Rust wants to claim to support, then it should do so properly, and be willing to make the changes needed.

Sorry for the rant, but.. yeah.

[Diggsey]

@angelsl You are getting confused again by why @alexcrichton means by "plugin". I think he's referring to compiler plugins rather than application plugins. (Might be clearer to refer to them as compiler plugins rather than just plugins, especially on a thread about dynamic linking!)

[retep998]

The API that a compiler plugin exposes to downstream dependencies is very different than the API that a compiler plugin exposes to the compiler. Because of this I see very little reason to be building a compiler plugin such that it can be used for both things. Good thing compiler plugins are unstable so that can be improved.

[angelsl]

@angelsl You are getting confused again by why @alexcrichton means by "plugin". I think he's referring to compiler plugins rather than application plugins. (Might be clearer to refer to them as compiler plugins rather than just plugins, especially on a thread about dynamic linking!)

Okay, my bad. Sorry.

So we can't make it illegal to have a dependency be linked in different ways in a single Cargo invocation. The only way left is really to build such dependencies twice.

[retep998]

Let's go back to this situation

• Executable A depends on B and plugin C
• Crate C also depends on B.

What happens if I am cross-compiling? Does C get built for the host architecture and B gets built twice, once for host and once for target? If so, can't that logic be extended to also be used when targetting -msvc such that B is built twice, once with a static std to be used in A, and again with a dynamic std to be used in C?

[arielb1]

@retep998

So we have x86-pc-msvc-static and x86-pc-msvc-dynamic targets?

[Diggsey]

The msvc solution is generally to treat it as part of the configuration (like debug vs release) rather than the target. That's why MSVC projects often have Release/ReleaseDLL or similar configurations.

I vote for not messing with the target triples any more...

[retep998]

To really do this right would require that Cargo has knowledge in advance of whether a dependency will be a plugin or not so that it can decide how to build it and its dependencies.