[SR-655] Private C modules

[drewcrawford]

Previous ID	SR-655
Radar	None
Original Reporter	@drewcrawford
Type	Bug

Environment

swift-DEVELOPMENT-SNAPSHOT-2016-01-25-a

Additional Detail from JIRA

| | | |------------------|-----------------| |Votes | 1 | |Component/s | | |Labels | Bug | |Assignee | @belkadan | |Priority | Medium | md5: b7c8feeb59db811a066d05fd7eed4ad1

relates to:

• SR-3205 Ability to restrict the import of implicit module dependencies

Issue Description:

I have a library, let's call it Foo.

As an implementation detail, Foo calls down to a C library. Following the [SwiftPM guidance| https://github.com/apple/swift-package-manager/blob/master/Documentation/SystemModules.md], I have a CFoo.modulemap that tells where to get the headers, and what to link, and so on. And I pass -I path/to/CFoo.modulemap so that the Swift compiler gets that information.

Foo builds successfully, the tests pass, everything works.

Now I want to use Foo from another program. But when I import Foo I get the error
error: missing required module 'CFoo'.

But I don't want to import CFoo from callers, because CFoo should be a hidden implementation detail.

How do I kill this error? Have I architected something wrong?

[drewcrawford]

cc: @belkadan

[belkadan]

Right now we conservatively assume you need every imported module to successfully import the interface of a library. This has lots of issues, this being the main one, but it's a non-trivial amount of work to fix, and there's really a semantic change about imports that goes with it. I'm hoping to tackle this in the not-too-distant future, but no promises.

[drewcrawford]

I reverse-engineered what Xcode does and now have a workaround.

Essentially, I have a module.modulemap with

module \(name) {
   umbrella header "Umbrella.h"
   export *
  module * { export * }"
}

Then I compile with -I path/to/modulemap -import-underlying-module. This lets me use Umbrella.h as an umbrella header.

Then inside Umbrella.h I can import arbitrary C header files, and this avoids the need to declare a CFoo in the first place, therefore callers do not have to import it.

This works very well, also working on Linux.

This works in spite of the fact that there is some debate about whether "bridging headers" (different from umbrella headers?) actually work. See SR-76 for example, this may be a workaround for that issue as well.

[belkadan]

That's not actually a workaround; it just means anyone importing your module is automatically getting all those headers too. It's actually worse because they're getting them publicly, since you've declared they're part of your module's interface.

[drewcrawford]

I don't actually distribute the modulemap–I just use it to build and then throw it away. Unless the bridging header gets squirreled away in the .swiftmodule file somewhere (that is a total black box to me), callers don't see it.

[belkadan]

Umbrella headers and bridging headers are not the same thing. I'm very surprised this is working for you with the module map gone, and it is not and will not be supported.

[drewcrawford]

It turns out this does get squirreled away somewhere in the swiftmodule. Bummer.

I guess I will have to return to exposing this internal detail to callers...

[drewcrawford]

Not only does Swift require these (indirect) modules to be imported, but it requires them to be imported in the correct order. e.g. if an application depends both on Dispatch and a module that depends on Dispatch, it must be 1. Dispatch, 2. module depending on Dispatch, 3. application.

I'm puzzled how this is going to work once Foundation takes a Dispatch dependency, which AFAIK is happening nowish. Are Foundation clients going to have to say

import Dispatch
import Foundation

[belkadan]

The order thing is just a bug, and if someone investigates why the behavior differs and fixes it, that's great.

Exposing knowledge about private dependencies is also a bug, but one that needs design, I think, to specify what should happen. Let's discuss it on swift-dev if you're interested.

[swift-ci]

Comment by Neon (JIRA)

A nice feature would be an additional search path for "private" modules. Used when needed by other modules but not importable by code. I think it does not solve the original problem but is somehow related.

This can be useful in cases when one has less trustworthy code and want to restrict it to an api that uses a more powerful module.

Being able to restrict what's importable instead of forcing everything.

[swift-ci]

Comment by Dmitry Shevchenko (JIRA)

I had the same issue and I think it can be "solved" by "emulating" a mixed-code framework:

1. Name both Swift and C modules the same, say Foo

2. When compiling the Swift module, pass -import-underlying-framework and the path to C's modulemap
3. Now, importing Foo will give you access to both C and Swift symbols.

How bad and unsupported is this approach? 🙂

[swift-ci]

Comment by Deepesh (JIRA)

dmishe (JIRA User) by having similar names it does work but debugging the swift code stops working as writing any PO statements would show up warnings about having a canonical name for module similar to one discussed here : https://openradar.appspot.com/40829112

I did add a underscore and everything started working fine within the framework. Provided the framework needs to be tested when using the external projects using PODS etc.