Only one Kotlin framework can be loaded currently

[solivares1-zz]

So.. i managed to make two kotlin-native frameworks for Swift but when trying to add them on the project i got the following message. (Only one Kotlin framework can be loaded currently)

Do we have any update on this? I prefer not to merge both frameworks into one, is there another workaround?

[SvyatoslavScherbina]

I prefer not to merge both frameworks into one

What's the reason behind this? Do these frameworks have something in common? Does one of these frameworks depend on the other one?

[solivares1-zz]

@SvyatoslavScherbina They don't depend on each other nor have something in common, and both of those frameworks can be used in other apps. So if i merge them when i create the .framework it will have unnecessary code from the unused one.

I think there's a workaround, creating tasks in the build.gradle to adjust for specific needs like, build A, build B or build A&B in only one .framework

But thats not optimal, we can't share several kotlin-native created libraries if we can't use more than one Kotlin framework per project

[kylejbrock]

@solivares1 That's what I've had to do to utilize multiple kotlin libraries as a single framework:

I actually support several apps, needing different features. They are both ios & android apps.

So, I've ended up with a function in gradle called: isPlatformTypeEnabled and another called isFeatureEnabled.

Then I have a block that looks similar to this:

if (isPlatformTypeEnabled("client") && isPlatformTypeEnabled("ios")) {
            iosMain {
                kotlin {
                    if (isFeatureEnabled("localization")) {
                        srcDir("$projectDir/src/localization/iosMain/kotlin")
                        srcDir("$projectDir/src/localization/mobileMain/kotlin")
                    }
                }
                dependencies {
                    implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime-native:$serialization_version"
                }
            }
            iosTest {
                dependencies {
                    implementation 'org.jetbrains.kotlin:kotlin-test'
                    implementation 'org.jetbrains.kotlin:kotlin-test-junit'
                }
            }
        }

This enables me to dynamically include different source sets based on what I pass to the gradle build.

It's definitely not ideal, but it's a sufficient work around.

This yields a single framework per application/platform. But, with the appropriate feature sets per application/platform.

[SvyatoslavScherbina]

@solivares1 thank you for providing the details. I suppose this case makes sense and can be eventually supported.

[loganj]

We (Square) have another use case, one we think is likely to be a factor for any large codebase (which is where multiplatform is very desirable!)

Our mobile apps are large, numerous, and highly modularized. On iOS, we ship multiple apps that are all built from different combinations of shared modules. For instance, we might have a printing module that's used by several apps (but not all).

We'd like to have our iOS modules consume our kotlin-native libraries as dependencies, but the single-framework limitation makes that difficult.

We could include all kotlin-native code for all apps in a single framework, but that would mean shipping unnecessary code, as @solivares1 pointed out. Or we could build a different framework for each app, but then we'd have to change the build for our shared printing module to use the appropriate framework depending on which app it's being included in – this is complicated and makes it impossible to build the printing module independently. Either way, we lose modularity and create unnecessary and unclear dependencies.

Ideally, our kotlin-native libraries would appear in our build dependency graph just like any other library.

[SvyatoslavScherbina]

@loganj

Or we could build a different framework for each app, but then we'd have to change the build for our shared printing module to use the appropriate framework depending on which app it's being included in

Do I understand correctly that you shared printing module may use different combinations of Kotlin modules depending on which application it is being included in?

[loganj]

No, but the printing module itself may or may not be included in any of our applications.

If we build a single Kotlin framework that supplies all (Kotlin) dependencies for all of our applications, then the printing module will need to depend on 1) non-printing code and 2) non-printing code that may not even be needed for an app that the printing module is included in.

Imagine this scenario (not real, but an example). Dependencies flow right to left, ie application depends on iOS module, iOS module depends on kotlin-native module.

Kotlin-Native module	iOS module	application
kotlin-print	printing	retail
kotlin-rtl	retail	retail
kotlin-rst	restaurants	restaurants

A single kotlin framework means that 1) printing cannot depend directly on kotlin-print, but instead will depend on a framework that also includes unrelated application-level code (kotlin-rtl, kotlin-rst) and 2) the retail application will end up including kotlin-rst, which is application-level code for restaurants.

[SvyatoslavScherbina]

Thank you for detailed explanation! I have one question left: does one of Kotlin/Native modules depend on another one?

[loganj]

Yes, certainly. There would be dependencies among the kotlin-native modules. The example above is not complex enough to illustrate it well, but you could imagine both direct and transitive dependencies from iOS modules to kotlin-native modules, and we'd like for those to resolve normally.

[SvyatoslavScherbina]

Unfortunately, this kinda conflicts with our current vision and efforts. Kotlin/Native follows "closed world" compilation model. This means that the entire Kotlin world is compiled to the single native binary, e.g. framework. Two Kotlin/Native frameworks would mean two different Kotlin "worlds", particularly they wouldn't have same Objective-C/Swift custom classes in their APIs. Some combination of too approaches may still be possible, but we are not working on this currently.

[loganj]

Any suggestions for how we might be able to adopt?

Is there any known way to resolve this at build time? e.g. specify dependencies from iOS modules onto kotlin libraries/modules, and from kotlin modules onto each other, and resolve to a single framework when building?

We have many, many places where shared code makes a great deal of sense, so we're motivated to solve this issue – but we've been working hard to modularize (for obvious reasons) and need to maintain that progress.

Edit: It would be helpful to better understand the "worlds" issue. Can you explain (or point to) a bit more about the ObjC/Swift custom class problem?

[SvyatoslavScherbina]

Are your iOS modules static or dynamic libraries?

Edit: It would be helpful to better understand the "worlds" issue. Can you explain (or point to) a bit more about the ObjC/Swift custom class problem?

Consider Kotlin module1, containing class A, and Kotlin module2, containing class B subclassing A. In our closed world compilation model all Kotlin code is supposed to be compiled into a single native binary, e.g. framework. So if it was possible to have framework1 with A and framework2 with B, then framework2 would contain its own version of A too, so there would be two A classes. This doesn't apply to some standard types like strings and collections (that's what I meant by non-custom), because these are mapped to standard Swift/Objective-C types.

[loganj]

Our modules are all frameworks (mostly defined in the same repo w/ cocoapods) which are statically linked into our executables.

Thanks for the (very clear) single-world explanation.

[beobyte]

The ability to include several frameworks in projects (in iOS, for example) that were created using Kotlin/Native will be very useful for developers who provide their solutions in the form of frameworks or libraries. In this case, consumers of such frameworks can use Kotlin/Native for their own code, as well as connect third-party solutions.

[xsveda]

I believe that the option to load multiple Kotlin frameworks would help the adoption of multiplatform. Our company is an agency that makes different apps for various customers. But some, mainly infrastructure parts, of the apps are the same so both Android and iOS teams have built many libraries to address that.

Android libraries are published and distributed through our company’s Maven Nexus. iOS libraries are distributed through CocoaPods.

We started to re-write some of these libraries to Kotlin Multiplatform. For Android it works still the same and projects, that depend on them do not need to change anything, they don’t even notice that the library is now shipped as multiplatform. We’d like to have similar experience for iOS projects as it is possible to build the multiplatform libraries to frameworks and also distribute through CocoaPods.

Due to the one Kotlin Framework limitation we can‘t just swap the libraries when the project uses more than one of them. Our current workaround is to publish the native variants of multiplatform libraries as .klib artifacts to Maven Nexus and each customer project has to create its own „integration“ Kotlin Multiplatform module for iOS Kotlin Multiplatform libraries, that depends on all libraries the project needs, export them and build framework that contains all Kotlin Multiplatform dependencies the project needs. It works but it requires a non-trivial change to the project structure that need to be maintained and this barrier don’t allow us to use this approach on more (all) projects we are currently working on.

[SvyatoslavScherbina]

@xsveda could you provide more details on your setup?

Are your libraries independent? E.g. do their APIs depend on each other or have common API dependencies?

[xsveda]

Are your libraries independent? E.g. do their APIs depend on each other or have common API dependencies?

The libraries are not independent and their APIs may depend on each other.

Our applications are well modularized (some have around 60 modules) and we use three layers of module types:

• library modules i.e. :logging, :printing
  • These modules are shared across various projects and might be extracted to public library
• generic modules i.e. :networking, :storage :locale
  • These modules are project specific but are shared across various :core modules
• core modules i.e. :products, :order, :payment
  • These modules are project specific and have the logic of single business use cases

Most of these modules are candidates to be re-done as multiplatform. They form a tree of dependencies where some of them are transitive. The root of the tree is an Android or iOS app that picks all the :core modules it needs.

[SvyatoslavScherbina]

Thanks! As I've explained above, Kotlin/Native follows closed world compilation model. However there is a way such a case may be eventually supported under this model: the entire "world" (meaning your collection of dependent modules) to be built from .klibs into a bunch of frameworks at the same time. Then one would be able to select consistent subset of these frameworks for an app, but frameworks from that single compilation would be compatible only with frameworks from the same compilation. I.e. different compilations wouldn't be mixable. (This doesn't exclude possibility of having different "worlds" independently). Does this sound suitable for your case?

[SvyatoslavScherbina]

Btw, do you distribute these libraries as different pods or as subspecs of a single pod? And could you clarify what do you mean by "requires a non-trivial change to the project structure that need to be maintained" here?

[xsveda]

Do you distribute these libraries as different pods or as subspecs of a single pod?

We distribute them as standalone pods and maintain them in separate repositories with separate versioning.

Could you clarify what do you mean by “requires a non-trivial change to the project structure that need to be maintained” here?

Our current aim is to start building libraries shared between different apps using Kotlin Multiplatform without any change to those projects. Current version of a library is being built from Swift codebase and distributed via Cocoapods - we want to build the next library version from Kotlin Multiplatform codebase. By non-trivial change I mean introducing the “integration” Kotlin Multiplatform module that puts all required Kotlin Multiplatform libraries into one framework for that particular iOS application.

Your suggestion to build all .klibs into frameworks at the same time is very interesting and might actually help us here. If I understand it correctly, we might set it up in the way where single “integration” project would fetch all the .klibs of all the libraries in our organisation’s projects & for each one of them would build a separate framework which could be integrated to the Swift project via Cocoapods. From the application project point of view nothing will change.

Am I understanding it correctly?

If so, few more details come to my mind:

1. How could we do the versioning of build set of frameworks? Let’s say we have lib-A v1.0 & v1.1, lib-B v1.0 & lib-B v1.1. In order to support all the combinations, we would need to maintain 4 build sets (A1.0 + B1.0, A1.1 + B1.1, A1.0 + B1.1, A1.1 + B1.1), which might quickly become unfeasible.

2. Let’s say we have 3 totally independent libraries. The only thing that is included in all of them is kotlin-stdlib. How many frameworks in a set built from them would be? One for every library and one common? Or a set of commons that would contain shared code just for libraries A+B, B+C, A+C and shared code for all A+B+C?

3. With this consistent set of frameworks build, do you see any option how to generate similar set of CocoaPods for delivery?

[epool]

could we use a Kotlin/Native module(let's name it A) as a kind of bridge and include in there all the other Kotlin/Native modules(let's name them B, C, etc.) and then only expose the A framework to iOS and cocopods?

[xsveda]

@epool yes, this works as described here

[SvyatoslavScherbina]

@xsveda Unfortunately, the idea described above doesn't look feasible with separate versioning. Do I understand correctly that you use different release cycles for your libraries?

[xsveda]

@SvyatoslavScherbina Yes, the ones that are shared across different applications have separate versioning and release cycle.

[sav007]

Hey folks, is there any updates? We (Shopify) sharing the same concern, single framework limitation destroys high modularization of shareable code. As for now we have only 2 KN modules that independent from each other but that for sure not scalable. Having a bridge module sounds like a hack solution and won't scale at large organization as well.

[SvyatoslavScherbina]

@sav007

As for now we have only 2 KN modules that independent from each other

Do you expect your modules to be independent further? Or form independent groups? Would support for multiple framework help you if it had the restriction described above?

but that for sure not scalable

Could you clarify?

Having a bridge module sounds like a hack solution and won't scale at large organization as well.

Is your Swift code (that consumes Kotlin modules) modularized too? If so, how do you manage dependencies between your Swift modules?

[sav007]

Do you expect your modules to be independent further? Or form independent groups? Would support for multiple framework help you if it had the restriction described above?

It's hard to say from our current state, as we still experimenting, but they might. For now we see couple independent KN modules that produces 2 iOS frameworks that will be consumed by iOS app. But with the current limitation we must generate one framework that will aggregate (depend on) all KN modules and produce one framework, right? That means we can't have them as independent modularized features.

Is your Swift code (that consumes Kotlin modules) modularized too? If so, how do you manage dependencies between your Swift modules?

All our swift code lives in monorepo, so basically we don't need to have package manager as any feature is subproject of the monorepo. We haven't decided how we will distribute frameworks produced by KN though, for now it might be "drag and drop".

[SvyatoslavScherbina]

Ok, thanks for the details. Supporting multiple independent frameworks doesn't conflict with any fundamental property of Kotlin/Native and is blocked mostly by certain technical issues (requiring some time to fix though).

[rharter]

Another use case that this issue prevents is the use of Kotlin multiplatform to develop SDKs.

Purely as an example, consider Firebase, which has client libraries for Android, iOS, Javascript and C++. It would be a huge win for Firebase, or others developing similar cross platform services, to be able to develop a single SDK in Kotlin, and distribute it to users on any of their platforms without the user having to know or care that it's a kotlin-mpp project.

An ideal scenario that I was looking into for a recent internal project of mine was to have something akin to a firebase-core modules (sticking with Firebase to avoid Foo). This would contain the functionality of the entire SDK and have lightweight interfaces to serve as the native api.

The android-sdk would simply expose the core api, while ios-sdk would be a swift project that depends on core, but exposes a more idiomatic Swift API that allows internal handling of deficiencies in Kotlin/Native, like the lack of full generic support. The js-sdk and cpp-sdk would be the same, simple pass-through if possible, or a thin API layer where needed.

Ideally one would be able to create such a project, and serve the ios-sdk as a Cocoapod, without the user having to know or care what technology it's built on. The fact that, if you attempt this currently, the build will fail if the user includes more than one Kotlin/Native based dependency blocks this type of use case.

In an ideal world, which I realize probably isn't feasibly at this point due the need for all K/N libraries to be compiled with the same version of K/N, the resulting assets, an aar for Android, a Cocoapod for iOS, a .so for C/C++, or an npm package for JS, would not contain the entire Kotlin world, but would depend on a stdlib style artifact that JetBrains maintains which provides that functionality. Then the native build systems could manage that dependency themselves.

Short of that, for the native side, at least, would it not be feasible to add to the generated header files a standard #ifndef KOTLIN_STD style declaration around the generated code for each exported module? This would be great for the Kotlin stdlib and types, but also for any exported dependencies so that if multiple Kotlin libraries depend on the same library (like SQLDelight, or ReactiveStreams), they won't conflict.

[SvyatoslavScherbina]

Another use case that this issue prevents is the use of Kotlin multiplatform to develop SDKs.

This use case is known and considered to be supported later.

In an ideal world, which I realize probably isn't feasibly at this point due the need for all K/N libraries to be compiled with the same version of K/N,

This is not correct. The requirement for the libraries is not the main blocker for implementing support for multiple dependent frameworks. See the discussion above for more details.

Short of that, for the native side, at least, would it not be feasible to add to the generated header files a standard #ifndef KOTLIN_STD style declaration around the generated code for each exported module?

This is also not the blocker for having multiple frameworks. Including the same library into multiple frameworks don't make their headers conflict.

[rharter]

Thanks for the details, @SvyatoslavScherbina. It's nice to hear that the use case is known.

This is also not the blocker for having multiple frameworks. Including the same library into multiple frameworks don't make their headers conflict.

I don't think this fits the problem. If you try to include two Kotlin/Native libraries, compiled as Objc/Swift Frameworks, in a single Xcode project (i.e. say Firebase was written using K/N, and you also included another K/N library via cocoapods) then you will experience compile errors as the headers do, in fact, conflict, as each framework will try to independently define the same classes, like KotlinBase, etc.

@interface KotlinBase : NSObject
- (instancetype)init __attribute__((unavailable));
+ (instancetype)new __attribute__((unavailable));
+ (void)initialize __attribute__((objc_requires_super));
@end;

@interface KotlinBase (KotlinBaseCopying) <NSCopying>
@end;

// Exported lib definitions

// Actual module definitions

I believe this is related to the "Whole World" idea mentioned above, in which each compiled K/N framework expects to supply the entire Kotlin world. This precludes library developers from allowing developers to consume libraries directly in their native projects, without an intermediary K/N project that outputs a single Framework, as mentioned above.

To solve the conflicting header issue, most C based libraries and frameworks using #ifndef macros to only define contents if they haven't been included and defined already. Assuming that is a blocker to including multiple K/N compiled libraries directly in a project via cocoapods (it is), it could be potentially alleviated in a similar fashion.

#ifndef KOTLIN_BASE
#define KOTLIN_BASE
@interface KotlinBase : NSObject
- (instancetype)init __attribute__((unavailable));
+ (instancetype)new __attribute__((unavailable));
+ (void)initialize __attribute__((objc_requires_super));
@end;

@interface KotlinBase (KotlinBaseCopying) <NSCopying>
@end;
#endif // KOTLIN_BASE

#ifndef KOTLIN_EXPORTED_LIB_1
#define KOTLIN_EXPORTED_LIB_1
...
#endif // KOTLIN_EXPORTED_LIB_1

#ifndef KOTLIN_MODULE_NAME
#define KOTLIN_MODULE_NAME
...
#endif // KOTLIN_MODULE_NAME

This would mean that if I have something like this in my cocoapods they'd be able to coexist:

pod "firebase-core-mpp"
pod "reactive-extensions-mpp"

Conditionally defining each exported module in each framework is important as it would allow multiple frameworks to rely on shared dependencies.

There are obvious problems with this. For instance, versioning would be quite difficult with this approach. An alternative might be to have the generated Cocoapod for a framework depend on a kotlin-native-base cocoapod, instead of hard coding the headers directly in the compiled binary.

[SvyatoslavScherbina]

as each framework will try to independently define the same classes, like KotlinBase, etc.

I have to repeat my statement:

Including the same library into multiple frameworks don't make their headers conflict.

KotlinBase is not a class from a library. It is special class, it is not supposed to be used directly and it will be easily renamed when supporting loading multiple frameworks.

To solve the conflicting header issue, most C based libraries and frameworks using #ifndef macros to only define contents if they haven't been included and defined already.

This approach is not correct here: Kotlin declaration emitted to different frameworks becomes different Objective-C declarations, you can't just import single version of them.

Assuming that is a blocker to including multiple K/N compiled libraries directly in a project via cocoapods (it is)

Formally speaking, having KotlinBase in multiple frameworks prevents one from importing them in the same file. However this is not the actual blocker.

Conditionally defining each exported module in each framework is important as it would allow multiple frameworks to rely on shared dependencies.

No, it wouldn't. Real limitation are much deeper and more complicated than simple conflicting declarations from headers.

[phirk]

This use case is known and considered to be supported later.

@SvyatoslavScherbina What's the timeline/roadmap for this?

Any workarounds (for the "closed world" principle) that enable us to continue developing our multiplatform SDK in Kotlin/Native (instead of reverting to Swift for iOS) that do not restrict our customers?

[wongk]

@phirk You will need one aggregate/umbrella MPP that pulls in each dependency, per app. If multiple apps use the same set of modules, they could reuse the same aggregate project.

[phirk]

@phirk You will need one aggregate/umbrella MPP that pulls in each dependency, per app. If multiple apps use the same set of modules, they could reuse the same aggregate project.

We distribute our SDK in binary format (frameworks for iOS) to our customers. Additionally, several of these customers don't use Kotlin, but Swift or Objective-C. So they can't make such an umbrella MPP. We neither, because wouldn't that mean creating a different SDK for each individual customer? (depending on the Kotlin MPP libraries they use)

[SvyatoslavScherbina]

@phirk

What's the timeline/roadmap for this?

No specific timeline or roadmap.

continue developing our multiplatform SDK in Kotlin/Native (instead of reverting to Swift for iOS) that do not restrict our customers?

How does anything restrict your customers now?

[phirk]

How does anything restrict your customers now?

Our customers, which write their App in Swift or Objective-C, and include our Kotlin/Native MPP library, cannot include any other library that is written using Kotlin/Native MPP as well.

Or did we misunderstand that?

[SvyatoslavScherbina]

This is correct. But does this actually restrict your customers currently?

[RockLobster]

@phirk You will need one aggregate/umbrella MPP that pulls in each dependency, per app. If multiple apps use the same set of modules, they could reuse the same aggregate project.

How do you even do that? We created an aggregate project to pull in 2 3rd party libraries, but even if we declare our dependencies as api dependencies, the created iOS framework ONLY exposes 3rd part library classes that are needed for usage of the public api of the aggregate.

Is there any way to expose all classes without manually writing fake code that references all possible public classes of the 3rd party libraries?

[phirk]

This is correct. But does this actually restrict your customers currently?

That depends on how serious JetBrains is taking this issue and when they will fix it.

Our company plans to write more libraries using Kotlin/Native MPP, but if this issue is not solved that might not happen, as our management might/will decide to revert to native iOS development.

[senagbe]

Hi, we have the same requirement regarding distributing libraries. How should we signal that this is a priority for us?

Is there an issue we can vote on?

Thanks in advance.

Sena

On Fri, 6 Sep 2019, 09:49 Rob, notifications@github.com wrote:

This is correct. But does this actually restrict your customers currently?

That depends on how serious JetBrains is taking this issue and when they will fix it.

Our company plans to write more libraries using Kotlin/Native MPP, but if this issue is not solved that might not happen, as our management might/will decide to revert to native iOS development.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/JetBrains/kotlin-native/issues/2423?email_source=notifications&email_token=AAA4K66Q4JAUYVXOP2HZFZ3QIIKQRA5CNFSM4GGP5B2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6CGKUQ#issuecomment-528770386, or mute the thread https://github.com/notifications/unsubscribe-auth/AAA4K65TNBX2RWHB4TGCSZDQIIKQRANCNFSM4GGP5B2A .

[SvyatoslavScherbina]

@phirk We take this serious enough. Simply allowing loading multiple independent Kotlin frameworks (is this your concern?) is planned for one of the next releases. Supporting multiple dependent frameworks (see discussion above) is much more complicated so I can't provide any specific details on this.

[phirk]

Can you be explicit about your definition of (in)dependent?

What if we create two libraries, A and B, that do not depend on each other, but both use a library C?

[SvyatoslavScherbina]

Please read the discussion above. For example: https://github.com/JetBrains/kotlin-native/issues/2423#issuecomment-466913808

In your case library C would be included into both frameworks under different namespaces. So you would be able to use both frameworks, but C classes from first framework would be incompatible with C classes from the second one.

[wongk]

Is there any way to expose all classes without manually writing fake code that references all possible public classes of the 3rd party libraries?

@RockLobster see https://kotlinlang.org/docs/reference/building-mpp-with-gradle.html#building-final-native-binaries, search for "Exporting dependencies in frameworks".

[wongk]

This is correct. But does this actually restrict your customers currently?

Creating a suite of modules with dependencies and making them available to 3rd parties seems like a very obvious use case to me.

[SvyatoslavScherbina]

Loading multiple independent Kotlin frameworks will be possible in 1.3.70.

[LouisCAD]

Does that mean that we will be able to publish libraries made in Kotlin through Cocoapods or Carthage for the whole iOS/macOS, linux, etc, communities to use, even if not used from Kotlin, or will there be some constraints?

[LouisCAD]

What if two Kotlin libraries depend on another same one (e.g. kotlinx.coroutines)? What about the stdlib?

[SvyatoslavScherbina]

Does that mean that we will be able to publish libraries made in Kotlin

Yes.

What if two Kotlin libraries depend on another same one (e.g. kotlinx.coroutines)? What about the stdlib?

Each framework is an isolated "closed world" and contains its own copy of every used library, including stdlib. That's what I mean by "independent" here. See more details above. Dead code elimination is applied as before, so "copy" here doesn't mean the entire library code included.