NativeScript Plugins

[ligaz]

State of affairs

Currently NativeScript provides support for what we call modules. Those are CommonJS compatible JavaScript modules that usually expose underlying native platform APIs by providing a common JavaScript hat on top of them. Out of the box we provide support for all platform stock APIs (part of the native SDKs) and we do have the option to reference a 3rd party libraries via the tns library add command.

Introducing NativeScript Plugins (Modules ++)

Writing real world application requires much more than what we currently provide in the form of NativeScript modules. A common use case is that a module usage it will require a change in a platform specific configuration file. Here are a two examples that illustrates this:

• The canonical one is geolocation which requires adding permissions in AndroidManifest.xml as well as a key in the Info.plist file for iOS.
• Another example is adding a chart component to your application. Charts are usually not available in the stock SDKs so you will be required to use a 3rd party library and here comes the problem if you want to go cross-platform. You will have to add each platform library separately and then expose a common JavaScript API. What if you want to share this with someone else?

There are a lot of other examples that can be given to proof that we need something more than what we currently have in the form of modules. We want a common abstraction that will allow us to easily package and redistribute those extended modules. We want to call this NativeScript Plugin.

What is a NativeScript plugin?

NativeScript plugin is a package that consist of the following:

• Metadata about the package itself: name, version, supported runtime versions, etc.
• CommonJS module (one or many) that expose the native API via a single JavaScript API.
• Declarative way of defining transformations for platform specific configuration files.
• Native libraries and resources.

The package format should be extensible so we can add new capabilities in the future.

Implementation details

The key thing about the implementation is that we will strive to sit on top of existing standards and we will use conventions over configurations.

Packing and distribution

We are already discussing that we want to standardize on NPM as distribution platform. It is logical that we want to use it for NativeScript plugins as well.

We can use an upcoming feature of NPM called ecosystems by adding ecosystem:nativescript in the keywords section of the plugin's package.json. Eventually NPM will provide dedicated search and package weighting for the ecosystem. More info on ecosystems can be found here.

We can use the engines key to designate which version of the runtime the plugin supports:

{ "engines" : { "nativescript" : ">=0.9 <0.10" } }

or

{  
  "engines" : { 
    "nativescript-android" : "0.9.0",
    "nativescript-ios" : "0.9.3" 
  }   
}

JavaScript module

The JavaScript module loading will use the same technique we are currently using in the runtimes when a module is required. If a JavaScript file is specified in the package.json's main key we will load that otherwise we will look for index.js file in the root.

Native libraries

For referencing native libraries we can use a convention we are currently using in the CLI. We can have a platforms folder with sub-folder for each platform. We can then search the platform specific folders and execute add library upon installation of the plugin. Here is an example of a sample chart plugin's file structure:

nativescript-charts/
|-- platforms/
|-- |-- Android/
|-- |-- `-- charts.jar
|-- |-- iOS/
|-- |-- `-- ChartKit.framework
`-- index.js/
`-- package.json

Platform configuration files transformations

Usually those platform configuration files are XML based (AndroidManifest.xml, Info.plist, etc.) which can be tricky to incorporate in our JavaScript/JSON based tooling. Here are two possible ways we can approach this.

Incorporate the transformations in package.json

We can use the package.json file to declare the required transformation we want to apply. For example we can have the following:

{
  "name": "nativescript-geolocation",
  "version": "0.9",
  "nativescript": {
    "android": {
      "permissions" : [
        "android.permission.ACCESS_COARSE_LOCATION",
        "android.permission.ACCESS_FINE_LOCATION"
      ]
    }
  }
}

The problem with such definition is that it is is not general purpose and is very platform specific. We will have to embed the knowledge that the permissions has to go in AndroidManifest.xml and also that they will have to be inserted as <uses-permission> tags in the XML. The same goes if we want to insert other elements. In order to make this more generic we will have to tweak the JSON a little bit:

{
  "name": "nativescript-geolocation",
  "version": "0.9",
  "nativescript": {
    "android": {
      "AndroidManifest" : {
        "uses-permissions" : [
          { "android:name" : "android.permission.ACCESS_COARSE_LOCATION" },
          { "android:name" : "android.permission.ACCESS_FINE_LOCATION" }
        ]
      }
    }
  }
}

Having this structure we can transform the JSON to XML in a generic way and just merge the transformed XML segments into the AndroidManifest.xml file. The downside of this is that the JSON becomes more verbose and the processing itself can be more complex. Note: For plist files we will have to create a separate output generator due to the strange structure of the property list.

Provide XML segments in designated file per platform

We can use /platforms structure of the plugin and place platform specific XML files in each platform directory. Here is an example:

nativescript-geolocation/
|-- platforms/
|-- |-- Android/
|-- |-- `-- AndroidManifest.xml
|-- |-- iOS/
|-- |-- `-- Info.plist
`-- ...

The XML files will contain just the segments we want to merge in the final configuration file. This technique give us better flexibility but will require a little bit more setup from plugin developers.

Update : There is an agreement that the approach with separate config files and merging is the one we should go after.

Other use cases

Here are some other use cases that can be considered for future implementation.

Including native source code files

There might be cases when we need to package native source code files. These cases should really be rare ones as by definition the native code should be able to be expressed in JavaScript but nevertheless. By convention those could go into the specific platform sub-folder of /platforms and will be included in the native projects for compilation.

Using packages from the platform's package manager

Each platform has its own package (dependency) manager and it would be good if we can declaratively include packages from them to be used from JavaScript. The de-facto package managers are Cocoa Pods for iOS and jCenter (Maven Central) for Android. Here is a an example of a possible package.json declaration of those:

{
  "name": "nativescript-http",
  "version": "0.9",
  "nativescript": {
    "android": {
      "dependencies": {
        "com.github.kevinsawicki:http-request" : "6.0"
      }
    },
    "ios" : {
      "pods" : {
        "AFNetworking" : ">=2.0"
      }
    }
  }
}

Note: In order to support Android packages we will have to migrate the Android tooling to Gradle.

Prior art

Apache Cordova's plugin format

Cordova uses a single plugin.xml file for its plugins definition. It is very verbose but handles most of the cases described above. Cordova is also moving to NPM for package distribution.

Telerik AppBuilder's .abproject file

AppBuilder uses JSON based project definition which includes Android, iOS and Windows Phone specific properties. Those are processed in a custom way for each platform and there is no generic way to define a platform specific property if it is not part of the JSON schema format defined for the project.

[vjeux]

Subscribe, we're currently in the same process of searching for a plugin format for React Native. This post is inspiring :)

[tjvantoll]

Awesome stuff here @ligaz. I like the distinction between modules as individual files (or a few files if you use .android.js, .ios.js, etc), and plugins as installable packages.

Usually those platform configuration files are XML based (AndroidManifest.xml, Info.plist, etc.) which can be tricky to incorporate in our JavaScript/JSON based tooling. Here are two possible ways we can approach this.

This is tough, but of the two options I actually prefer the XML segment approach, aka this one:

nativescript-geolocation/
|-- platforms/
|-- |-- Android/
|-- |-- `-- AndroidManifest.xml
|-- |-- iOS/
|-- |-- `-- Info.plist
`-- ...

We're going to have two types of developers building these plugins: those with native iOS/Android experience, and web developers with little to zero native experience. The native developers already know how these configuration files work and will want to write in those syntaxes—not learn some new abstraction we create.

Web developers are going to Google how implement “feature x on Android/iOS”, and the docs they'll find will talk about the raw configuration files, e.g. AndroidManifest.xml and Info.plist. By using XML snippets directly, these web developers can just copy and paste from the internet and avoid having to translate the configuration changes into a JSON format we create.

Using XML snippets also avoids the mess of having to specify Info.plist changes in JSON, which makes me cringe just thinking about it :)

[fealebenpae]

How would we go about handling dependencies? Say we've got plugins A and B, which both depend on different versions of plugin C, which includes some native artifact. We can only ever have a single copy of C because of its native code, but which one are we gonna choose?

I really like npm as a distribution mechanism, but npm doesn't have to solve this problem because node is happy with infinite copies of a module.

[ligaz]

Good point about handling dependencies. Something that is intentionally not mentioned in the issue is how plugins will be installed via the NativeScript CLI. There will be a separate command tns plugin add that will fetch the plugin from NPM and execute the installation procedure. Having this allow us to process the plugin dependencies and if those are NativeScript plugins we can install (via tns plugin add) them as well. This yields that NativeScript plugins will be installed top-level only if you look from the lenses of a native project.

Regarding the dependency resolution mechanism we can construct the whole dependency graph and resolve the correct dependencies. If the resolution is impossible we will error out the user and she will have to fix it. Here is how Cocoa Pods are doing this.

[fealebenpae]

Yes, this will certainly work if we use the NPM infrastructure but not the NPM tooling to install plugins, although I'm in favor of going NPM all the way, down to the tooling level, as it's got a lot of traction and momentum behind it.

I like the idea of building a dependency graph - and Molinillo looks like a great starting point for a dependency resolver of our own. I think prepare is a good place to analyze both the interdependencies between plugins and the dependency of a plugin on certain platform aspects such as minimum supported API level or SDK version. I want to suggest prepare because the invariants of a plugin could change between adding it and building the project, and then we'll end up with a build or runtime error that's hard to track down.

I have a thought on how our tooling would add the native pieces of a plugin to the platform-specific build system. I would like to propose that we deal with this at the plugin tooling level, rather than defer to library add, because this would enable more scenarios such as static libraries on iOS as library add can't deal with those directly, as well as adding native source code files. Eventually I'd like to phase out library add in favor of plugins, and transition it into plugin-scaffolding tooling.

[ligaz]

Relying solely on NPM infrastructure can be tricky because the only way we can execute custom code upon installation is via install script. This means that each plugin should bundle our installation script which is nightmare to support. Also having the plugin installation code in single place (in the NativeScript CLI or a library that it uses) will give us better flexibility to support it and add new functionality.

Analyzing dependencies in prepare could lead to performance problems if you have many plugins. Also the purpose of prepare is to update your app assets in the native projects. We should try to keep the prepare step as lean as possible. If the only official way to mangle the native project is plugin add we can guarantee its correctness.

I'm also in favor of gradually phasing out the library add command. I was just referring to it as an abstract entity.

[atanasovg]

Great proposal and discussion!

As for removing the library add command - if we agree that what the command currently does will be the minimal supported plugin, then it is fine to obsolete it. Otherwise we definitely do not want to disable our users to refer, let's say, a plain *.JAR file directly, but instead force them to wrap it in a plugin before we may consume it.

[ligaz]

I agree that the library add command has its value especially when you want to target a single platform. We should definitely keep it around until it is used but we should advice that the preferred way will be to use a plugin, because we can give so much more with it.

Note: I just looked at analytics data and we have just 63 usages of the library add command for the last month.

[fealebenpae]

The rationale behind my wanting to sunset library add is that build-time metadata generation will obsolete it. Right now it exists mostly to automate generating metadata for native libraries. With that out of the way in the new builds, it would be better to just let developers add a native library as they normally would (e.g. by dropping a jar in the correct place) instead of providing a command that will be little else but a glorified file copier. I'm days away from gutting the metadata generation step out of library add for iOS in any case.

[teobugslayer]

It would be an interesting challenge sandboxing npm install scripts for the AppBuilder build infrastructure.

[atanasovg]

My only goal is to consider the simplest scenario together with the more complex setups. With that said, if we can consume plain libraries as plugins then we are good to remove the library add command. Putting a jar manually into the platforms/android folder is something we do not want our user to do. Not to mention the Android library projects - no way to manually refer these (except if we advise for project.properties file edits).

How about some kind of automated generation of plugins - e.g.

tns plugin generate <path-to-library-folder>

This way we will speed-up the initial preparation of the plugin directory structure, put the library in the proper place and generate the required plugin metadata (not the Runtime metadata).

[pavele]

The closer it is to npm the better - https://docs.npmjs.com/cli/init. This way all the npm (and bower) users will immediately feel at home.

One other thing is that if we have both modules and plugins it will become a mess. Think of a real application there you'll have your own app components (authentication, data, etc), also modules then the tns modules and also plugins

Right now the modules are limited and what ligaz suggests is the natural evolution to way have a real modular system.

[fealebenpae]

This way we will speed-up the initial preparation of the plugin directory structure, put the library in the proper place and generate the required plugin metadata (not the Runtime metadata).

Yes, that's exactly what I had in mind when I suggested to transition library add into plugin tooling. I'm glad we had the same thought, @atanasovg :smiley:

[ligaz]

:innocent: Shameless plug: "Great minds think alike"

[zephraph]

Native Configuration

I'm with @tjvantoll in that I'd like to see a segmented approach. I'd rather have the tooling figure out native configurations than have to bootstrap a bunch of options in the package.json. If I'm writing native code I'll be using native tooling which will generate native configuration. Let's not force extra configuration through json.

Native Dependencies

A big discussion we need to have is when a user should package up native code within a plugin.

My position is that if you're wanting to depend on a native library then that library shouldn't be packaged with your plugin. So if you're using android and you want some native charting library then you should specify the dependency (in the package.json I'm assuming). You should be able to pull this from jCenter, Maven Central, etc. Maybe the android dependency section could directly map to gradle dependencies.

When a user decides native code should be packaged with their plugin, I think there should be some namespace enforcement or something to ensure there aren't collisions. The native code should be somehow sectioned off and the plugin should export an api in the traditional CommonJS style. The idea here is that if native code is packaged with a plugin, it's specific to that plugin and not meant to be reused in its native form by other plugins.

Ultimately this would mean that you couldn't redistribute native code in a NativeScript plugin without some slight modification. With that being said I'll go back to my original point: 3rd party native dependencies that are used in your plugin should come from places that actually managed native dependencies. Not NPM.

Lastly, I'm on the fence about removing add library. In some sense I see a project and plugin being quite similar. Both should be able to have 3rd party native dependencies (like jCenter as mentioned previously), and they each should be able to include custom native code. I think the custom native code makes complete sense to stuff in a generated plugin. A project should also just be able to pull down a random library that isn't in a place like jCentral and still be able to use it though. A library like this isn't meant to be redistributed and thus doesn't make sense to place in a plugin. Indeed, by my earlier requirement of namespacing native code inside a plugin it wouldn't work to just drop an unaltered library into a plugin and run with it. Now, if all the functionality of add library is taken out and moved to build time then I'd be fine with just telling people "Hey, dump your unpackaged 3rd party native libraries here."

[ligaz]

Thanks for your input @zephraph

Regarding configuration files everyone is leaning towards the separate files and not trying to reinvent the wheel in package.json. I guess I have to update the issue to note this.

You raised some valid concerns about native code integration. As outlined at the end of the issue we want to support including packages from native package managers. This means that if two plugins want to depend on the same native package we will use the built-in resolution mechanism of the package manager. The sample package.json I have given at the end is a very realistic one as it uses two of the most popular native (Android and iOS) libraries for working with HTTP. The dependencies section of the android key directly relates to the dependencies in the Gradle file and the pods section directly relates to the Podfile. This means that when tns plugin add command is executed the NativeScript CLI will update the build.gradle and Podfile to include the listed native packages.

However there are cases when the native library cannot be published to a public package manager (ex. proprietary source code) and we want to support this scenario as well. That's the reason you should be able to just add the library in the /platforms folder of the plugin and it will be included in your native project upon tns plugin add. If we detect that two plugins are adding the same library we can error out and provide guidance to the user how this can be fixed.

[zephraph]

Okay, cool. That's in line with most of my thoughts.

What about native code availability? If you add native code to a plugin will it be available to other plugins?

[jamlfy]

@ligaz Interestingly enough, this in a JSON exactly the package.json, especially that there are libraries that are not exactly NativeScript however help programming. However how it would be if the same library works for both? And based thereby mount all librarys NPM?

[ligaz]

Hey @alejonext, can you give me an example with a particular library and I can try to write it down for it?

[jamlfy]

@ligaz The best example would be Admob, as have library for Android and iOS, and managed different and complementary programming is different. Although it believes it best to create a library for NativeScript

[RangerMauve]

Would splitting up the NativeScript-specific modules to be installed via NPM be related to this? More specifically, I'm referring to the existing modules being used.

[ligaz]

@alejonext Yes, the best way for Admob integration is to create a NativeScript plugin that uses the native libraries and exposes a unified JavaScript API on top of them.

@RangerMauve Yes it is related. In the long run each core NativeScript modules might make more sense to be a separate plugin but we are yet to figure this all out. I guess we will start with the usual suspects like geolocation which require their own Android permissions and modifications to the Info.plist file.

[jamlfy]

@ligaz Framework is that everything is not divided into small modules, great would seem that continue to maintain a model like Cordova. Thus, the size of the applications is reduced, that is there or org.nativescript.camara or org.nativescript.geolocation. So one could use this method, of course this is a proposal !!!!

[PanayotCankov]

Google Maps SDK for iOS

We have prepared a handmade example with the Google Maps SDK for iOS used in NativeScript for iOS.

Their SDK is packaged as static framework and has quite a few steps to setup. This is what is interesting for us to support in our packaging mechanism, for example declaratively describing the following steps in the package json:

Adding Static Frameworks

"2. Drag the GoogleMaps.framework bundle to your project. When prompted, select Copy items if needed."

Adding Bundle Resources

"4. Drag the GoogleMaps.bundle from the Resources folder to your project. When prompted, ensure Copy items into destination group's folder is not selected."

Add Dependencies

"6. Open the Build Phases tab, and within Link Binary with Libraries, add the following frameworks:

• AVFoundation.framework
• CoreData.framework
• CoreLocation.framework
• libc++.dylib
• libicucore.dylib
• libz.dylib
• ..."

Add Flags To a Target or Project

"7. Choose your project, rather than a specific target, and open the Build Settings tab.

• In the Other Linker Flags section, add -ObjC..."

Stripping

Some C functions and constants will not be linked in the App and therefore won't be accessible from JavaScript. If this proves to be a problem we will have to research a solution. We may have to force the linker to add these symbols.

Dynamic Frameworks

Interesting approach would be to try and pack such static frameworks in dynamic frameworks, so that they could describe their dependencies, and have less configuration in the package.json.

[ligaz]

Thanks for this Pana and right on time as we are currently implementing this. We will make sure that this use case is handled as well.

[PanayotCankov]

Further this is our findings from the CocoaPods in NativeScript for iOS research.

It seems that CocoaPods now can generate dynamic/shared frameworks, and the integration with the iOS runtime is quite smooth. We automatically pick such frameworks and generate metadata for them. Using shared framework however puts the constrain to use iOS8+. With WWDC approaching it may be reasonable to support cocoa pods when targeting iOS8+.

[jamlfy]

@PanayotCankov What you can do is build a method of release, the plugin automatically. Thus, there will be no errors for the plugin.

[NathanaelA]

Just a small interjection on my sqlite plugin in; on the android side I didn't need to include anything. It is pure javascript. However on the iOS side I had a similar issue as @PanayotCankov in that you have to open up xcode environment; add a already present (installed with xcode) sqlite3.dylib to the project. AND I also had to create a module.modulemap that has to be put into the /platform/ios folder (has to be in the path for the metagenerator).

So this is another thing that may need to be coordinated is the module.modulemap file in case their are a couple plugins that require modulemap..

In addition a second question; are plugins going to be able to overwrite / add any native functionality or is this something totally out of scope. For example say I want to overwrite the "require" (which is Java code on Android). My LiveSync currently has to have a runtime patch that we have been debating in the Android Runtime Pull Request for it to do its magic of being able to live edit the code and see the changes on the emulator in real time.

[jamlfy]

@Nathanaela I understand your proposal, what I feel is that there is a fairly complex problem standard. The solution should be more consistent and that does not affect developers using plugins. And finally users the applications they will demonstrate whether this works. In short, the question is "How will you run the third-party libraries?"

[NathanaelA]

@alejonext - I assume english must not be your primary language -- because I am having a hard time understanding what you are asking/saying.

The .dylib & module.modulemap issue I assume will be solved in the upcoming plugin changes; as I can see this happening a lot. I just wanted to make sure it didn't get forgotten in the other static, cocoapods and dynamic library discussions.

Now as for the ability to create plugins that actually override native behavior (i.e. like replace "require" and add my needed __ClearRequireCachedItem) those are something I'm not 100% is in scope for this specific project -- that is why I asked if it was in scope. Is that what you were saying you agree that this is probably out of scope for this issue?

[jamlfy]

@Nathanaela You're right, English is not my primary language. I feel that there is much uncertainty about puligns, and its development. Not if this is resolved when the time.

[RangerMauve]

@alejonext I think the 1.2 release is supposed to have better plugin support coming with it.

[adaptivedev]

Is there something to do native FB login, access camera, GPS, etc?

Can we use Cordova plugins in the meantime? How?

[dtopuzov]

{N} already support plugins, for any further discussions please use {N} Google group: https://groups.google.com/forum/#!forum/nativescript

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.