Porting my sbt to sbt-native-packager

[behrad]

Thank you @muuki88 for your nice work, can you help me port the following sbt to native-packager?

import com.typesafe.sbt.SbtAspectj._
import com.typesafe.sbt.SbtNativePackager._
import NativePackagerKeys._

// my normal sbt stuff ....

libraryDependencies ~= { _.map(_.exclude("org.slf4j", "slf4j-simple")) }

aspectjSettings

javaOptions in (run) <++= AspectjKeys.weaverOptions in Aspectj

javaOptions in (run) ++= Seq(
  "-Djava.library.path=src/main/resources/sigar",
  "-javaagent:" + System.getProperty("user.home") + s"/.ivy2/cache/org.jolokia/jolokia-jvm/jars/jolokia-jvm-1.2.2-agent.jar=port=8181",
//"-XX:+ExtendedDTraceProbes"
  "-Xms128m",
  "-Xmx3G"
)

val buildSettings = Defaults.defaultSettings ++ Seq(
  javaOptions += "-Xmx3G"
)

mainClass in Compile := Some("ir.behrad.adp.boot.Run")

packageArchetype.akka_application

Revolver.settings.settings

mappings in Universal <+= (packageBin in Compile, sourceDirectory ) map { (_, src) =>
  val conf = src / "main" / "resources" / "application.conf"
  conf -> "conf/application.conf"
}

1) I am using SbtAspectj to weave aspectJ and don't know how to integrate him with native-packager ?

2) I don't know how to add my jvm options without duplicating them to a conf/jvmopts

3) I read https://github.com/sbt/sbt-native-packager/issues/375 but didn't find the relation to sbt stage. Should I move my src/main/resources/sigar to where so that native-packager adds it?

[muuki88]

I am using SbtAspectj to weave aspectJ and don't know how to integrate him with native-packager ?

What's the problem? If you have compile-time weaving everything will just work? I'm not that of an expert with aspectj, so I never used runtime-weaving ;)

I don't know how to add my jvm options without duplicating them to a conf/jvmopts

Currently there is no default way to instruct sbt-native-packager to package the javaOptions in run. If you don't want to copy them, you can generate a task that does this for you, e.g. something like

lazy val TaskSetting[File]("etc-default-from-javaOptions") := {
     // -X options need a -J upfront
     val options = (javaOptions in run).value map { option =>
         if(option startsWith "-X") "-J" + option else option
     }
     val etcDefault = sourceDirectory.value / "templates" / "etc-default"
     // write the file
     sbt.IO.write(etcDefault, options mkString "\n", Charset forName "UTF-8")
     etcDefault
}

(not tested). See Configuration

Should I move my src/main/resources/sigar to where so that native-packager adds it?

I just pushed an example (forgot about it after the sbt-native-packager issue)

[behrad]

Thank you @muuki88 for your detailed answer, however I am a newbie to sbt-native-packager and have some serious questions

What's the problem?

1) I need runtime weaving and it is not woven in sbt stage with the above build.sbt, I may consider -javaagent option for this in my sbt-native-packager at last

(not tested). See Configuration

I couldn't use your snippet (I am not a sbt expert), so for starters I tried hand written src/templates/etc-default however I saw no changes in my generated startscript. I am using packageArchetype.akka_application archetype and sbt stage goal. Should sbt stage do this for me!? or should I need rpm:stage? rpm:packageBin ? I'm fooled with these :(

My primary need is not to use 'sbt run` which forks 2 JVMs in my production server, and found sbt-native-packager promising.

I just pushed an example (forgot about it after the sbt-native-packager issue)

I merged your build.sbt snippets with the above, but nothing included in my runtime after sbt stage

Pordon for my dumb questions....

[muuki88]

No problem ;)

My primary need is not to use 'sbt run`

This is not the goal of sbt-native-packager. sbt-np creates an application package in your desired packaging format (zip, tgz, deb, rpm, msi, docker). You can install this packaging with your tools of choice (dpkg -i, yum, unzip, .. ) or upload to a repository (nexus, artifactory, deb repo, yum repo, docker registry) and use a package manager/tool.

which forks 2 JVMs in my production server

If I get you correctly the 2 JVMs are sbt and your application, right? Assuming you have debian based system (debian, ubuntu) you could for instance

sbt debian:packageBin

This will generate a debian package you can simply install with sudo dpkg -i your-package.deb. Having said that, I'm still not sure about your requirements.

1. Did you choose the correct archetype? akka_application is a very special case
2. If you need runtime weaving you have to use the javaagent option. Also you need to include the jars which do the weaving in your library dependencies so they are available in your runtime system. Otherwise the files are woven during compile time, you don't see any change in the output (well, at bytecode level you do).

[behrad]

If I get you correctly the 2 JVMs are sbt and your application, right? Assuming you have debian based system (debian, ubuntu) you could for instance

Right, and I was happy with sbt stage on my local system if it worked, I'd be also happy be scp-ing my local target/universal/stage into my server. May stage not work for me but xyz:packageBin do?

Did you choose the correct archetype? akka_application is a very special case

I have a 3 node (clustered) akka application, one of which contains 2 HTTP services (one spray rest app + jetty-hawtio ), the other two are plain actor apps, akka Bootable suites me :) does that mean all these work only when using java_server ?

[muuki88]

Well, the etc-default only works with java_server as it expects an installed service.

With your setup you don't really need sbt-native-packager. However I would recommend you to look into the java_server archetype to install your applications instead of sbt running them.

[behrad]

I don't see why these don't work for akka_application. Does packaging and configuration stuff relate/depend to language or application internals? be it a java_server, or any other app, there should be main method for app to start, some config to read, ....

[muuki88]

This has nothing to do with your app internals, but the way you want to deploy it. Currently you aren't deploying anything, but using an intermediate output to run your application. sbt stage creates all necessary files, but no necessarily in the right place.

This is done by packageBin. And you have to choose an output format (debian, rpm, ...)

the archetypes have a simple purpose to preconfigure your settings so you can just run format:packageBin and get a working package. The akka_application is a bit of a special case as it comes with its own bash startup script and is very specific to akka.

java_app creates a simple, self-contained package that you can run. java_server extends java_app so you can install it like a service on server with all the default configuration you would normally have. This includes etc-default support.

Customizing of the bash-script can also be done. This can all be found inside the documentation. Good starting points maybe

• GettingStartedApplications
• GettingStartedServers
• Archetypes Overview

IF you want to use sbt running everything. I'll recommend to run it with sbt in the first place on your servers, too (but I don't recommend.)

[behrad]

good explanation @muuki88 but why not to add support for akka_application ? What prevents it from being a packed app which can be installed also as a service.

However the sbt stage would also be enough for me (to have all necessary files under a common directory)

and it seems pretty a normal requirement for sbt stage to support adding runtime configuration to any application (like etc-default, libsigar, ...) no matter what archetype is it of

[muuki88]

The akka_application is pretty new and everything was just copied from the akka-microkernel project, which is not further developed inside akka (AFAIK). So there's definitely room for improvement :)

As I said, sbt stage just creates everything that could be packaged. However it doesn't create any sort of application. E.g. you have less files, sbt stage will compile them to css and put the inside the target directory. At this point you don't have an application, just a bunch of "ready-to-be-packaged" files.

You always want to deploy a bundle. And if you just deploy a plain zip file that gets extracted. You create it with universal:packageBin.

However we had some discussions of using the javaOptions in run and process them to generate an etc-default that can be packaged. You can also add the configuration via the bashscriptExtraDefines (also documented), which is a bit messy, but works if you don't want to use the java_server.