Build without tests

[raphaeljoie]

Hi, I want to build my own jar of this great project but I'm not an expert in Scala and I hit an error in the tests each time I run sbt 'set test in assembly := {}' assembly

[error] (restSqsTestingAmazonJavaSdk / Test / test) sbt.TestsFailedException: Tests unsuccessful

Can you explain how to run sbt assembly without the tests in non-interactive mode ? Or how to update build.sbt to remove those tests ?

best regards,

You can recreate the issue with the following Dockerfile

FROM openjdk:8-alpine

ENV SBT_VERSION  1.1.4
ENV SBT_HOME /usr/local/sbt
ENV PATH ${PATH}:${SBT_HOME}/bin

RUN apk add --update \
    git \
  && rm -rf /var/cache/apk/*

# Install sbt
RUN apk add --update curl ca-certificates bash && \
    curl -sL "https://piccolo.link/sbt-$SBT_VERSION.tgz" | gunzip | tar -x -C /usr/local && \
  echo -ne "- with sbt $SBT_VERSION\n" >> /root/.built &&\
  apk del curl

# Define working directory
WORKDIR /app

RUN git clone https://github.com/adamw/elasticmq.git . && git checkout tags/v0.13.10

RUN sbt exit
RUN sbt 'set test in assembly := {}' clean assembly

error is

18:35:06.871 [elasticmq-akka.actor.default-dispatcher-5] ERROR o.e.rest.sqs.TheSQSRestServerBuilder - Cannot start SQS rest server, bind address :9322
akka.stream.impl.io.ConnectionSourceStage$$anon$2$$anon$1: Bind failed because of Address in use
Caused by: java.net.BindException: Address in use
    at sun.nio.ch.Net.bind0(Native Method)
    at sun.nio.ch.Net.bind(Net.java:433)
    at sun.nio.ch.Net.bind(Net.java:425)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
    at akka.io.TcpListener.liftedTree1$1(TcpListener.scala:56)
    at akka.io.TcpListener.<init>(TcpListener.scala:53)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at akka.util.Reflect$.instantiate(Reflect.scala:65)
    at akka.actor.ArgsReflectConstructor.produce(IndirectActorProducer.scala:96)
    at akka.actor.Props.newActor(Props.scala:213)
    at akka.actor.ActorCell.newActor(ActorCell.scala:626)
    at akka.actor.ActorCell.create(ActorCell.scala:652)
    at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:525)
    at akka.actor.ActorCell.systemInvoke(ActorCell.scala:547)
    at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:282)
    at akka.dispatch.Mailbox.run(Mailbox.scala:223)
    at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

[adamw]

sbt "set test in assembly := {}" server/assembly works for me - you only want to build the assembly for the server project.

The result will then be in server/target/scala-2.12/elasticmq-server-assembly-VERSION.jar

[raphaeljoie]

Thanks a lot. It's working as expected. Open question : why did you decide to load the jar from the mirroring site, instead of building the project directly in the Docker image ?

Here is my final Dockerfile:

FROM openjdk:8-alpine

ENV SBT_VERSION  1.1.4
ENV SBT_HOME /usr/local/sbt
ENV PATH ${PATH}:${SBT_HOME}/bin
ENV VERSION 0.13.10

RUN apk add --update \
    git \
  && rm -rf /var/cache/apk/*

# Install sbt
RUN apk add --update curl ca-certificates bash && \
    curl -sL "https://piccolo.link/sbt-$SBT_VERSION.tgz" | gunzip | tar -x -C /usr/local && \
  echo -ne "- with sbt $SBT_VERSION\n" >> /root/.built &&\
  apk del curl

# Define working directory
WORKDIR /app

RUN git clone https://github.com/adamw/elasticmq.git . && git checkout tags/v${VERSION}

# do a first run to isolate sbt initialisation to the other sbt commands
RUN sbt exit
RUN sbt "set test in assembly := {}" server/assembly

COPY custom.conf custom.conf

RUN echo "#!/bin/sh" > entrypoint.sh \
    && echo "java -Dconfig.file=custom.conf -jar server/target/scala-2.12/elasticmq-server-assembly-${VERSION}.jar" >> entrypoint.sh \
    && chmod +x entrypoint.sh

ENTRYPOINT ["./entrypoint.sh"]

EXPOSE 9324

[adamw]

Well I suppose the Docker image could run the build on the host, and use that - however I'm not the author of the Dockerfile, so I can't say what the exact reasoning was.

I don't think there's normally a need to build the project in docker? And even if, the image to build the server would be different from the image containing the server. That latter doesn't need to include sbt, for example, only java.

[raphaeljoie]

You're right, including jar directly in the Docker image has the same effect. The advantage to put the build instructions in the Dockerfile is to provide a complete working example of this process.

Anyway, great library ! I tested it this weekend and will certainly use it for tests in CI.

[adamw]

@raphaeljoie Right, though isn't it still the "standard practice" to build outside of docker, and only create docker images with the build artifacts, vs. building inside the docker image?

Anyway, at least documenting how to build locally is definitely a good idea :)

[raphaeljoie]

To be honest, I don't know about this practice since I've never published any Docker image. I only use it for dev, tests and internal deployments. that said, I could understand this practice since it's way faster to download a package than to build it from scratch.

From my experience I know that the build process of an opensource library often fails because of a slight environment change. No matter the quality of the doc. And I personally find it easier to get the build I want if the repo's owner gives the exact commands he used to gets its own, starting from git clone to the final run.sh in a neutral environment (i.e. a Docker image).

but maybe the best solution to keep the image fast to build is to put the "docker-like" instructions directly in the doc ?

[adamw]

I'm no expert on docker best practices as well :) I think it might make sense to have two docker images: one for building the project, the other for using it. No idea how to implement this though :)

Adding instructions in the docs is a great idea - could you submit a PR maybe?