App Engine flex JAR new project wizard

[chanseokoh]

For Maven projects, a Spring Boot app would be simple to bring in. Not so decisive for native projects, as a simple Spring Boot app would still require a load of JARs. Also it's hard to imagine not using Maven/Gradle with Spring Boot.

[elharo]

I'm not sure this should be SpringBoot based. I was thinking a bare minimum Hello World app. SpringBoot might be something we could address through docs and/or a separate wizard.

[patflynn]

Hello World servlet app? How would you make that work without springboot?

On Tue, Oct 10, 2017 at 1:41 PM, Elliotte Rusty Harold < notifications@github.com> wrote:

I'm not sure this should be SpringBoot based. I was thinking a bare minimum Hello World app. SpringBoot might be something we could address through docs and/or a separate wizard.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/2470#issuecomment-335551855, or mute the thread https://github.com/notifications/unsubscribe-auth/AHf5HerDjnHS24OXEbHSC8fOig9VoaXmks5sq6w6gaJpZM4P0Qs8 .

[elharo]

Springboot is not required for flex. I want the bare minimum of code and dependencies to make a flex app work.

[chanseokoh]

This will respond to web browsers, but if the code is too minimal like this, I think it won't help users get started.

public class HelloWorldServer {

  public static void main(String[] args) {
    try (ServerSocket serverSocket = new ServerSocket(8080)) {
      while (true) {
        try (Socket socket = serverSocket.accept();
            OutputStream out = socket.getOutputStream()) {
          out.write("HTTP/1.1 200 OK\n\nHello World!\n\n".getBytes(StandardCharsets.UTF_8));
        }
      }
    } catch (IOException e) {
      System.err.println("Server going down due to error:");
      e.printStackTrace(System.err);
    }
  }
}

[chanseokoh]

Also, it doesn't have to be a web application, although that would be one of the easiest and most popular choices. For example, an endpoint could just serve a JSON, without the HTTP transport. Or, it could be an FTP server.

[elharo]

The advantage of a truly minimal app is that it lets users build in directions that aren't anticipated.

That said, I think we could add a servlet engine, probably Jetty, to the basic example.

[elharo]

Roughly something like this should suffice:

https://github.com/GoogleCloudPlatform/getting-started-java/tree/master/helloworld-servlet

though I think we can make this example a little simpler.

[chanseokoh]

I like the idea to use an embedded Jetty. Simple code, yet easy to extend to do useful tasks, with the servlet technology readily available.

The sample you linked is a flex WAR project, packaging and deploying a .war file. I think something like below will work. (The final code might be slightly different depending on which Jetty version we pick.)

public class HelloWorldServer {

    private static class HelloWorldHandler extends AbstractHandler {
        @Override
        public void handle(String target, Request baseRequest,
                HttpServletRequest request, HttpServletResponse response)
                throws IOException, ServletException {
            // Declare response encoding and types
            response.setContentType("text/html; charset=utf-8");

            // Declare response status code
            response.setStatus(HttpServletResponse.SC_OK);

            // Write back response
            response.getWriter().println("<h1>Hello World</h1>");

            // Inform jetty that this request has now been handled
            baseRequest.setHandled(true);
        }
    }

    public static void main(String[] args) throws Exception {
        Server server = new Server(8080);
        server.setHandler(new HelloWorldHandler());
        server.start();
        server.join();
    }
}

[patflynn]

How do you handle the extra Jetty dependency? Are we going to create our own uber-jar?

On Tue, Oct 10, 2017 at 5:06 PM, Chanseok Oh notifications@github.com wrote:

I like the idea to use an embedded Jetty. Simple code, yet easy to extend to do useful tasks, with the servlet technology readily available.

The sample you linked is a flex WAR project, packaging and deploying a .war file. I think something like below will work. (The final code might be slightly different depending on which Jetty version we pick.)

public class HelloWorldServer {

private static class HelloWorldHandler extends AbstractHandler {
    @Override
    public void handle(String target, Request baseRequest,
            HttpServletRequest request, HttpServletResponse response)
            throws IOException, ServletException {
        // Declare response encoding and types
        response.setContentType("text/html; charset=utf-8");

        // Declare response status code
        response.setStatus(HttpServletResponse.SC_OK);

        // Write back response
        response.getWriter().println("<h1>Hello World</h1>");

        // Inform jetty that this request has now been handled
        baseRequest.setHandled(true);
    }
}

public static void main(String[] args) throws Exception {
    Server server = new Server(8080);
    server.setHandler(new HelloWorldHandler());
    server.start();
    server.join();
}

}

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/2470#issuecomment-335608208, or mute the thread https://github.com/notifications/unsubscribe-auth/AHf5HRlfNs-sDcYvV0mXRFbdM289ipvbks5sq9xmgaJpZM4P0Qs8 .

[chanseokoh]

The App Engine flex currently requires a single fat JAR, so yes, we have to create a uber JAR, which is also what Spring Boot does. However, it would have been ideal if the Java runtimes supported multiple JARs (https://github.com/GoogleCloudPlatform/runtime-builder-java/issues/14).

For Maven projects, there are a few plugins we can use to create a fat JAR AFAIK (although I haven't tried them yet), so I just hope it is not difficult to do so.

For native projects, althought the concept of creating a fat JAR is simple, we don't have a good, scaleable, and general solution.

[patflynn]

the native scenario is my biggest concern with this approach.

On Wed, Oct 11, 2017 at 9:55 AM, Chanseok Oh notifications@github.com wrote:

The App Engine flex currently requires a single fat JAR, so yes, we have to create a uber JAR, which is also what Spring Boot does. However, it would have been ideal if the Java runtimes supported multiple JARs ( GoogleCloudPlatform/runtime-builder-java#14 https://github.com/GoogleCloudPlatform/runtime-builder-java/issues/14).

For Maven projects, there are a few plugins we can use to create a fat JAR AFAIK (although I haven't tried them yet), so I just hope it is not difficult to do so.

For native projects, althought the concept of creating a fat JAR is simple, we don't have a good, scaleable, and general solution.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/2470#issuecomment-335817733, or mute the thread https://github.com/notifications/unsubscribe-auth/AHf5HSIIVOHw20lN1kjS6nJ--eD8RRnKks5srMjMgaJpZM4P0Qs8 .

[briandealwis]

Perhaps we could use com.sun.net.httpserver.HttpServer?

[chanseokoh]

I guess the native scenario needs more research. Using Spring Boot doesn't help either, as it's an irrelevant matter. For the initial iteration, we'll focus on the Maven case. It's a bit unclear at this point how the native case will turn out, but I guess there will be something we could do.

[chanseokoh]

HttpServer could work very well too. We still have to bundle an external JAR, as the package starts with com.sun.net, right? However, the Jetty example above is already pretty simple, and it's more capable as it allows users to use servlet APIs.

[elharo]

No, it actually is bundled with the JDK.

On Wed, Oct 11, 2017 at 10:49 AM, Chanseok Oh notifications@github.com wrote:

HttpServer could work very well too. We still have to bundle an external JAR, as the package starts with com.sun.net, right? However, the Jetty example above is already pretty simple, and it's more capable as it allows users to use servlet APIs.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/2470#issuecomment-335836532, or mute the thread https://github.com/notifications/unsubscribe-auth/AA9X6NPRQLpcdJf6dr7PP7kmh-XucfK7ks5srNV9gaJpZM4P0Qs8 .

-- Elliotte Rusty Harold elharo@macfaq.com

[briandealwis]

However, the Jetty example above is already pretty simple, and it's more capable as it allows users to use servlet APIs.

If they wanted the servlet API, wouldn't they be better off using the WAR runtime?

The difficulty with Jetty is that we need multiple jars, and we'll have to build the uber-jar. Otherwise I'd have suggested SparkJava as it's beautifully simple:

import static spark.Spark.*;

public class HelloWorld {
    public static void main(String[] args) {
        get("/hello", (req, res) -> "Hello World");
    }
}

But SparkJava is based on Jetty and requires at least 5 other artifacts, and there is no jar-with-dependencies option up on Maven Central.

[chanseokoh]

If they wanted the servlet API, wouldn't they be better off using the WAR runtime?

I don't really agree. The difference of WAR and JAR is whether you need a servlet engine runtime with right versions and configurations or just a JRE. With a JAR, you have maximum portability (a similar analogy to having a Docker image). Also, people may not like our Java Jetty runtime.

However, now knowing that HttpServer does not require an external dependency, I really like it. We can just punt out to users the future work of figuring out how to pack a fat JAR, for both Maven and native scenarios. We could just comment like "if you add new dependencies or JARs, be sure to create a fat JAR that includes all the dependencies in it".

[patflynn]

Does the https://marketplace.eclipse.org/content/spring-tools-aka-spring-ide-and-spring-tool-suite offer any support for native Eclipse projects?

On Wed, Oct 11, 2017 at 12:18 PM, Chanseok Oh notifications@github.com wrote:

If they wanted the servlet API, wouldn't they be better off using the WAR runtime?

I don't really agree. The difference of WAR and JAR is that whether you need a servlet engine runtime with right versions and configurations or just a JRE. With a JAR, you have maximum portability (a similar analogy to having a Docker image). Also, people may not like our Java Jetty runtime.

However, now knowing that HttpServer does not require an external dependency, I will like it. We can just punt out the future work of figuring out how to pack a fat JAR, for both Maven and native scenarios. We could just comment like "if you add new dependencies or JARs, be sure to create a fat JAR that includes all the dependencies in it".

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/2470#issuecomment-335864989, or mute the thread https://github.com/notifications/unsubscribe-auth/AHf5HfA9gJFgO04FZrXkwIRhyZAF65u3ks5srOppgaJpZM4P0Qs8 .

[chanseokoh]

The plugin forces to use either Maven or Gradle for a start project. I'm skeptical that it will have support for native Java projects.

[chanseokoh]

Regarding com.sun.net.httpserver.HttpServer, it shows an error when I select "JavaSE-1.8 (jdk8-google-v7-64)" as a JRE system library.

What is so strange is that, if I select the workspace default JRE ("jdk8-google-v7-64", which is of course the same JRE configured for "JavaSE-1.8"), the error goes away.

This SO answer says

More specifically, com.sun.net.httpserver.HttpServer is a class which is not guaranteed to be included in all Java 8 runtime implementations.

We might not have any problem using it in practice, but if it can potentially show an error in Eclipse like this, we should do something to suppress it.

[elharo]

I think that error (The type is not API) is an Eclipse/OSGI thing. I.e. it could find and use it, but it's flagging it because it's in the com.sun packages. I don't think it should be doing that here, though it maybe should for other cases. com.sun is weird. Some of it's public and some of it is internal.

Which version of Eclipse are you using? Might be fixed in a more recent version.

[chanseokoh]

I will list a few build plugins (there may be more) that I tested for creating a fat JAR. I verified that all of them work after deploying.

1. onejar-maven-plugin

Add the following to build dependencies and run mvn package.

      <plugin>
        <groupId>com.jolira</groupId>
        <artifactId>onejar-maven-plugin</artifactId>
        <version>1.4.4</version>
        <executions>
          <execution>
            <goals>
              <goal>one-jar</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <filename>${project.build.finalName}.jar</filename>
        </configuration>
      </plugin>

It creates a fat JAR with a custom loader. JAR package structure:

├── com
│   └── simontuffs
│       └── onejar
    ...
│           ├── Boot.class
    ...
│           ├── JarClassLoader.class
    ...
├── doc
│   └── one-jar-license.txt
├── lib
    ...
│   ├── guava-23.0.jar
    ...
├── OneJar.class
└── src
    ...

FYI, the plugin puts one-jar-license.txt into the JAR, but looks like it is a BSD-style license.

2. spring-boot-maven-plugin

Add the following to build dependencies and run mvn package. Note that this is a build plugin. The program to build doesn't need to use any Spring technology.

      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <version>1.5.7.RELEASE</version>
        <executions>
          <execution>
            <goals>
              <goal>repackage</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

Similar to onejar-maven-plugin, it injects a custom launcher.

├── BOOT-INF
│   ├── classes
    ...
│   └── lib
    ...
│       ├── guava-23.0.jar
    ...
└── org
    └── springframework
        └── boot
            └── loader
     ...
                ├── JarLauncher.class
     ...
                ├── Launcher.class

Probably there won't be any license issue, as this must be what people currently do a lot.

3. maven-shade-plugin

This is from org.apache.maven.plugins. Official.

Add the following to build dependencies and run mvn package.

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.1.0</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <transformers>
                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                  <mainClass>AwesomeServer</mainClass>
                </transformer>
              </transformers>
            </configuration>
          </execution>
        </executions>
      </plugin>

The above configuration just packs everything together after unpacking dependencies (guava here).

├── AwesomeServer$HellWorldHandler.class
├── com
│   └── google
│       ├── common
│       │   ├── annotations
    ...
│       │   │   └── VisibleForTesting.class

However, this build config is not scalable, as files will clash as more dependencies are added. It also causes the duplicate class loading problem if this fat JAR is loaded together with actual dependency JARs. So in practice, people relocate dependency classes in the fat JAR when shading (such as com/google/appengine/repackaged/com/google/common/base/StringUtil, although this StringUtil may have been relocated for a different reason). Relocating can be done with additional configuration, but to do so, you need to know ahead what/which classes to relocate (or not) to where, so it'd be difficult for the new project wizard to put a future-proof config.

[elharo]

Sounds good.

You could also try the maven assembly plugin: https://maven.apache.org/plugins/maven-assembly-plugin/

and we should also figure out if the Eclipse Runnable jar File exporter can be accessed from code:

http://help.eclipse.org/neon/index.jsp?topic=%2Forg.eclipse.jdt.doc.user%2Freference%2Fref-export-runnable-jar.htm&cp=1_4_9_8

On Wed, Oct 11, 2017 at 6:53 PM, Chanseok Oh notifications@github.com wrote:

I will list a few build plugins (there may be more) that I tested for creating a fat JAR. I verified that all of them work after deploying.

1. onejar-maven-plugin

Add the following to build dependencies and run mvn package.

  <plugin>
    <groupId>com.jolira</groupId>
    <artifactId>onejar-maven-plugin</artifactId>
    <version>1.4.4</version>
    <executions>
      <execution>
        <goals>
          <goal>one-jar</goal>
        </goals>
      </execution>
    </executions>
  </plugin>

It creates a fat JAR with a custom loader. JAR package structure:

├── com │ └── simontuffs │ └── onejar ... │ ├── Boot.class ... │ ├── JarClassLoader.class ... ├── doc │ └── one-jar-license.txt ├── lib ... │ ├── guava-23.0.jar ... ├── OneJar.class └── src ...

However, I noticed the existence of one-jar-license.txt, and by looking into it, I think we cannot use this build plugin:

• Redistribution and use in source and binary forms, with or without
• modification, are permitted provided that the following conditions are met: ...
• Including this file inside the built One-JAR file conforms with these terms.

2. spring-boot-maven-plugin

Add the following to build dependencies and run mvn package. Note that this is a build plugin. The program to build doesn't need to use any Spring technology.

  <plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <version>1.5.7.RELEASE</version>
    <executions>
      <execution>
        <goals>
          <goal>repackage</goal>
        </goals>
      </execution>
    </executions>
  </plugin>

Similarly onejar-maven-plugin, it injects a custom launcher.

├── BOOT-INF │ ├── classes ... │ └── lib ... │ ├── guava-23.0.jar ... └── org └── springframework └── boot └── loader ... ├── JarLauncher.class ... ├── Launcher.class

Probably there won't be any license issue, as this must be what people currently do a lot.

3. maven-shade-plugin

This is from org.apache.maven.plugins. Official.

Add the following to build dependencies and run mvn package.

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>3.1.0</version>
    <executions>
      <execution>
        <phase>package</phase>
        <goals>
          <goal>shade</goal>
        </goals>
        <configuration>
          <transformers>
            <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
              <mainClass>AwesomeServer</mainClass>
            </transformer>
          </transformers>
        </configuration>
      </execution>
    </executions>
  </plugin>

The above configuration just packs everything together after unpacking dependencies (guava here).

... ├── AwesomeServer$HellWorldHandler.class ├── com │ └── google │ ├── common │ │ ├── annotations ... │ │ │ └── VisibleForTesting.class

However, this build config is not scalable, as files will clash as more dependencies are added. It also causes the duplicate class loading problem if this fat JAR is loaded together with actual dependency JARs. So in practice, people relocate dependency classes in the fat JAR when shading (such as com/google/appengine/repackaged/com/google/common/base/StringUtil, although this StringUtil may have been relocated for a different reason). Relocating can be done with additional configuration, but to do so, you need to know ahead what/which classes to relocate (or not) to where, so it'd be difficult for the new project wizard to put a future-proof config.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/2470#issuecomment-335971517, or mute the thread https://github.com/notifications/unsubscribe-auth/AA9X6DZlxYgmwFyPYWeY73lxDH02FAA1ks5srUbTgaJpZM4P0Qs8 .

-- Elliotte Rusty Harold elharo@macfaq.com

[chanseokoh]

According to the official maven-assembly-plugin doc you linked,

If your project wants to package your artifact in an uber-jar, the assembly plugin provides only basic support. For more control, use the Maven Shade Plugin.

Did some investigation, and indeed, maven-assembly-plugin doc is superseded by maven-shade-plugin when it comes to creating a runnable fat JAR. It basically does the same thing of exploding external dependency but no fine control over hiding/relocating/removing them.

[patflynn]

To summarize the latest discussion about this. The wizard will only support creating a Maven based Springboot app for the first iteration.

[elharo]

Recent discussion said we'll do Maven first. Springboot will not be in the picture. It should work but it is not required.

[patflynn]

Springboot is what we demo and it's what we have in our jar based flex examples. What would be a better starting point for a jar based flex project?

On Thu, Oct 12, 2017 at 1:05 PM, Elliotte Rusty Harold < notifications@github.com> wrote:

Recent discussion said we'll do Maven first. Springboot will not be in the picture. It should work but it is not required.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/2470#issuecomment-336201856, or mute the thread https://github.com/notifications/unsubscribe-auth/AHf5HT6x-DKPK-h5E0y5-POuggubU3c8ks5srkblgaJpZM4P0Qs8 .

[elharo]

A straightforward pure Java application that does not have unnecessary dependencies.

Not everyone uses or wants to use Springboot.

[patflynn]

You mean a non servlet based application?

@lesv @saturnism @arouzrokh any thoughts on this?

[chanseokoh]

BTW, what is the definition of a servlet-based app? Is the Java example at http://sparkjava.com/ a servlet-based or not? How about this: https://projects.spring.io/spring-boot/#quick-start?

[elharo]

It will run a simple HTTP server. It may or may not be based on servlets depending on how we implement it.

[lesv]

Pure Java will probably have an embedded servlet container of some kind.

If you are talking Flex, then an app will need to at least listen on port 8080 and respond, no matter what it is doing. All Jetty & Tomcat are is a simplification later on that.

A pure Java app would have to do the same thing.

Everyone uses either embedded Jetty or Tomcat to deal w/ that. Technically, you could just listen on 8080, do some kind of reply -- But that would actually end up being quite a complex sample.

I'm a fan of SparkJava and SpringBoot quick-start. It's unfortunate our deployment tools won't just upload the target/output. (ie. a classpath w/ individual jar's)

So, what you are looking for is a light weight Framework of some kind the popular ones I know are:

• Dropwizard
• Ninja
• Play Framework >9k *'s on GH
• SparkJava
• JODD

Article about this.

[chanseokoh]

Technically, you could just listen on 8080, do some kind of reply -- But that would actually end up being quite a complex sample.

Not really. https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/2470#issuecomment-335586879 or below, which all use only the JDK libraries:

public class SunHttpServer {

  private static class HellWorldHandler implements HttpHandler {
    @Override
    public void handle(HttpExchange t) throws IOException {
      String response = "Hello World!";
      t.sendResponseHeaders(200, response.length());
      try (OutputStream os = t.getResponseBody()) {
        os.write(response.getBytes(StandardCharsets.UTF_8));
      }
    }
  }

  public static void main(String[] args) throws IOException {
    HttpServer server = HttpServer.create(new InetSocketAddress(8080), 0);
    server.createContext("/", new HellWorldHandler());
    server.start();
  }
}

Using an embedded Jetty is simple too: https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/2470#issuecomment-335608208

[saturnism]

I really don't feel we should use HttpServer :)

For building fat JAR, use assembly plugin and specify manifest main class is the most straight forward.

We have plenty of examples in https://github.com/GoogleCloudPlatform/getting-started-java repo. I'd use similar frameworks.

I.e., Spring Boot, Spark are fine choices. If you really want something low level, use Netty (but make sure you also test w/ gRPC client libraries).

[patflynn]

@chanseokoh @elharo

Can you finalize on this issue what the example app will be based on?

hiint: It should be based on a framework included in the examples linked by @saturnism

[chanseokoh]

We have plenty of examples in getting-started-java repo. I'd use similar frameworks.

I see there is only one starter example there: helloworld-springboot. There is no other fat JAR example.

[elharo]

Ideally we'd have multiple examples: springboot, jetty, etc. However given that we don't want to invest more time in flex than we have to, we should probably go with a combination of the simplest, most generic solution.

[chanseokoh]

Yeah, adding a couple more examples would be trivial. The UI could have a combo for selecting a choice.

[patflynn]

The simplest most common solution is a Springboot app. I think that's what devrel is recommending and they are best placed to make the call.

On Tue, Oct 31, 2017 at 10:17 AM, Chanseok Oh notifications@github.com wrote:

Yeah, adding a couple more examples would be trivial. The UI could have a combo for selecting a choice.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/2470#issuecomment-340775845, or mute the thread https://github.com/notifications/unsubscribe-auth/AHf5HT2LvHhFi1PfnvErR55qjudzFdI2ks5sxyvigaJpZM4P0Qs8 .

[JoeWang1127]

close as not planned