Current Scope - Githubissues

frank-dspeed commented 4 years ago

@frank-dspeed we do have quite some list, but i must be honest, i can't test them all, all the time: https://github.com/vert-x3/vertx-auth/tree/master/vertx-auth-oauth2/src/main/java/io/vertx/ext/auth/oauth2/providers the other issue is that the documentation tries to be generic so it tells you about the underlying details on how to set up everything manually. I think we need some work to refactor the documentation to use these utilities first and fallback to the low level if some is not provided (and ask users to contribute new ones of course)

Examples oAuth

on vert.x 3.x:

    // In order to use a template we first need to create an engine
    final HandlebarsTemplateEngine engine = HandlebarsTemplateEngine.create(vertx);

    // To simplify the development of the web components we use a Router to route
    // all HTTP requests
    // to organize our code in a reusable way.
    final Router router = Router.router(vertx);
    // We need cookies and sessions
    router.route().handler(CookieHandler.create());
    router.route().handler(SessionHandler.create(LocalSessionStore.create(vertx)));
    // Simple auth service which uses a GitHub to authenticate the user
    OAuth2Auth authProvider = GoogleAuth.create(vertx, CLIENT_ID, CLIENT_SECRET);
    // We need a user session handler too to make sure the user is stored in the session between requests
    router.route().handler(UserSessionHandler.create(authProvider));
    // we now protect the resource under the path "/protected"
    router.route("/protected").handler(
      OAuth2AuthHandler.create(authProvider, "http://localhost:8080/callback")
        // we now configure the oauth2 handler, it will setup the callback handler
        // as expected by your oauth2 provider.
        .setupCallback(router.route("/callback"))
        // for this resource we require that users have the authority to retrieve the user emails
        .addAuthority("email")
    );
    // Entry point to the application, this will render a custom template.
    router.get("/").handler(ctx -> {
      // we pass the client id to the template
      JsonObject data = new JsonObject()
        .put("client_id", CLIENT_ID);
      // and now delegate to the engine to render it.
      engine.render(data, "views/index.hbs", res -> {
        if (res.succeeded()) {
          ctx.response()
            .putHeader("Content-Type", "text/html")
            .end(res.result());
        } else {
          ctx.fail(res.cause());
          res.cause().printStackTrace();
        }
      });
    });
    // The protected resource
    router.get("/protected").handler(ctx -> {
      System.out.println("HERE!!!");
      User user = ctx.user();
      System.out.println(user);
      ctx.response().end();
    });

    vertx.createHttpServer().requestHandler(router).listen(8080);

On vert.x 4.0.0-SNAPSHOT:

    // In order to use a template we first need to create an engine
    final HandlebarsTemplateEngine engine = HandlebarsTemplateEngine.create(vertx);

    // To simplify the development of the web components we use a Router to route
    // all HTTP requests
    // to organize our code in a reusable way.
    final Router router = Router.router(vertx);
    // We need cookies and sessions
    router.route().handler(SessionHandler.create(LocalSessionStore.create(vertx)));
    // Simple auth service which uses a GitHub to authenticate the user
    OAuth2Auth authProvider = GoogleAuth.create(vertx, CLIENT_ID, CLIENT_SECRET);
    // we now protect the resource under the path "/protected"
    router.route("/protected").handler(
      OAuth2AuthHandler.create(vertx, authProvider, "http://localhost:8080/callback")
        // we now configure the oauth2 handler, it will setup the callback handler
        // as expected by your oauth2 provider.
        .setupCallback(router.route("/callback"))
        // for this resource we require that users have the authority to retrieve the user emails
        .extraParams(new JsonObject()
          .put("scope", "email")));

    // Entry point to the application, this will render a custom template.
    router.get("/").handler(ctx -> {
      // we pass the client id to the template
      JsonObject data = new JsonObject()
        .put("client_id", CLIENT_ID);
      // and now delegate to the engine to render it.
      engine.render(data, "views/index.hbs", res -> {
        if (res.succeeded()) {
          ctx.response()
            .putHeader("Content-Type", "text/html")
            .end(res.result());
        } else {
          ctx.fail(res.cause());
          res.cause().printStackTrace();
        }
      });
    });
    // The protected resource
    router.get("/protected").handler(ctx -> {
      System.out.println("HERE!!!");
      User user = ctx.user();
      System.out.println(user);
      ctx.response().end();
    });

    vertx.createHttpServer().requestHandler(router).listen(8080);

frank-dspeed commented 4 years ago

kotlin with vertex

Kotlin with Vertx.
  val router: Router = Router.router(vertx).apply {
    getAsync("/test", ::test)
    postAsync("/register", ::registerUser)
  }

  private suspend fun test(ctx: RoutingContext) {
    ctx.response().endAwait("TEST OK")
  }

  private fun Router.getAsync(path: String, fn: suspend (RoutingContext) -> Unit): Route {
    return get(path).handlerAsync(fn)
  }

  private fun Router.postAsync(path: String, fn: suspend (RoutingContext) -> Unit): Route {
    return post(path).handlerAsync(fn)
  }

  private fun Route.handlerAsync(fn: suspend (RoutingContext) -> Unit): Route {
    handler { ctx ->
      launch(ctx.vertx().dispatcher()) {
        try {
          fn(ctx)
        } catch (e: Exception) {
          ctx.fail(e)
        }
      }
    }
    return this
  }

frank-dspeed commented 4 years ago

Logging Infra the Enterprisebus Pattern

Frank Lemanschik @frank-dspeed März 09 12:33 the question here is how much traffic do you handle? do you plan to run many instances? or just a single server? i ask that because logging can slow down your application if it is done wrong :) remember what logging means do something on every request Vlad @vladostaci März 09 12:35 No, I don't. I just want to build a simple Verticle with websockets and a few rest endpoints for regsistration, resetting password, etc. I'd like to log to the console on my local PC and log to some web service in production. Frank Lemanschik @frank-dspeed März 09 12:36 do you expect much traffic? about how many concurrent users are we talking? i my self operate at scale i have my app architecture based on that logging needs i write every reqest into redis there my workers get the request and write the response to redis so i can simply use redis for realtime request and also processing logging i know exactly how long processing took on my side and i can even messure client results via google analaytics i do not log to console as that would cause to much io that is not needed logging to file also needs much disk io i prefer to use that for other stuff :) so when you need something that scales this is a nice design you can replace redis by kafka or anything that you like and that fits your needs in case of write read speed Frank Lemanschik @frank-dspeed März 09 12:41 so 1 vertical tracks the connection and holds it and waits for the response event from the worker verticel that is processing that the nice thing with this design is you can scale connections independent from processing Vlad @vladostaci März 09 12:46 It shouldn't be more than 1K concurrent at launch. I will optimize it later as needed. Your design makes sense. So your solution is to use Redis as logging. When you log a http request to Redis for example, are you using some custom logging (timestamp method, headers, body)? Ideally I would use some available Open Source solution which cab be used in production, and has a free tier. Frank Lemanschik @frank-dspeed März 09 12:47 i log all headers and body + connection id the connection id is needed so that the client can respond with the result its the reference to the open connection it listens then for pub sub with that connectionid Vlad @vladostaci März 09 12:48 @frank-dspeed How do you view your logs and errors later? Also the performance and resource utilization, custom as well? Frank Lemanschik @frank-dspeed März 09 12:49 i get the performance data out of the time stemps and the diffrence between req response and yes its all custom i track other metrics diffrent not in redis redis is only my append only reqest response log stream and its also a response cache so i do not need to process something that is already done for example i have some websites with some million pages i generate them on demand if they get requested if they are not in the cache else i return them directly from the cache or when there is a template update i invalidate the cache i save the request response data inside a redis cluster that growth endless at present and i can query that i have no problem with sharding as i can shard per date :) Frank Lemanschik @frank-dspeed März 09 12:55 So Conclusion the key for a good distributed system with logging is to use a central eventbus :) thats how i would call such a structure :) and the events in the eventbus are the logs when a worker node has a failure this goes also in the log if its a processing error but when the hardware of a worker node has a failure then you do not care for that as workers should be designed to pull work so a dead worker will not pull work :) no need to handle that inside of the application and if a worker dies while he is processing work you will reach timeouts and your retry logic will handle that Vlad @vladostaci März 09 13:09 @frank-dspeed Sounds like you had everything covered. I'll probably go with a simpler sulution then just to save time, https://cloud.google.com/logging/ looks good. I'll also try to integrate https://cloud.google.com/debugger because I'm already using Cloud Compute. I will also probably only log the rest requests initially.

thats how i would call such a structure :) and the events in the eventbus are the logs when a worker node has a failure this goes also in the log if its a processing error

My current setup is more tranditional, without using the EventBus, do you know any good articles which show something similar to your architecture ?

Frank Lemanschik @frank-dspeed März 09 13:10 At present, I am preparing such articles there to exist some explanations about such patterns but no good examples But maybe in 1-2 Years we will have good documentation and books about that patterns :) the problem is my implementation is at present written in node js i am new to java in general i know it and understand it but i never liked to use it and i am here because of this graal project and the vertx ecosystem that offers js integrations for that i want to improve the vertx ecosystem with some design concepts out of the nodejs world so that i could then use vertx + graaljs for my current node application and in nodejs this is really easy to archive :) you code a entry server then you use this https://github.com/bee-queue/bee-queue and your near done :D bee que has the pub sub messaging job managment web interface all this stuff ;) i am still testing if that will work out of the box with graal and vertx web but who knows :) if it does not then i will need to code something similar in java for vertx redis

frank-dspeed commented 4 years ago

https://quarkus.io/ is using vertx maybe we can directly integrate that also

frank-dspeed commented 4 years ago

Executor async pattern

You could use an Executor:

ExecutorService executor = Executors.newCachedThreadPool(); Callable

stealify / graaljs-bindings-vertx

Current Scope #1

Examples oAuth

kotlin with vertex

Logging Infra the Enterprisebus Pattern

Executor async pattern

Most Importent