Making my way through Lab01

[dmg46664]

Hi Jeff, Thanks for providing this online. I'm going to try make my way through these.

Please see some inline comments on your instructions below.

== Get Logs to One Place

With MicroServices it is a must to have a single place to go through logs for everything at once. For this workshop we will be using the ELK stack, hosted locally.

The work will be to setup logging targets of logstash for all the microservices.

⚠️ We haven't introduced logstash yet in the original guide!? (How is someone unfamiliar supposed to know this is the L in ELK?), or what its purpose is. It's okay if you have mentioned this in the talk, but it needs to be in the notes as well in case anyone isn't following.

== Dynamic Log Filtering

Now that we have all the logs in one place, we quickly start having too many logs.

⚠️ We haven't done anything yet to put all the logs in one place? Are there no further instructions?

Next know we want some debug level logging but not all of it. Using Turbo Filters in logback is a great place to start. DynamicThresholdFilter will allow the most flexibility.

Checkout the logback https://logback.qos.ch/manual/filters.html[filter docs].

== Formatting

I don't think it's wise to have guidance in both a pdf and task file. Ideally there should be one source of truth to act as a guide and I would make it the task file (otherwise a reader's concentration has to be split).

I think that each lab should fully introduce and contain all the information required. I.e. the following:

Lab 01 rewrite

docker-compose up starts up various programs defined in infra/docker-compose.yml and the subdirectories below it.

Next open links.html in the browser. The items under Infra links provides links to the docker containers if they have come online properly.

Kibana & Logstash & Elastic search make up the ELK stack.

• Logstash collects and transforms the logs.
• Elastic search stores the transformation.
• Kibana is a web interface to search these logs. See https://www.elastic.co/elk-stack for more information.

Run the first application

edge-ratpack> gradle run
...
... logs should show up
...
<===========--> 90% EXECUTING [1m 5s]
> :lab-01:edge-ratpack:run

But where have the logs been saved to in your application?

Get Logs to One Place

With MicroServices it is a must to have a single place to go through logs for everything at once. For this workshop we will be using the ELK stack, hosted locally.

The work will be to setup logging targets of logstash for all the microservices.

You need to do this now...

Let's start with the first application edge-ratpack.

Hint: Add the following to the logback.xml at the appropriate place.

      <appender-ref ref="stash"/> 

Now relaunch the application

⚠️ Note that you cannot expect users to get familiar with your application structure. Usually Java devs are writing code, and some may not have had to change an appender in years.

⚠️ Further up in the stack I see the following. Ideally there shouldn't be any errors not mentioned in the tutorial.

time=23:12:01.577, loggingId=, level=DEBUG, logger=r.util.internal.TransportDetector, message=io.netty.channel.epoll.Epoll unavailability cause
java.lang.ExceptionInInitializerError: null
        at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:38)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at ratpack.util.internal.TransportDetector$NativeTransport.invokeIsAvailable(TransportDetector.java:180)
        at ratpack.util.internal.TransportDetector$NativeTransport.isAvailable(TransportDetector.java:152)
        at ratpack.util.internal.TransportDetector$NativeTransport.<init>(TransportDetector.java:109)
        at ratpack.util.internal.TransportDetector.determineTransport(TransportDetector.java:40)
        at ratpack.util.internal.TransportDetector.<clinit>(TransportDetector.java:37)
        at ratpack.exec.internal.DefaultExecController.<init>(DefaultExecController.java:61)
        at ratpack.server.internal.DefaultRatpackServer.start(DefaultRatpackServer.java:126)
        at ratpack.server.RatpackServer.start(RatpackServer.java:93)
        at ratpack.groovy.GroovyRatpackMain.main(GroovyRatpackMain.java:38)
Caused by: java.lang.IllegalStateException: Only supported on Linux
        at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:192)
        at io.netty.channel.epoll.Native.<clinit>(Native.java:61)
        ... 14 common frames omitted
time=23:12:01.578, loggingId=, level=DEBUG, logger=r.util.internal.TransportDetector, message=io.netty.channel.kqueue was not found

I cannot see the logs in Kibana. ⚠️ There is no link to logstash in links.html, so I assume that this all just feeds into Kibana by default or via your configuration (this should be mentioned explicitly in the guide), and I've assumed the error above is the reason why it's not working. Does this mean familiarity with a groovy project is required to sort this out? Your guide should ideally mention ALL changes for users to make and why in order for everyone to get the most out of it. If there are multiple applications, then the guide should deal with getting one working at a time. I can see the diff between the answers includes groovy files.

[dmg46664]

I guess the stacktrace isn't due to an error given it's a DEBUG log level, just unusual to see stacktraces for non-errors.

[beckje01]

Thanks for the feedback, the main changes needed are just changing configuration in logback.xml but I didn't call that out explicitly. But it's clear another section that is missing is the theory of what all the log aggregation bits are.

That error is just trying to use epoll with netty which can't work on non linux, which isn't something I run into so I forgot to include a note about it thanks for calling that out.

I like the level of detail you are providing I have worried its too much in the past but given the experiences I think it worth adding more of this.

[dmg46664]

No worries :-) This course is useful so I'm happy to go through it, and it should help others. :-)

I don't have all that much time, so it will be done in drips and drabs. I thought I'd post it here and you can take review my comments rather than me submitting pull requests which might not have associated reasoning.

[beckje01]

I wanted to let you know I incorporated this feedback into the new version of this talk thanks for the time.

[dmg46664]

Great! Unfortunately my mac is in for repairs and so I have been unable to continue for the moment.