Check-in code changes for camel-k implementation.

[mkelly1495]

Implemented a decoder that parses PC2 messages.

closes #4

[durandom]

@mkelly1495 nice job. is this WIP or ready for review?

[durandom]

also, the code should be buildable via a camel-k integration, as outlined at https://github.com/b4mad/racing/tree/master/camel-k#install-integration

kamel run --name ingress Ingress.java Decoder.java --dev --pod-template template.yaml

[durandom]

see https://camel.apache.org/camel-k/1.7.x/installation/installation.html for development environment. I suggest Red Hat CodeReady Containers (CRC), since the prod version runs on OpenShift

[mkelly1495]

Hello,

I just wanted to get some feedback to make sure I was on the right track. The instructions are a little unclear with regard to the integration, but it looks like the new comments have the info I need. I need to tweak my environment to be able to test everything (I’m developing on windows). I’ll address the comments and push up the corrections.

Michael Kelly

On Nov 28, 2021, at 2:37 PM, Marcel Hild @.***> wrote:

 @mkelly1495 nice job. is this WIP or ready for review?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[mkelly1495]

Also - this is my first attempt at a gitcoin bounty, so I apologize if I didn’t follow the normal process. Thanks!

Michael Kelly

On Nov 28, 2021, at 3:26 PM, Mike Kelly @.***> wrote:

Hello,

I just wanted to get some feedback to make sure I was on the right track. The instructions are a little unclear with regard to the integration, but it looks like the new comments have the info I need. I need to tweak my environment to be able to test everything (I’m developing on windows). I’ll address the comments and push up the corrections.

Michael Kelly

On Nov 28, 2021, at 2:37 PM, Marcel Hild @.***> wrote:

 @mkelly1495 nice job. is this WIP or ready for review?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[goern]

@mkelly1495 let us know how you test and whats missing, maybe we can provide you with some more infra to test..

[goern]

Implemented a decoder that parses PC2 messages.

real nice and fast! +1

[mkelly1495]

So apparently CRC doesn't run on windows home edition LOL. I'm trying some of the other options to see if I can get something running locally.

[mkelly1495]

Alright - I now have the Decoder running on minikube. The dependency resolution for kamel is not very good. I had to put pretty much all of the classes in the same file for the class dependencies to resolve. The other option is to build the additional classes as a separate dependency. Anyway, the kamel route deploys, but sending messages via nc doesn't seem to do anything. I don't know if I'm just not hitting the right IP address or what. There doesn't seem to be any indication that messages are being received. I even tried you're stub code and got the same result. How do I determine the appropriate IP address to send the udp message to?

[mkelly1495]

I went ahead and pushed up my current changes. Perhaps it will work for you. The route doesn't appear to be receiving any messages no matter what I try. ugh.

[durandom]

@mkelly1495 thanks. I'll try your current code and provide some feedback.

The networking bits are tough. Do you have minikube deployed in a VM? Feel free to jump onto discord and ask questions https://discord.gg/N8ENrbES

[durandom]

when deploying the integration and sending data to it I get

[1] 2021-11-29 13:27:27,479 INFO  [io.quarkus] (main) Profile prod activated.
[1] 2021-11-29 13:27:27,479 INFO  [io.quarkus] (main) Installed features: [camel-bean, camel-core, camel-direct, camel-java-joor-dsl, camel-k-core, camel-k-runtime, camel-netty, cdi]
[1] 2021-11-29 13:28:41,564 WARN  [org.apa.cam.com.net.NettyConsumer] (Camel (camel-1) thread #1 - NettyConsumerExecutorGroup) Closing channel as an exception was thrown from Netty. Caused by: [io.netty.handler.codec.DecoderException - java.lang.UnsupportedOperationException: direct buffer]: io.netty.handler.codec.DecoderException: java.lang.UnsupportedOperationException: direct buffer
[1]     at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:98)
[1]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[1]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[1]     at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
[1]     at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
[1]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
[1]     at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
[1]     at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
[1]     at io.netty.channel.nio.AbstractNioMessageChannel$NioMessageUnsafe.read(AbstractNioMessageChannel.java:97)
[1]     at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
[1]     at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
[1]     at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
[1]     at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
[1]     at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
[1]     at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
[1]     at java.base/java.lang.Thread.run(Thread.java:829)
[1] Caused by: java.lang.UnsupportedOperationException: direct buffer
[1]     at io.netty.buffer.PooledUnsafeDirectByteBuf.array(PooledUnsafeDirectByteBuf.java:226)
[1]     at Ingress$PC2PacketParser.parse(Ingress.java:392)
[1]     at Ingress$Decoder.decode(Ingress.java:50)
[1]     at Ingress$Decoder.decode(Ingress.java:38)
[1]     at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
[1]     ... 15 more
[1]

[durandom]

First you'll need the external IP of your cluster: https://stackoverflow.com/questions/59998627/how-to-find-out-the-ip-address-of-the-nodeport In my case it's 128.52.60.32 Now I can ping 128.52.60.32 Then I run tcpdump -i en7 -n icmp -e which shows me the relevant info for the tcprewrite cmds.

❯ tcpdump -i en7 -n icmp -e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en7, link-type EN10MB (Ethernet), capture size 262144 bytes
14:35:24.754105 00:50:b6:a1:77:87 > b4:fb:e4:24:0f:4d, ethertype IPv4 (0x0800), length 98: 192.168.1.111 > 128.52.60.32: ICMP echo request, id 58157, seq 55, length 64

Use those values for https://github.com/b4mad/racing/tree/master/camel-k#replaying-the-udp-stream and you should be able to replay the udp stream

[mkelly1495]

Thanks so much guys! I'll dive a little deeper on this a bit later today (gotta go work by "real" job for a bit LOL).

[mkelly1495]

Just FYI - I'm using minikibe and docker to test everything.

[mkelly1495]

Thanks for the reminder about discord. I'm such a noob.

[mkelly1495]

First you'll need the external IP of your cluster: https://stackoverflow.com/questions/59998627/how-to-find-out-the-ip-address-of-the-nodeport In my case it's 128.52.60.32 Now I can ping 128.52.60.32 Then I run tcpdump -i en7 -n icmp -e which shows me the relevant info for the tcprewrite cmds.

❯ tcpdump -i en7 -n icmp -e
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en7, link-type EN10MB (Ethernet), capture size 262144 bytes
14:35:24.754105 00:50:b6:a1:77:87 > b4:fb:e4:24:0f:4d, ethertype IPv4 (0x0800), length 98: 192.168.1.111 > 128.52.60.32: ICMP echo request, id 58157, seq 55, length 64

Use those values for https://github.com/b4mad/racing/tree/master/camel-k#replaying-the-udp-stream and you should be able to replay the udp stream

This did not work for me. There is no external IP provided for the service. I've been banging my head against the wall for the last couple of hours trying to access the service through the telemetry-ingress service and none of the suggestions online worked. I finally was able to send messages to the kamel-k listener on 10666 by attaching directly to the pod kubectl exec --stdin --tty ingress-57f8cc7b5c-kvrrs -- /bin/bash. I'm not sure if it's an issue with minikube or something else, but it appears to be impossible to expose a service with an external IP with minikube. Anyway, I'm good now that I can at least send messages to the kamel-k pod.

[durandom]

@mkelly1495 sorry about the networking issues - my experience with minikube is limited :/ glad you're able to send messages now.

I've uploaded telemetry streams recorded with Project Cars 2, in Project Cars and Project Cars v2 format. https://github.com/b4mad/racing/tree/master/camel-k

From my inspection with Wireshark it's matching the protocol as defined in https://www.projectcarsgame.com/two/project-cars-2-api/#1526544680534-1e10fcf7-b72a

[durandom]

@mkelly1495 looks like it works :)

[1] 2021-12-01 06:56:42,497 INFO  [io.quarkus] (main) Installed features: [camel-bean, camel-core, camel-java-joor-dsl, camel-k-core, camel-k-runtime, camel-netty, cdi]
[1] 2021-12-01 07:04:24,101 INFO  [udp-dispatcher] (Camel (camel-1) thread #1 - NettyConsumerExecutorGroup) Raw Packet: BasePacket{packetNumber=5545, categoryPacketNumber=-38, partialPacketIndex=1, partialPacketNumber=1, packetType=GameState, packetVersion=2}

[durandom]

I don't like that all of the classes are in one file PC2Decoder, but the dependency tracing in kamel makes it hard to do otherwise. I think a better solution would be to use an additional dependency when deploying the decoder. Something like kamel run -d mvn:pc2-decoder Ingress.java ... where the pc2-decoder is a module within the repo that contains all of the class required for decoding. This would allow for the addition of unit tests as well as actual compiling with an IDE. I have a version of the repo that uses this approach if you are interested.

@mkelly1495 could you that solution in this PR - that would be super awesome? Just alongside the current solution, which works :) Once that's in, we can merge and approve the gitcoin thing

[sesheta]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: durandom

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/b4mad/racing/blob/master/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[mkelly1495]

I didn’t create a settings.xml since I’m a little weak with the kubernetes stuff. However, based on the docs here: https://camel.apache.org/camel-k/1.7.x/configuration/dependencies.html, it looks like you can either register your own nexus repository with your openshift/kubernetes instance or upload the artifact or to maven central. Sadly, it doesn’t yet support use of artifacts in the local maven cache.

Michael Kelly

On Dec 2, 2021, at 6:26 AM, Marcel Hild @.***> wrote:

 @durandom approved this pull request.

LGTM.

Thanks. Good Work. And bonus points for the unit tests 😻

In camel-k/README.md:

+Note: This folder contains two versions of the PC2 packet decoder. +1. PC2DecoderStandAlone.java contains all of the classes for decoding in a single file. In order to run this file, just do kamel run --name ingress PC2DecoderStandAlone.java --dev --pod-template template.yaml +2. Ingress.java provides a much more maintainable version of the PC2 decoder in that it utilizes the pc2-decoder module. The Ingress.java file simply defines the route and delegates decoding to the PC2PacketParser defined in the pcs-decoder module. In order to deploy this version of the decoder, you need to include the pc2-decoder module as a dependency for kamel. Unfortunately, at the moment, camel-k doesn't work with local jars. Therefore, you either have to push the pcs-converter jar to maven central or configure the maven settings according to https://camel.apache.org/camel-k/1.7.x/configuration/maven.html. Once the configuration is completed, run kamel run -d mvn:io.ppatierno:f1-telemetry-pc2-decoder:1.0 --name ingress Ingress.java --dev --pod-template template.yaml @mkelly1495 do you happen to have a settings.xml or kamel install --maven-settings cmd that would configure the maven settings correctly to include the dependency? I guess I'd have to upload the dependencies to a cluster local repository?! Or can I still have them locally?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.