Edge Diagnostics App v1

[juan-castrillon]

This is far from finished but will eventually close #95.

I wanted to create this pull request now, because I thought I would share the structure I gave my application, given that I see that Theo created issues to refactor his and I believe this structure eases the whole deployment process a lot (No more sh scripts or manually typing dependencies).

The client and server apps are created as subprojects in gradle, and now is configured to run as a multi-project build. This allows tasks and sources to be specified for only client or server (Apart from handling dependencies independently and automatically). A task was created for both which creates a fatJar (with dependencies included) for them

Dockerfiles are created using docker multi-stage builds, in which the first stage, executes the gradle custom task and creates the jar, and in the second stage a lightweight image is created (with just JRE running in Alpine), the jar from the last stage is copied and executed. Arguments can be passed direcly to the container (docker build) and default parameters can be specified.

Right now my application does nothing, but the structure is ready and may be a good idea to implement also in the opencv app, what do you think @alyhasansakr ?

[alyhasansakr]

@juan-castrillon That is an excellent solution since we are going to have multiple applications that may share the same dependencies. @theodorelu27 Could you implement the same for the OpenCV application?

[juan-castrillon]

Progress so far:

• Gradle subproject structure is ready and building is very easy. A third project CommonUtils was created for shared classes between the client and server applications. Dependency to this project is included in the gradle build files
• Both client and server now work communicating with the platform (Tested manually and added one automatic test). To do this they implement classes that abstract the connection and the request sending.
• Both client and server have now a concrete implementation of LoadManager, which allows to set the protocol and load type (in runtime) for both client and server. Right now, the only load implemented is TCP PING
• TCP Ping is implemented, sending a fixed number of bytes from the client and reporting round trip time. (This is still very basic, I plan to make the number of bytes variable). The server acts like an echo server, and just decodes the message and sends it back

[juan-castrillon]

Diagnostics Application V1

All commits contain the V1 of the Diagnostics Application.

Main features:

• Client and Server side application that communicate with the platform for matchmaking and heartbeat.
• Instructions are send by the platform via REST to control both sides of the application.
• 3 different types of load can be sent with the application and obtain latency:
  • TCP Ping: Sends a random n bytes message over tcp, server responds with the same.
  • TCP File: Sends a dummy file with size n bytes, server creates the file on its size and then responds
  • UDP Ping: Sends a random n bytes message over udp
• In all load types, all parameters in the communication stack can ve varied. Some of this include:
  • Size of the data to send
  • Times to perform a single load to get an average latency value
  • Interval between single load messages
  • Buffer sizes for sockets
  • TCP window size (send and receive)
• Client and Server are subprojects in gradle and dependencies can be independent from the platform. Also, a CommonUtils subproject is used for client and server as a common dependency
• Client and Server easily deployed as docker image. Both include dockerfiles and docker tasks in gradle

Additionally, an external tool was developed to perform experiments. This tool takes "Experiment Files" and turns them into scenarios for the platform to run. This allows to synchronize commands, and to use AdvantEDGE and RemoteControl modules in addition to the Diagnostics App.

Experiment files, define base parameters for client and server and also changes in either one of the possible categories (client, server, network, compute). This changes are explicitly declared specifing a range of values (from and to), a step in between and a scale. Current version supports changes in just one category and only decimal scale.

The application has room for improvement and as discussed, I will submit it to review to change whatever necessary.

Gradle Structure

As of right now, application subprojects are part of the root project from the platform like this:

Root project 'Edge-Diagnostic-Platform'
\--- Project ':EdgeDiagnosticsApp'
     +--- Project ':EdgeDiagnosticsApp:CommonUtils'
          - build.gradle
     +--- Project ':EdgeDiagnosticsApp:DiagnosticsClient'
          - build.gradle
     \--- Project ':EdgeDiagnosticsApp:DiagnosticsServer'
          - build.gradle
build.gradle
settings.gradle

This however, makes tasks in the Edge-Diagnostic-Platform execute also tasks in the application subprojects with the same name. For example calling gradle run on the Edge-Diagnostic-Platform, also executes run in client and server which is not desired. For this, the solution in the commits was to add a parameter to skip the tasks in the application (e.g dockerDClient, runDServer, etc.).

I believe this does not scale well, and if we are to continue doing the applications with this manner, we should restructure the whole project like this:

Root project 'ParentProject'
\--- Project ':Edge-Diagnostics-Platform'
  - build.gradle
\--- Project ':EdgeDiagnosticsApp'
     +--- Project ':EdgeDiagnosticsApp:CommonUtils'
          - build.gradle
     +--- Project ':EdgeDiagnosticsApp:DiagnosticsClient'
          - build.gradle
     \--- Project ':EdgeDiagnosticsApp:DiagnosticsServer'
          - build.gradle
settings.gradle

This is the recommended way by gradle documentation and would allow us to run tasks independently on platform and apps (just by running them from the folders or with the complete name), but also have some common things in the ParentProject. This will also allow to easily add new applications.

I did not make this structure change until know, because I believe is a big change, workflows would have to be modified and I wanted to check your opinion first @alyhasansakr

[alyhasansakr]

What you say is correct, the infrastructure manager is just a component in the edge diagnostics platform. We will do this refactor when both you and Theodore finish their work.

[juan-castrillon]

Latest changes include:

• Fixing a bug in the platform where a Windows path (e.g. "C:\Some\Folder") could not be passed because the command mapping would take away the \ character (Resulting in "C:SomeFolder")
• Adding an argument for the client for the path to store the latency report (results of experiment)
• Fixing input reading from console to work when deployed as K8s pod

If you think the code is fine, then I think we can merge this @alyhasansakr