search

Together-Java / TJ-Bot

TJ-Bot is a Discord Bot used on the Together Java server. It is maintained by the community, anyone can contribute.
https://togetherjava.org
GNU General Public License v3.0
101 stars 90 forks source link

JShell in AWS #1158

Open surajkumar opened 2 months ago

surajkumar commented 2 months ago

Context

The kind developers have created a JShell solution for our server however it's been around for a few months and has been entirely unstable since it's launch. There have been problems with it's stability, code quality and high barrier to entry for new contributors. There have been talks about using AWS Lambda to host and run the entire JShell backend and this issue is to explore that as an alternative solution to JShell or at the least as a "fall-over" for JShell.

Why is this important?

JShell is difficult to maintain. We have been adding additional logging, debugging on the VPS and going in circles trying to resolve issues with the project. All of that can be avoided by allowing an external, free service to do the heavy lifting for us. The user experience has been severely affected, not many people are able to use JShell right now as there are bugs that haven't been resolved for months such as the containers not cleaning up. While we have attempted to fix some of these issues, there is no certainty that the fixes will work. Nobody to date has been able to reproduce any of the problems locally.

It's now important for us to evaluate the JShell implementation and be mature on how we tackle it forward.

I am proposing one of the following 2 options:

  1. Continue to maintain the existing JShell service but use AWS Lambda as a fall-over for when our JShell service goes down
  2. Entirely, replace the solution with the lightweight AWS solution

Benefits of AWS

Cons of the AWS solution

No solution is perfect and there are some cons to the solution.

Sessions: We will no longer have sessions as we currently do. I however, do not see this a problem as since release, nobody has used them. If we want to retain this functionality, we can send the previous commands as part of the JShell request however, this is pretty error prone.

Startup Script: We probably shouldn't support the use of a startup script. It takes away from the vanilla Java experience by providing shorthand methods like println however, we can support it.

Latency: Serverless and Java come with something called cold starts. This means a typical API request from a sleeping server can take around 6-10 seconds. This however can be less of an issue if we used GraalVM and compile native (drops the latency to 400ms) or we can explore SnapStart which is a solution that AWS provides for Java Lambdas. Snapstart is the easiest to get started with, GraalVM requires some knowledge on setup (in both Gradle and AWS). Though, everything is very well documented.

Development Cost

The development overhead for such a system is not as big as you'd imagine. Here's a gist that we can literally lift and shift for the AWS Lambda: https://gist.github.com/surajkumar/ada17480ade44f229bbb1788f2c93a18 - it will respond in the same way that the current JShell Backend produces.

Our development cost will be creating the slash command. AFAIK the current JShell slash command is fairly complex and it might not just be a case of switching the URL. This is primarily beause we need to provide a request body to the AWS API { "code": "System.out.println(\"hello, world\");" } (real sample). And remove the unsupported sessions.

This issue is open for discussions.

surajkumar commented 2 months ago

You can play with this sample API for testing:

cURL:

curl -X POST "https://33ykaxnvin5lusynogiwrevqgm0ovara.lambda-url.eu-west-2.on.aws/" \
     -H "Content-Type: application/json" \
     -d '{"code": "System.out.println(\"hello, world\");"}'
surajkumar commented 2 months ago

Implementation start: https://github.com/Together-Java/TJ-Bot/compare/develop...surajkumar:TJ-Bot:1158-aws-jshell

Missing the slash command impl.