Docker based webservice converting source to .elf native scripts

[fire]

We want low latency editor editing of CppScript and RustScript conversion to .elf.

Optional: Provide script source so it can be verified for correctness.

Two tech stacks come to mind:

Web stack

1. postgresql database
2. filesystem
3. server language like elixir or golang
4. Create a web service.

Local stack

1. generate container2wasm for all our platforms for the rust and cpp docker images

[fire]

Discussed with @fwsGonzo to try prototyping:

linux container with elixir, postgres, filesystem mount and REST web api

[fire]

Develop a webservice using Elixir, including a mounted volume and PostgreSQL database, ensuring compatibility with Linux OS. The service must compile RustScript and C++Script into .elf format. Consider using Docker for deployment flexibility and potential reuse of existing containers. Address performance concerns due to latency issues from Brisbane, Australia to Accra, Ghana, which may require efficient data handling and possibly regional server deployment. API functionality requirements outlined by fwsgonzo include: -o output for building programs, --version for API versioning, and line printing functionalities. Input structure will be a nested directory of source files, potentially received as a zip file, which will be extracted by the web service.

[fire]

Brew install:

brew install riscv64-elf-binutils riscv64-elf-gcc

[fire]

Development Steps

1. Start
2. Set Up Your Elixir Project
   • Create a new project
   • Add dependencies in mix.exs
3. Configure the Phoenix Application
   • Set up routing
   • Configure database settings
4. Implement API Endpoints
   • Create controllers
   • Compile RustScript & C++Script
   • Handle command-line options
5. Handle File Inputs
   • Accept zip file
   • Extract files for processing
6. Setup Compilation Logic
   • Integrate compilation logic
   • Ensure .elf format output
7. Dockerize the Application
   • Create Dockerfile
   • Install dependencies
   • Configure container to start server
8. Optimize Performance
   • Consider latency issues
   • Implement data handling practices
   • Optionally set up regional servers
9. Testing and Deployment
   • Test in dev environment
   • Deploy to production
   • Monitor performance and reliability
10. End

GDScript Archiving Process

1. Start
2. Prepare Your Godot Environment
   • Open/Create Godot project
   • Write GDScript
3. Create a GDScript for Archiving
   • Write script to traverse directories
   • Add files to zip archive
4. Integrate the Script in Godot
   • Attach script to node
   • Execute script via user interaction or automatically
5. Test the Archiving Functionality
   • Run script in editor
   • Check output zip file
6. Deployment
   • Receive .elf
7. End

[fire]

Did an initial design.

godot-sandbox.txt

[fire]

Design change

Instead of a separate website and a separate docker system unify both into one docker that hosts a webservice.

[fire]

https://hexdocs.pm/alcove/readme.html can use chroot or cgroups to sandbox os process executes

[fire]

• rust is almost instant
• cpp 800ms, which is tolerable++ but the C++ API is pretty big now, so 800ms is quite OK
• Zig doesnt have an API and it's 2000ms

[fire]

We should cache for all languages.

I can punt by using ccache and there's also the cache by mozilla https://github.com/mozilla/sccache

[fire]

Zig has a build option ReleaseSmall that optimized, but produces small executables

now it only takes 900ms

[fire]

• Cpp docker https://github.com/libriscv/godot-sandbox/blob/main/program/cpp/Dockerfile
• Rust docker https://github.com/libriscv/godot-sandbox/blob/main/program/rust/Dockerfile

[fire]

Need to use ubuntu:24.04

[fire]

Closed as https://github.com/libriscv/godot-sandbox/pull/151/files is more active.