search

lmos-ai / arc

The goal of the Arc project is to utilize the power of Kotlin DSL and Kotlin Scripting to define a language optimized for building LLM powered solutions.
https://www.telekom.com/imprint
64 stars 3 forks source link

Pre Compiled Highly Optimised Arc Agents For Production Builds #41

Open jas34 opened 4 days ago

jas34 commented 4 days ago

Issue: Currently, arc agents can be compiled only at runtime. This way of managing arc agents leads to the following problems:

  1. Production-like environment need to run on JDK rather than JRE. Secure systems like banking and trading can't have JDK on production.
  2. A huge memory spike is seen during the agent's compile time (at the start-up or hot reload).
  3. The runtime-compiled agent is not completely optimized and the overall memory footprint of the service increases.

Target: Arc agents should be precompiled for a production-like environment.