search

konveyor-ecosystem / kai

Konveyor AI - static code analysis driven migration to new targets via Generative AI
Apache License 2.0
8 stars 10 forks source link

readme

Konveyor AI (kai)

Konveyor AI (kai) is Konveyor's approach to easing modernization of application source code to a new target by leveraging LLMs with guidance from static code analysis augmented with data in Konveyor that helps to learn how an Organization solved a similar problem in the past.

Pronunciation of 'kai': https://www.howtopronounce.com/kai

Blog Posts

Approach

Our approach is to use static code analysis to find the areas in source code that need to be transformed. 'kai' will iterate through analysis information and work with LLMs to generate code changes to resolve incidents identified from analysis.

This approach does not require fine-tuning of LLMs, we augment a LLMs knowledge via the prompt, similar to approaches with RAG by leveraging external data from inside of Konveyor and from Analysis Rules to aid the LLM in constructing better results.

For example, analyzer-lsp Rules such as these (Java EE to Quarkus rulesets) are leveraged to aid guiding a LLM to update a legacy Java EE application to Quarkus

Note: For purposes of this initial prototype we are using an example of Java EE to Quarkus. That is an arbitrary choice to show viability of this approach. The code and the approach will work on other targets that Konveyor has rules for.

What happens technically to make this work?

Pre-Requisites

Access to a Large Language Model (LLM)

LLM API Keys

IBM BAM Service

OpenAI Service

Selecting a Model

We offer configuration choices of several models via config.toml which line up to choices we know about from kai/model_provider.py.

To change which llm you are targeting, open config.toml and change the [models] section to one of the following:

IBM served granite

[models]
  provider = "ChatIBMGenAI"

  [models.args]
  model_id = "ibm/granite-13b-chat-v2"

IBM served mistral

[models]
  provider = "ChatIBMGenAI"

  [models.args]
  model_id = "mistralai/mixtral-8x7b-instruct-v01"

IBM served codellama

[models]
  provider = "ChatIBMGenAI"

  [models.args]
  model_id = "meta-llama/llama-2-13b-chat"

IBM served llama3

  # Note:  llama3 complains if we use more than 2048 tokens
  # See:  https://github.com/konveyor-ecosystem/kai/issues/172
[models]
  provider = "ChatIBMGenAI"

  [models.args]
  model_id = "meta-llama/llama-3-70b-instruct"
  parameters.max_new_tokens = 2048

Ollama

[models]
  provider = "ChatOllama"

  [models.args]
  model = "mistral"

OpenAI GPT 4

[models]
  provider = "ChatOpenAI"

  [models.args]
  model = "gpt-4"

OpenAI GPT 3.5

[models]
  provider = "ChatOpenAI"

  [models.args]
  model = "gpt-3.5-turbo"

Kai will also work with OpenAI API Compatible alternatives.

Setup

Running Kai's backend involves running 2 processes:

Steps

  1. Clone Repo and Ensure you have the virtual environment setup
    1. git clone https://github.com/konveyor-ecosystem/kai.git
    2. cd kai
    3. python3 -m venv env
      • We've tested this with Python 3.11 and 3.12
    4. source env/bin/activate
    5. pip install -r ./requirements.txt
    6. pip install -e .
  2. Run the Postgres DB via podman
    1. Open a new shell tab
    2. source env/bin/activate
    3. Let this run in background: make run-postgres
  3. Run the Kai server in background
    1. Open a new shell tab
    2. source env/bin/activate
    3. Let this run in background: make run-server
      • If you want to run with cached LLM responses run with DEMO_MODE=true
      • Replace the above command and instead run: DEMO_MODE=true make run-server
      • The DEMO_MODE option will cache responses and play them back on subsequent runs.
      • If you want to run with debug information set the environment variable LOG_LEVEL=debug
      • Example: LOG_LEVEL=debug make run-server
  4. Load data into the database
    1. source env/bin/activate
    2. Fetch sample apps: pushd samples; ./fetch_apps.py; popd
    3. make load-data
      • This will complete in ~1-2 minutes

How to use Kai?

Client Usage

Demo

Demo Overview

What are the general steps of the demo?

  1. We launch VSCode with our Kai VS Code extension from konveyor-ecosystem/kai-vscode-plugin
  2. We open a git checkout of a sample application: coolstore
  3. We run Kantra inside of VSCode to do an analysis of the application to learn what issues are present that need to be addressed before migrating to Quarkus
  4. We view the analysis information in VSCode
  5. We look at the impacted files and choose what files/issues we want to fix
  6. We click 'Generate Fix' in VSCode on a given file/issue and wait ~45 seconds for the Kai backend to generate a fix
  7. We view the suggested fix as a 'Diff' in VSCode
  8. We accept the generated fix
  9. The file in question has now been updated
  10. We move onto the next file/issue and repeat

Demo Video

DemoVideo

Guided walk-through using Kai

Notes on DEMO_MODE and cached responses

The kai server will always cache responses in the kai/data/vcr/<application_name>/<model> directory. In non-demo mode, these responses will be overwritten whenever a new request is made. When the server is run with DEMO_MODE=true, these responses will be played back. The request will be matched on everything except for authorization headers, cookies, content-length and request body.

DEMO_MODE Cached Responses

DEMO_MODE Updating Cached Responses

There are two ways to record new responses:

  1. Run the requests while the server is not in DEMO_MODE
  2. Delete the specific existing cached response (under kai/data/vcr/<application_name>/<model>/<source-file-path-with-slashes-replaced-with-dashes.java.yaml>), then rerun. When a cached response does not exist, a new one will be recorded and played back on subsequent runs.

Contributors

Updating requirements.txt

Ensure you have the source code for the sample applications checked out locally

  1. cd ./samples
  2. ./fetch_apps.py
    • This will checkout the sample app source code to: ./samples/sample_repos
      • This directory is in .gitignore

(OPTIONAL) Run an analysis of a sample app (example for MacOS)

Note: We have checked in analysis runs for all sample applications so you do NOT need to run analysis yourself. The instructions below are ONLY if you want to recreate, this is NOT required

  1. Install podman so you can run Kantra for static code analysis
  2. cd samples
  3. ./fetch_apps.py # this will git clone example source code apps
  4. cd macos
  5. ./restart_podman_machine.sh # setups the podman VM on MacOS so it will mount the host filesystem into the VM
  6. ./get_latest_kantra_cli.sh # fetches 'kantra' our analyzer tool and stores it in ../bin
  7. cd ..
  8. ./analyze_apps.py # Analyzes all sample apps we know about, in both the 'initial' and 'solved' states, expect this to run for ~2-3 hours.

Analysis data will be stored in: samples/analysis_reports/{APP_NAME}/<initial|solved>/output.yaml

Linting

  1. Install trunk via: https://docs.trunk.io/check#install-the-cli
  2. Run the linters: trunk check
  3. Format code: trunk fmt

Testing

How to run regression tests

  1. Install the prerequisites in Setup and activate the python virtual environment
  2. Ensure you've checked out the source code for sample applications: Run: ./samples/fetch_sample_apps.sh
  3. Run: ./run_tests.sh

Prototype

This repository represents a prototype implementation as the team explores the solution space. The intent is for this work to remain in the konveyor-ecosystem as the team builds knowledge in the domain and experiments with solutions. As the approach matures we will integrate this properly into Konveyor and seek to promote to github.com/konveyor organization.

Code of Conduct

Refer to Konveyor's Code of Conduct here.