Research: Structure Map Generation Using LLMs

[lincmba]

Extracted from https://github.com/onaio/canopy/issues/3094

Background:

The goal is to enhance Efsity by introducing StructureMap generation capabilities. This involves sending a Questionnaire Response (QR) to a Large Language Model (LLM) and obtaining resources, such as Patient and Encounter. The subsequent step is to instruct the LLM to create a StructureMap for converting QR to Patient and Encounter.

The process can be extended by running the generated StructureMap against the QR, comparing it with the requested output, and iterating until a maximum number of loops or convergence is achieved. This combined approach involves generating QR and StructureMap consecutively, ensuring seamless integration.

Initial Attempt:

Started with Code Llama using the setup instructions provided here. Downloaded the 7B Large Language Model (LLM) and utilized code similar to example_completion.py for local testing. Encountered an issue as the code required GPU, which was not explicitly mentioned in the installation instructions.

Troubleshooting:

1. Attempted cloud options for LLM, including the Code Llama Hugging Face model here.
2. Observed successful functioning with smaller textual data but encountered errors when the input text exceeded a certain length.

Next Steps:

Currently exploring two additional cloud options:

1. SantaCoder LLM
2. Replit Code LLM

Resources: OpenSource Code LLMs https://github.com/eugeneyan/open-llms#open-llms-for-code

[pld]

let's discuss before we continue on this, I want to make sure the rest of tooling is solid so that we can align work on this with the other tooling