Document generation can be started on multiple instances

[Ashtonville]

Add mechanism to prevent starting document generation on multiple parallel instances

[aleneum]

I think this should be tackled on two levels: First, when submitting a generation request, the submit button should be deactivated until the result is returned. Second, an option to limit the allowed parallel executions on the backend.

[aleneum]

I just tried to disable a button in a streamlit GUI and -- oh boy -- that is way harder than necessary, especially when it should be possible to reactivate the button after processing is done. I am not sure whether streamlit is the right choice for an interactive (chat-like) app. I will have a look at Gradio.

https://discuss.streamlit.io/t/is-there-a-way-to-disable-st-form-submit-button-after-input/35130 https://discuss.streamlit.io/t/make-input-fields-required-using-st-form/30822 https://discuss.streamlit.io/t/validation-of-data-entry-fields-and-focus/26691

Streamlit seems to redraw the entire GUI after a button was clicked. During the redraw the clicked button's return value will be true. During the redraw, button states will be adjusted according to the state at draw time. When a state changes during a process, this will not update the button until a redraw is triggered. The redraw will, however, also clear all dynamically set fields as well. Like for instance the output of a request. Do deal with this, one could set the output as a state variable itself and reassign the value after the redraw.

I do had some weird effects that changed field return values passed to field verification triggered by the on_click callback of the submit button required two reloads to be effective. Maybe that's also something that is better handed over to the session state...

[aleneum]

Gradio ist etwas gewöhnungsbedürftig, aber verhindert unter anderem, dass ein Model doppelt ausgeführt wird. Außerdem werden mehrere submits von mehreren Verbindungen auch 'gequeued', was quasi beide von mir angesprochenenden Probleme löst. Für die Feldvalidierung muss man sich etwas strecken, aber es geht auch. So sieht das Ergebnis aus:

@Ashtonville, @Depo14: Was haltet ihr davon? Ich denke, dass Gradio für den Einsatzbereich etwas besser geeignet ist. Das Programmieren von Streamlit ist irgendwie schöner, aber bei komplexen Konfigurationen und Verifikationen kommt es anscheinend schnell an seine Grenzen. Den QA-Fall kann ich auch relativ schnell umstellen. Der sieht dann nicht groß anders aus, wenn Ihr sagt, ihr macht mit Gradio weiter.

[aleneum]

Damit ihr seht, wie Gradio tickt: So sieht die gen_frontend.py aktuell aus:

import logging
from typing import List, Tuple

import gradio as gr

from team_red.backend import TRANSPORTER
from team_red.config import CONFIG
from team_red.transport import PromptConfig

_LOGGER = logging.getLogger(__name__)
_LOGGER.addHandler(logging.NullHandler())

PROMPT = """Du bist ein hilfreicher Assistant.\
Du wandelst Eckdaten in ein fertiges Dokument um.
Du gibst ausschließlich das fertige Dokument zurück und nichts anderes.\
Die Eckdaten lauten wie folgt:
Der Aufenthaltsverlauf des Patienten: {history}\n
Der Name des Arztes der im Anschreiben angegeben werden soll: {doctor_name}\n
Der Name des Patienten, um den es geht: {patient_name}\n
Das Krankenhaus, bei dem der Patient behandelt wurde: {hospital}\n
Generiere daraus das Dokument:"""

_field_labels = {
    "history": "Patientengeschichte",
    "doctor_name": "Name des behandelnden Hausarztes",
    "patient_name": "Name des Patienten",
    "hospital": "Name des Krankenhauses",
}

def _pairwise(fields: Tuple[gr.Textbox]) -> List[gr.Textbox]:
    a = iter(fields)
    return zip(a, a, a)

def generate(*fields: Tuple[gr.Textbox]) -> str:
    params = {}
    for key, name, value in _pairwise(fields):
        if not value:
            msg = f"Feld '{name}' darf nicht leer sein!"
            raise gr.Error(msg)
        params[key] = value
    response = TRANSPORTER.generate(params)
    return response.text

with gr.Blocks() as demo:
    config = TRANSPORTER.set_gen_prompt(PromptConfig(text=PROMPT))
    if not config.parameters:
        config.parameters = {}
    fields = []
    for key in config.parameters:
        fields.append(gr.Textbox(key, visible=False))
        fields.append(gr.Textbox(_field_labels.get(key, key), visible=False))
        fields.append(gr.Textbox(label=_field_labels.get(key, key)))
    output = gr.TextArea(label="Dokument")
    submit_button = gr.Button("Generiere Dokument")
    submit_button.click(fn=generate, inputs=fields, outputs=output)

if __name__ == "__main__":
    logging.basicConfig(level=logging.DEBUG)
    demo.launch()

[aleneum]

I tried to get a better grip on streamlits state model and found a solution that also works:

import logging
from typing import TYPE_CHECKING, List

import streamlit as st

from team_red.backend import TRANSPORTER
from team_red.transport import PromptConfig

if TYPE_CHECKING:
    from streamlit.delta_generator import DeltaGenerator

_LOGGER = logging.getLogger(__name__)
_LOGGER.addHandler(logging.NullHandler())

PROMPT = """Du bist ein hilfreicher Assistant.\
Du wandelst Eckdaten in ein fertiges Dokument um.
Du gibst ausschließlich das fertige Dokument zurück und nichts anderes.\
Die Eckdaten lauten wie folgt:
Der Aufenthaltsverlauf des Patienten: {history}\n
Der Name des Arztes der im Anschreiben angegeben werden soll: {doctor_name}\n
Der Name des Patienten, um den es geht: {patient_name}\n
Das Krankenhaus, bei dem der Patient behandelt wurde: {hospital}\n
Generiere daraus das Dokument:"""

_field_labels = {
    "history": "Patientengeschichte",
    "doctor_name": "Name des behandelnden Hausarztes",
    "patient_name": "Name des Patienten",
    "hospital": "Name des Krankenhauses",
}

def gen_frontend() -> None:
    if "processing" not in st.session_state:
        st.session_state.processing = False
        st.session_state.error = ""
        st.session_state.output = ""

    def process() -> None:
        st.session_state.processing = True

    # Define the Streamlit app layout
    st.title("Dokument-Generator mit Llama 2")
    fields = {}
    with st.form("Form zur Generierung eines Dokumentes"):
        # User input for letter of dismissal
        st.markdown("### Details")
        config = TRANSPORTER.set_gen_prompt(PromptConfig(text=PROMPT))
        if not config.parameters:
            config.parameters = {}
        for key, value in config.parameters.items():
            fields[key] = st.text_input(_field_labels.get(key, key), value=value)

        # Generate LLM repsonse
        generate_cover_letter = st.form_submit_button(
            "Generiere Dokument", disabled=st.session_state.processing, on_click=process
        )

    dynamics: List[DeltaGenerator] = []
    if st.session_state.error:
        message = st.empty()
        message.error(st.session_state.error)
        dynamics.append(message)

    elif st.session_state.output:
        # Offering download link for generated cover letter
        done = st.empty()
        done.success("Fertig!")
        header1 = st.empty()
        header1.subheader("Generiertes Dokument:")
        output = st.empty()
        output.markdown(st.session_state.output)
        header2 = st.empty()
        header2.subheader("Download generiertes Dokument:")
        download_button = st.empty()
        download_button.download_button(
            "Download generiertes Dokument als .txt",
            st.session_state.output,
            key="cover_letter",
        )
        dynamics = [done, header1, output, header2, download_button]

    if generate_cover_letter:
        if all(val for val in fields.values()):
            for m in dynamics:
                m.empty()
            with st.spinner("Generiere Dokument..."):
                st.session_state.output = TRANSPORTER.generate(fields).text
                st.session_state.error = ""
        else:
            st.session_state.error = "Eingaben unvollständig"
        st.session_state.processing = False
        st.experimental_rerun()

[Ashtonville]

Streamlit seems to redraw the entire GUI after a button was clicked. During the redraw the clicked button's return value will be true. During the redraw, button states will be adjusted according to the state at draw time. When a state changes during a process, this will not update the button until a redraw is triggered. The redraw will, however, also clear all dynamically set fields as well. Like for instance the output of a request. Do deal with this, one could set the output as a state variable itself and reassign the value after the redraw.

Yes, I read about that too. I think it would be a better choice overall to continue with gradio.

[Depo14]

Looks to me like Gradio is the better choice. From my point of view, we are welcome to switch

[aleneum]

I opened a PR and asked for your review

[aleneum]

this should be handled via gradio