jakublala / smoltalk-legacy

Natural language interface with 3dmol web app
MIT License
5 stars 0 forks source link

Prompts + Results Database #3

Open jakublala opened 1 year ago

jakublala commented 1 year ago

Design and set up a postgres database (on Amazon AWS?) that collects all prompts and results from the LLM.

These will be later used for further development, e.g. reinforcement learning human feedback.

seankwarren commented 1 year ago

Proposed DB design. This issue does not cover user auth, but will build the infrastructure for it now. Auth can be tackled at a later date.

@jakublala Do you foresee any other functionality we should plan for now? Much easier to build it now, from scratch, than it is to change later once it has data in it.

`users`: This table stores information about registered users.
    `id`: unique user ID (primary key)
    `username`: user's name or email
    `password`: user's hashed password (use bcrypt for hashing)

`prompts`: This table stores the user prompts.
    `id`: unique prompt ID (primary key)
    `user_id`: foreign key referencing the users table
    `prompt_text`: text of the user's prompt
    `created_at`: timestamp for when the prompt was created

`responses`: This table stores the LLM-generated responses.
    `id`: unique response ID (primary key)
    `prompt_id`: foreign key referencing the prompts table
    `response_text`: text of the LLM-generated response
    `created_at`: timestamp for when the response was generated

`feedback`: This table stores user feedback on prompt/response pairs.
    `id`: unique feedback ID (primary key)
    `user_id`: foreign key referencing the users table
    `response_id`: foreign key referencing the responses table
    `rating`: integer representing positive (1) or negative (-1) feedback
    `created_at`: timestamp for when the feedback was submitted
jakublala commented 1 year ago

I think this is all good.

One thing we might also consider is somehow storing the order of prompts/responses within a single session? As in have also a session_id maybe that collects all prompts/responses within a single session (i.e. chain of commands)?

And then on top of that, at some point it would be nice to have an option, where users could edit (or remove) their previous commands (prompts) within the chain of commands. It would be nice to store that maybe as well, but unsure how to do that exactly - we would have to somehow also store the order of events rather than just order of prompts and responses.

To give an example, the user does the following:

  1. Writes display a hemoglobin structure
  2. Writes colour chain A yellow
  3. Writes colour all cysteines red
  4. Deletes the 2. command and chain A goes from yellow back to uncoloured.

What do you think?