Watts-Lab / atlas

The product of all our research cartography
https://atlas.seas.upenn.edu
GNU Affero General Public License v3.0
1 stars 1 forks source link

Schema for expert input on gpt_responses #54

Open markwhiting opened 1 week ago

markwhiting commented 1 week ago

A quick thought here: perhaps the main edit models are as rows or columns, e.g., either a paper was misunderstood or a column needs retraining.

I suspect that this way of thinking about it lets us kind of focus on these things separately.

  1. if a paper is needs editing, the row should be editable and, although it might improve training a little bit through something like DSM, it doesn't do too much. Perhaps also if new columns are needed for that paper, it reminds us to check it‚ because we know the paper is strange.
  2. if a column is needs editing we should make the column (or set of columns editable and let a user adjust values across papers for that column. This should then be used to run something like DSM or some mechanism for improving the prompt, to achieve better results downstream