mc-bench / mc-bench-backend

Other
2 stars 0 forks source link

Prompt Response post processing #8

Open huntcsg opened 1 week ago

huntcsg commented 1 week ago

There are a handful of categorical errors a model might make.

We should develop an easily extendable cascade of post processing steps, including removal of external libraries not likely to be present, but also to include issues like:

  1. putting code in backticks
  2. using different tags than the ones we specific (<inspiration>, <description>, <code>)
huntcsg commented 5 days ago

Note to self: I did NOT implement this as an extendable cascade.

I just did it: