Reduce the amount of API calls to OpenAI

[jll38]

Reduce these API calls as much as possible to reduce user waiting as well as less utilization.

IDEAS

• Generate only one meal per day (or allow one refresh per meal of the day)
• Store pre-generated meal in session/local storage with an expiry time
• Optimize prompt as much as possible to avoid invalid output to be parsed, thus requiring another api call for a correct response.
• Work around token limit for prompt and responses

[Vladimir-G4]

I think if possible we should implement the following:

• Set a hard limit as to the # of OpenAI API calls on a per-day basis (for the application as a whole).
• For each user, upon requesting a meal recommendation, ask ChatGPT for 3 meal recommendations. Capture all of them but show just the first one, and every time that button is pressed again, cycle through each of the 3 recommended meals. Reset each day. This would be for each of the three buttons (breakfast/lunch/dinner).

We could also look at switching to Google Vertex for now.

If you think there's a better or more efficient implementation feel free to comment below.