While it seems to be great for general prompt tunning (looking into prompt-completion pairs, costs, changing the prompt and seeing the result, annotating, etc.) it seems to be suboptimal for storage of arbitrary prediction-related data.
But before using anything else, check out if we can:
store arbitrary large and complex data in langfuse in reasonable manner
for example, prediction market dump, scraped websites, etc.
if we can update the data with resolution, after it's known
if we can index them and query them effectively
for example, if we would like all the data between dates X and Y for the agent A where he was wrong, do we need to query everything and filter manually (would be problem in the future) or filter for them as with SQL in Postgres?
If all seems okay with Langfuse, let's use that, if not, Postgres is our go-to choice atm. We need:
Dumped prediction market data at the time of prediction
Queries to search and their results (including scraped data)
We have two pending issues for Langfuse:
While it seems to be great for general prompt tunning (looking into prompt-completion pairs, costs, changing the prompt and seeing the result, annotating, etc.) it seems to be suboptimal for storage of arbitrary prediction-related data.
But before using anything else, check out if we can:
If all seems okay with Langfuse, let's use that, if not, Postgres is our go-to choice atm. We need: