-
Add `chat` prompt type for HumanEvalPack tasks for more realistic and fair model evaluation and comparison.
## Motivation
As implied in the [docs](https://github.com/bigcode-project/bigcode-eval…
-
### Describe the bug
I am typing a simple error fix as it's stuck in some kind of loop where it starts to fix a file but then half way through it just stops and errors out.
Now i'm just getting p…
-
In the DataComp paper (original one for VLM's), some of the heuristics were based on features from the datasets that were used for evaluations. Is this permitted in the filtering track for DCLM? For e…
-
### Windows Version
Microsoft Windows [Version 10.0.26100.2033]
### WSL Version
2.3.24.0
### Are you using WSL 1 or WSL 2?
- [ ] WSL 2
- [ ] WSL 1
### Kernel Version
PS C:\Users\Chita > cat /pr…
-
hi when i run demo.ipynb
with open('/root/llm-reasoners/examples/CoT/blocksworld/prompts/pool_prompt_v1.json') as f:
prompt = json.load(f) #/root/llm-reasoners/examples
evaluator …
-
### Validations
- [X] I believe this is a way to improve. I'll try to join the [Continue Discord](https://discord.gg/NWtdYexhMs) for questions
- [X] I'm not able to find an [open issue](https://githu…
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a sim…
-
Project Role- Put your writing skills to the test to beat the cutting edge models! Create your own prompts and improve the model response to be perfect!!
?
Thnx.
-
**Describe the feature request**
At the moment when there are unsaved changes to a datasource the page can be exited without confirming the change. This can be easy to miss as e.g. the rest datasourc…