-
excellent work!I'm writing to inquire about the possibility of adding support for multi-GPU evaluation to your evaluation framework. Currently, it seems that the existing evaluations are only designed…
-
# Model presets
LeapfrogAI currently has two primary models that are used on the backend, but more should be added/tested. By implementing certain small models and evaluating their efficacy from a hu…
-
Can some one kindly help me solve this problem?
**When I run this:**
`python3 ./pytorch/train.py evaluate --config_path=./configs/car.fhd.config --model_dir=/home//catkin_ws/src/SECOND-ROS/KITTI_P…
-
@wilkox
Thanks for creating GPTscreenR! It’s a super useful tool for scoping reviews. I noticed that it currently supports GPT-4 through the OpenAI API, which works great, but I was wondering if y…
-
### Prerequisite
- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the ex…
-
Hi! We tried evaluating the base models using the starting kit evaluation pipeline. Here are some points/issues:
1. For phi2 and llama models, we are getting 'prediction not found' error.
2. Could …
-
Please check the updated [README](https://github.com/amazon-science/chronos-forecasting). We have also released an evaluation script and backtest configs to compute the WQL and MASE numbers as reporte…
-
## Description
As part of the deliverables for an MVP Evals framework, we will need a short list of LLMs we are evaluating on as part of LFAI. The models chosen should fit the following criteria (with…
-
### Describe the feature and motivation
Currently the methods related to camera calibration (`calibrateCamera`) or pose estimation (flavours of `solvePnP`) return overall RMS reprojection errors. As …
-
_This issue has been discussed verbally in recent history; nonetheless, being made explicit in this fashion allows others to provide their thoughts and list considerations for this task_
Once is c…