instructlab / instructlab-bot

GitHub bot to assist with the taxonomy contribution workflow
Apache License 2.0
14 stars 16 forks source link

Resolve precheck context passing #325

Closed nerdalert closed 5 months ago

nerdalert commented 5 months ago
2024-05-02T03:23:43.835Z    INFO    cmd/generate.go:342 Running the precheck command: /usr/local/bin/ilab chat --quick-question can you solve the following riddle? Two turkeys in front of the turkey, two turkeys behind the turkey, tell me how many total turkeys? --context Form HOTDOGS HAMBURGERS PANCAKES --tls-insecure --endpoint-url https://merlinite-7b-vllm-openai.apps.fmaas-backend.fmaas.res.ibm.com/v1 --model /shared_model_storage/transformers_cache/models--ibm--merlinite-7b/snapshots/233d12759d5bb9344231dafdb51310ec19d79c0e
2024-05-02T03:23:49.463Z    INFO    cmd/generate.go:342 Running the precheck command: /usr/local/bin/ilab chat --quick-question question-2 can you solve the following riddle? Two turkeys in front of the turkey, two turkeys behind the turkey, tell me how many total turkeys? --context HOTDOGS HAMBURGERS PANCAKES HOTDOGS --tls-insecure --endpoint-url https://merlinite-7b-vllm-openai.apps.fmaas-backend.fmaas.res.ibm.com/v1 --model /shared_model_storage/transformers_cache/models--ibm--merlinite-7b/snapshots/233d12759d5bb9344231dafdb51310ec19d79c0e
2024-05-02T03:23:54.234Z    INFO    cmd/generate.go:342 Running the precheck command: /usr/local/bin/ilab chat --quick-question question-3 can you solve the following riddle? Two turkeys in front of the turkey, two turkeys behind the turkey, tell me how many total turkeys? --context HOTDOGS HAMBURGERS PANCAKES HOTDOGS --tls-insecure --endpoint-url https://merlinite-7b-vllm-openai.apps.fmaas-backend.fmaas.res.ibm.com/v1 --model /shared_model_storage/transformers_cache/models--ibm--merlinite-7b/snapshots/233d12759d5bb9344231dafdb51310ec19d79c0e
2024-05-02T03:24:00.355Z    INFO    cmd/generate.go:342 Running the precheck command: /usr/local/bin/ilab chat --quick-question question-4 can you solve the following riddle? Two turkeys in front of the turkey, two turkeys behind the turkey, tell me how many total turkeys? --context HOTDOGS HAMBURGERS PANCAKES HOTDOGS --tls-insecure --endpoint-url https://merlinite-7b-vllm-openai.apps.fmaas-backend.fmaas.res.ibm.com/v1 --model /shared_model_storage/transformers_cache/models--ibm--merlinite-7b/snapshots/233d12759d5bb9344231dafdb51310ec19d79c0e
2024-05-02T03:24:08.244Z    INFO    cmd/generate.go:342 Running the precheck command: /usr/local/bin/ilab chat --quick-question question-5 can you solve the following riddle? Two turkeys in front of the turkey, two turkeys behind the turkey, tell me how many total turkeys? --context HOTDOGS HAMBURGERS PANCAKES HOTDOGS --tls-insecure --endpoint-url https://merlinite-7b-vllm-openai.apps.fmaas-backend.fmaas.res.ibm.com/v1 --model /shared_model_storage/transformers_cache/models--ibm--merlinite-7b/snapshots/233d12759d5bb9344231dafdb51310ec19d79c0e

Closes #306

nerdalert commented 5 months ago

@mingxzhao have we opened a bug in instructlab yet? I will pop one if not. I can replicate the bug with just the cli. I had left it as is in the patch.

Example:


[root@f27bd496fb8f data]# /usr/local/bin/ilab chat --quick-question can you solve the following riddle? Two turkeys in front of the turkey, two turkeys behind the turkey, tell me how many total turkeys?  --context HOTDOGS HAMBURGERS PANCAKES
INFO 2024-05-02 20:30:58,031 chat.py:479 Context HOTDOGS not found in the config file. Using default.

Only reluctance to patch it is when it gets fixed upstream there will be a float period but since this is time sensitive I'll add a buffer word. If you can help us track it we can hopefully not break for long when it gets fixed. Ty!

mingxzhao commented 5 months ago

No no bug opened yet, and I will keep an eye out for a fix when merged.