Run tests in build pr4 Phonetic 5 qa with context - Githubissues

nerdalert / taxonomy

Apache License 2.0

0 stars 0 forks source link

Run tests in build pr4 Phonetic 5 qa with context #5

Open nerdalert opened 6 months ago

nerdalert commented 6 months ago

If your PR is related to a contribution to the taxonomy, please, fill out the following questionnaire. If not, replace this whole text and the following questionnaire with whatever information is applicable to your PR.

Describe the contribution to the taxonomy

...
...
...

Input given at the prompt

...

Response from the original model

...

Response from the fine-tuned model

...

Contribution checklist

[ ] The contribution was tested with lab generate
[ ] No errors or warnings were produced by lab generate
[ ] All commits are signed off (DCO)
[ ] The qna.yaml file contains at least 5 seed_examples
[ ] The qna.yaml file was linted and prettified (yaml-validator can do both)

nerdalert commented 6 months ago

@brents-pet-robot boop

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Sorry, I don't understand that command

nerdalert commented 6 months ago

@brents-pet-robot generate-local

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 4.

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 an error occurred while processing your request, please review the following log for job id 4 :

Error running command (/usr/local/bin/ilab generate --num-instructions 10 --output-dir /data/generate-pr-5-274cb0fc9045dd0770d307a12ee60ea7457e7bb8): exit status 1. 
Details: INFO 2024-04-09 21:50:35,067 _client.py:1026 HTTP Request: GET http://127.0.0.1:8000/v1/models "HTTP/1.1 200 OK"
ERROR 2024-04-09 21:50:35,171 generate_data.py:642 Validation error in compositional_skills/extraction/inference/qualitative/siblings/qna.yaml on .seed_examples[0]: Unevaluated properties are not allowed ('attribution' was unexpected)
ERROR 2024-04-09 21:50:35,172 generate_data.py:642 Validation error in compositional_skills/extraction/inference/qualitative/siblings/qna.yaml on .seed_examples[1]: Unevaluated properties are not allowed ('attribution' was unexpected)
ERROR 2024-04-09 21:50:35,172 generate_data.py:642 Validation error in compositional_skills/extraction/inference/qualitative/siblings/qna.yaml on .seed_examples[2]: Unevaluated properties are not allowed ('attribution' was unexpected)
ERROR 2024-04-09 21:50:35,172 generate_data.py:642 Validation error in compositional_skills/extraction/inference/qualitative/siblings/qna.yaml on .seed_examples[3]: Unevaluated properties are not allowed ('attribution' was unexpected)
ERROR 2024-04-09 21:50:35,172 generate_data.py:642 Validation error in compositional_skills/extraction/inference/qualitative/siblings/qna.yaml on .seed_examples[4]: Unevaluated properties are not allowed ('attribution' was unexpected)
5 taxonomy files with errors! Exiting.

nerdalert commented 6 months ago

@brents-pet-robot generate-local

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 5.

nerdalert commented 6 months ago

@brents-pet-robot generate-local

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 5.

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 an error occurred while processing your request, please review the following log for job id 5 :

Error running command (/usr/local/bin/ilab generate --num-instructions 10 --output-dir /data/generate-pr-5-274cb0fc9045dd0770d307a12ee60ea7457e7bb8): exit status 1. 
Details: INFO 2024-04-09 22:39:33,730 _client.py:1026 HTTP Request: GET http://127.0.0.1:8000/v1/models "HTTP/1.1 200 OK"
INFO 2024-04-09 22:39:33,814 rouge_scorer.py:83 Using default tokenizer.

  0%|          | 0/10 [00:00<?, ?it/s]INFO 2024-04-09 22:39:33,817 generate_data.py:445 Selected taxonomy path compositional_skills->extraction->inference->qualitative->siblings
INFO 2024-04-09 22:42:40,690 _client.py:1026 HTTP Request: POST http://127.0.0.1:8000/v1/chat/completions "HTTP/1.1 200 OK"

 20%|██        | 2/10 [03:06<12:27, 93.47s/it]INFO 2024-04-09 22:42:40,760 generate_data.py:445 Selected taxonomy path compositional_skills->extraction->inference->qualitative->siblings
INFO 2024-04-09 22:44:03,154 _client.py:1026 HTTP Request: POST http://127.0.0.1:8000/v1/chat/completions "HTTP/1.1 200 OK"

 40%|████      | 4/10 [04:29<06:16, 62.74s/it]INFO 2024-04-09 22:44:03,218 generate_data.py:445 Selected taxonomy path compositional_skills->extraction->inference->qualitative->siblings
INFO 2024-04-09 22:45:16,763 _client.py:1026 HTTP Request: POST http://127.0.0.1:8000/v1/chat/completions "HTTP/1.1 200 OK"

 60%|██████    | 6/10 [05:43<03:23, 50.90s/it]INFO 2024-04-09 22:45:16,826 generate_data.py:445 Selected taxonomy path compositional_skills->extraction->inference->qualitative->siblings
INFO 2024-04-09 22:46:39,157 _client.py:1026 HTTP Request: POST http://127.0.0.1:8000/v1/chat/completions "HTTP/1.1 200 OK"
INFO 2024-04-09 22:46:39,159 generate_data.py:445 Selected taxonomy path compositional_skills->extraction->inference->qualitative->siblings
INFO 2024-04-09 22:47:43,929 _client.py:1026 HTTP Request: POST http://127.0.0.1:8000/v1/chat/completions "HTTP/1.1 500 Internal Server Error"
INFO 2024-04-09 22:47:43,930 _base_client.py:1020 Retrying request to /chat/completions in 0.870917 seconds
INFO 2024-04-09 22:47:44,802 _base_client.py:1020 Retrying request to /chat/completions in 1.775271 seconds

 60%|██████    | 6/10 [08:12<05:28, 82.13s/it]

nerdalert commented 6 months ago

@brents-pet-robot generate-local

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 6.

nerdalert commented 6 months ago

@brents-pet-robot generate-local

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 1.

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 an error occurred while processing your request, please review the following log for job id 1 :

Error running command (/usr/local/bin/ilab generate --num-instructions 10 --output-dir /data/generate-pr-5-274cb0fc9045dd0770d307a12ee60ea7457e7bb8 --endpoint-url https://merlinite-7b-vllm-openai.apps.fmaas-backend.fmaas.res.ibm.com/v1 --model /shared_model_storage/transformers_cache/models--ibm--merlinite-7b/snapshots/233d12759d5bb9344231dafdb51310ec19d79c0e): exit status 1. 
Details: INFO 2024-04-09 22:52:45,435 rouge_scorer.py:83 Using default tokenizer.

  0%|          | 0/10 [00:00<?, ?it/s]INFO 2024-04-09 22:52:45,437 generate_data.py:445 Selected taxonomy path compositional_skills->extraction->inference->qualitative->siblings
INFO 2024-04-09 22:52:45,501 _base_client.py:1020 Retrying request to /chat/completions in 0.811311 seconds
INFO 2024-04-09 22:52:46,328 _base_client.py:1020 Retrying request to /chat/completions in 1.645390 seconds

  0%|          | 0/10 [00:02<?, ?it/s]

nerdalert commented 6 months ago

@brents-pet-robot generate-local

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 1.

nerdalert commented 6 months ago

@brents-pet-robot generate-local

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 2.

nerdalert commented 6 months ago

@brents-pet-robot generate-local

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 3.

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 an error occurred while processing your request, please review the following log for job id 3 :

Error running command (/usr/local/bin/ilab generate --num-instructions 10 --output-dir /data/generate-pr-5-274cb0fc9045dd0770d307a12ee60ea7457e7bb8): exit status 1. 
Details: INFO 2024-04-10 00:14:03,365 _base_client.py:1020 Retrying request to /models in 0.994307 seconds
INFO 2024-04-10 00:14:04,361 _base_client.py:1020 Retrying request to /models in 1.799336 seconds
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:   no
ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes
ggml_cuda_init: found 1 CUDA devices:
  Device 0: Tesla T4, compute capability 7.5, VMM: yes
ERROR 2024-04-10 00:14:07,520 generate_data.py:642 Validation error in compositional_skills/extraction/inference/qualitative/siblings/qna.yaml on .seed_examples[0]: Unevaluated properties are not allowed ('attribution' was unexpected)
ERROR 2024-04-10 00:14:07,520 generate_data.py:642 Validation error in compositional_skills/extraction/inference/qualitative/siblings/qna.yaml on .seed_examples[1]: Unevaluated properties are not allowed ('attribution' was unexpected)
ERROR 2024-04-10 00:14:07,520 generate_data.py:642 Validation error in compositional_skills/extraction/inference/qualitative/siblings/qna.yaml on .seed_examples[2]: Unevaluated properties are not allowed ('attribution' was unexpected)
ERROR 2024-04-10 00:14:07,521 generate_data.py:642 Validation error in compositional_skills/extraction/inference/qualitative/siblings/qna.yaml on .seed_examples[3]: Unevaluated properties are not allowed ('attribution' was unexpected)
ERROR 2024-04-10 00:14:07,521 generate_data.py:642 Validation error in compositional_skills/extraction/inference/qualitative/siblings/qna.yaml on .seed_examples[4]: Unevaluated properties are not allowed ('attribution' was unexpected)
5 taxonomy files with errors! Exiting.

nerdalert commented 6 months ago

@brents-pet-robot generate

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 3.

nerdalert commented 6 months ago

@brents-pet-robot generate

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 4.

nerdalert commented 6 months ago

@brents-pet-robot generate

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 5.

nerdalert commented 6 months ago

@brents-pet-robot generate

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 7.

nerdalert commented 6 months ago

@brents-pet-robot generate

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 1.

nerdalert commented 6 months ago

@brents-pet-robot generate

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 2.

nerdalert commented 6 months ago

@brents-pet-robot generate

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 Generating test data for your pull request.

This will take several minutes...

Your job ID is 2.

bot-dev-brent[bot] commented 6 months ago

Beep, boop 🤖 an error occurred while processing your request, please review the following log for job id 2 :

unexpected status code 504: activator request timeout

bot-dev-brent[bot] commented 5 months ago

Generating test data for your PR with the job type: precheck. Your Job ID is 5. The reults will be presented below in the pull request status box. This will take several minutes...

nerdalert commented 5 months ago

@brents-pet-robot generate

nerdalert commented 5 months ago

@brents-pet-robot generate

nerdalert commented 5 months ago

@brents-pet-robot generate

nerdalert commented 5 months ago

@brents-pet-robot enable

bot-dev-brent[bot] commented 5 months ago

Beep, boop .. Hi, I'm @brents-pet-robot and I'm going to help you with your pull request. Thanks for you contribution! 🎉

I support the following commands:

@brents-pet-robot precheck -- Check existing model behavior using the questions in this proposed change.
@brents-pet-robot generate -- Generate a sample of synthetic data using the synthetic data generation backend infrastructure.
@brents-pet-robot generate-local -- Generate a sample of synthetic data using a local model.

[!NOTE] Results or Errors of these commands will be posted as a pull request check in the Checks section below.

nerdalert commented 5 months ago

@brents-pet-robot generate

bot-dev-brent[bot] commented 5 months ago

Beep, boop 🤖, Generating test data for your PR with the job type: sdg-svc. Your Job ID is 1. The results will be presented below in the pull request status box. This may take several minutes...

nerdalert commented 5 months ago

@brents-pet-robot generate

nerdalert commented 5 months ago

@brents-pet-robot enable

bot-dev-brent[bot] commented 5 months ago

Beep, boop .. Hi, I'm @brents-pet-robot and I'm going to help you with your pull request. Thanks for you contribution! 🎉

I support the following commands:

@brents-pet-robot precheck -- Check existing model behavior using the questions in this proposed change.
@brents-pet-robot generate -- Generate a sample of synthetic data using the synthetic data generation backend infrastructure.
@brents-pet-robot generate-local -- Generate a sample of synthetic data using a local model.

[!NOTE] Results or Errors of these commands will be posted as a pull request check in the Checks section below.

nerdalert commented 5 months ago

@brents-pet-robot generate

nerdalert commented 5 months ago

@brents-pet-robot generate

Next