A small problem encountered in the use of

[runningZ1]

I am getting this error in my usage. My educational background is not in computers and my programming foundation is weak, so I hope the reply is easy to understand. Also I would like to say that this program and method I really like !

[Eladlev]

Hi, Thanks for the feedback! The issue you encounter occurs when using an LLM as the annotator and providing instructions which are in conflict with the label_schema. Please provide all the changes you made in the config files. If you need help with optimizing the config settings for your specific task, you can also contact us in the discord channel

[runningZ1]

I took a screenshot of all the files about label_schema in the config folder, could you please help me point out where there is a problem, and also direct me to new information if it is not enough! 🤗

[Eladlev]

Hi, Can you please provide the following information:

1. What script are you using (run_pipeline or run_generation_pipeline)?
2. What are the config files you are using in this script (for example in the batch_config_path file is it empty)? Please provide all the config yaml files that you are using in this script
3. Please provide the 'annotator' values of the relevant config file that you are using

[runningZ1]

I am using the run_generation_pipeline command This is all the contents of the config folder, please also point out the error 👇 config.zip

[Eladlev]

Please observe that you modify the label_schema from the default values to: label_schema: True The label schema according to the instruction you provide in the annotator should be the default ones: label_schema: ["Yes", "No"]

[runningZ1]

Thank you for the help you provided earlier🙏. I have tried to follow your advice to solve my problem but unfortunately I am still experiencing some difficulties. I may not have fully understood your guidance😞

The main problem I'm currently facing is that I can't find the label_schema configuration in the annotator section of the config_default.yml file, which seems to be all in the dataset section. I'm trying to run the script run_generation_pipeline.py, and although I've scrutinized the examples documentation, I'm still confused as to how to fix this.

[qiulongquan]

It's a great product. I love it. but, I have the same problem happening, running the run_generation_pipeline.py program gives At least one label specified must be in y_true error. I have packed all the relevant configuration files as well as the py program files as attachments, can you please help me with this issue? AutoPrompt_20240301.zip

[Eladlev]

Thank you for the help you provided earlier🙏. I have tried to follow your advice to solve my problem but unfortunately I am still experiencing some difficulties. I may not have fully understood your guidance😞

The main problem I'm currently facing is that I can't find the label_schema configuration in the annotator section of the config_default.yml file, which seems to be all in the dataset section. I'm trying to run the script run_generation_pipeline.py, and although I've scrutinized the examples documentation, I'm still confused as to how to fix this.

I think the simplest thing is do the following steps:

1. Revert all the changes you made in the config files (go back to default setting)

2. In the 'config/config_diff' folder create a new config files 'config_llm.yml', the file should contain the following content: annotator: method: "llm" config: llm: type: "OpenAI" name: "gpt-4-1106-preview" instruction: "Assess whether the text contains content that is not relevant to the topic. Answer Yes if it does and No otherwise." num_workers: 5 prompt: "prompts/predictor_completion/prediction.prompt" mini_batch_size: 5 mode: "annotation"

3. In the run_pipeline.py script, run it with the flag: --batch_config_path config/config_diff/config_llm.yml

This will override the default annotator field, with your llm configuration. Since the default label scheme is 'Yes','No' the revert will resolve your issue.

We are aware of the challenges of handling the config files, we are working on simplify this process and make it more 'user friendly'

[Eladlev]

It's a great product. I love it. but, I have the same problem happening, running the run_generation_pipeline.py program gives At least one label specified must be in y_true error. I have packed all the relevant configuration files as well as the py program files as attachments, can you please help me with this issue? )

@qiulongquan Please look at the following issue, changing the annotator instructions to do ranking in scale 1-5 will resolve your problem https://github.com/Eladlev/AutoPrompt/issues/36

[qiulongquan]

It's a great product. I love it. but, I have the same problem happening, running the run_generation_pipeline.py program gives At least one label specified must be in y_true error. I have packed all the relevant configuration files as well as the py program files as attachments, can you please help me with this issue? )

@qiulongquan Please look at the following issue, changing the annotator instructions to do ranking in scale 1-5 will resolve your problem #36

Thanks for the reply I've solved the problem, but I've found a new one. I'll open a new issue separately