[BUG] BuilderConfig 'qa' not found, when finetunnig custom embedding models

javigsg10 commented 1 month ago

Prerequisites

[X] I have read the documentation.
[X] I have checked other issues for similar problems.

Backend

Local

Interface Used

CLI

CLI Command

autotrain --config /config.yml

UI Screenshots & Parameters

Error Logs

INFO | 2024-06-04 10:02:26 | autotrain.cli.autotrain:main:56 - Using AutoTrain configuration: /config.yml INFO | 2024-06-04 10:02:26 | autotrain.parser:post_init__:124 - Running task: sentence_transformers INFO | 2024-06-04 10:02:26 | autotrain.parser:post_init__:125 - Using backend: local INFO | 2024-06-04 10:02:26 | autotrain.parser:run:185 - {'data_path': 'autotrain-', 'model': 'intfloat/multilingual-e5-large', 'lr': 2e-05, 'epochs': 5, 'max_seq_length': 512, 'batch_size': 8, 'warmup_ratio': 0.1, 'gradient_accumulation': 1, 'optimizer': 'adamw_torch', 'scheduler': 'linear', 'weight_decay': 0.0, 'max_grad_norm': 1.0, 'seed': 42, 'train_split': 'qa:train', 'valid_split': None, 'logging_steps': -1, 'project_name': 'autotrain-qa-', 'auto_find_batch_size': False, 'mixed_precision': 'fp16', 'save_total_limit': 1, 'token': '', 'push_to_hub': True, 'evaluation_strategy': 'epoch', 'username': 'javiergsg', 'log': 'tensorboard', 'early_stopping_patience': 5, 'early_stopping_threshold': 0.01, 'trainer': 'qa', 'sentence1_column': 'query', 'sentence2_column': 'answer', 'sentence3_column': 'sentence3', 'target_column': 'target'} INFO | 2024-06-04 10:02:26 | autotrain.backends.local:create:8 - Starting local training... INFO | 2024-06-04 10:02:26 | autotrain.commands:launch_command:386 - ['accelerate', 'launch', '--cpu', '-m', 'autotrain.trainers.sent_transformers', '--training_config', 'autotrain-qa-/training_params.json'] INFO | 2024-06-04 10:02:26 | autotrain.commands:launch_command:387 - {'data_path': 'autotrain-', 'model': 'intfloat/multilingual-e5-large', 'lr': 2e-05, 'epochs': 5, 'max_seq_length': 512, 'batch_size': 8, 'warmup_ratio': 0.1, 'gradient_accumulation': 1, 'optimizer': 'adamw_torch', 'scheduler': 'linear', 'weight_decay': 0.0, 'max_grad_norm': 1.0, 'seed': 42, 'train_split': 'qa:train', 'valid_split': None, 'logging_steps': -1, 'project_name': 'autotrain-qa', 'auto_find_batch_size': False, 'mixed_precision': 'fp16', 'save_total_limit': 1, 'token': '', 'push_to_hub': True, 'evaluation_strategy': 'epoch', 'username': 'javiergsg', 'log': 'tensorboard', 'early_stopping_patience': 5, 'early_stopping_threshold': 0.01, 'trainer': 'qa', 'sentence1_column': 'query', 'sentence2_column': 'answer', 'sentence3_column': 'sentence3', 'target_column': 'target'} The following values were not passed to accelerate launch and had defaults used instead: --num_processes was set to a value of 0 --num_machines was set to a value of 1 --mixed_precision was set to a value of 'no' --dynamo_backend was set to a value of 'no' To avoid this warning pass in values for each of the problematic parameters or run accelerate config. No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda' ERROR | 2024-06-04 10:02:42 | autotrain.trainers.common:wrapper:120 - train has failed due to an exception: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/autotrain/trainers/common.py", line 117, in wrapper return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/autotrain/trainers/sent_transformers/main.py", line 52, in train train_data = load_dataset( File "/usr/local/lib/python3.10/dist-packages/datasets/load.py", line 2587, in load_dataset builder_instance = load_dataset_builder( File "/usr/local/lib/python3.10/dist-packages/datasets/load.py", line 2296, in load_dataset_builder builder_instance: DatasetBuilder = builder_cls( File "/usr/local/lib/python3.10/dist-packages/datasets/builder.py", line 374, in init self.config, self.config_id = self._create_builder_config( File "/usr/local/lib/python3.10/dist-packages/datasets/builder.py", line 599, in _create_builder_config raise ValueError( ValueError: BuilderConfig 'qa' not found. Available: ['default']

ERROR | 2024-06-04 10:02:42 | autotrain.trainers.common:wrapper:121 - BuilderConfig 'qa' not found. Available: ['default'] INFO | 2024-06-04 10:02:45 | autotrain.parser:run:190 - Job ID: 23091

Additional Information

config.yml

task: sentence-transformers:qa base_model: intfloat/multilingual-e5-large project_name: autotrain-qa log: tensorboard backend: local

data: path: autotrain- train_split: qa:train valid_split: null column_mapping: sentence1_column: query sentence2_column: answer

params: max_seq_length: 512 epochs: 5 batch_size: 8 lr: 2e-5 optimizer: adamw_torch scheduler: linear gradient_accumulation: 1 mixed_precision: fp16

hub: username: javiergsg token: **** push_to_hub: true

javigsg10 commented 1 month ago

I am running autotrain in my local machine and also in colab, and I got the same error, any idea what's going on?

javigsg10 commented 1 month ago

My bad! I changed train_split: train in config.yml and It's working

huggingface / autotrain-advanced