ASR TypeError: '<' not supported between instances of 'NoneType' and 'int'

[nitinyadav250]

!bash start_train_ctc.sh [NeMo W 2024-04-05 16:16:10 nemo_logging:349] /usr/local/lib/python3.10/dist-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default. See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information. ret = run_job(

[NeMo I 2024-04-05 16:16:10 speech_to_text_ctc_bpe:99] Hydra config: name: Conformer-CTC-BPE-Large model: sample_rate: 16000 log_prediction: true ctc_reduction: mean_batch skip_nan_grad: false train_ds: manifest_filepath: /content/drive/MyDrive/drive-download-20240405T032246Z-001/train.json sample_rate: ${model.sample_rate} batch_size: 24 shuffle: true num_workers: 16 pin_memory: true use_start_end_token: false trim_silence: false max_duration: 30 min_duration: 0.1 is_tarred: false tarred_audio_filepaths: null shuffle_n: 2048 bucketing_strategy: synced_randomized bucketing_batch_size: 8 validation_ds: manifest_filepath: /content/drive/MyDrive/drive-download-20240405T032246Z-001/validated.json sample_rate: ${model.sample_rate} batch_size: 1 max_duration: 30 shuffle: false num_workers: 8 pin_memory: true use_start_end_token: false test_ds: manifest_filepath: /content/drive/MyDrive/drive-download-20240405T032246Z-001/test.json sample_rate: ${model.sample_rate} batch_size: 4 shuffle: false num_workers: 8 pin_memory: true use_start_end_token: false tokenizer: dir: /content/drive/MyDrive/1T type: bpe preprocessor: target: nemo.collections.asr.modules.AudioToMelSpectrogramPreprocessor sample_rate: ${model.sample_rate} normalize: per_feature window_size: 0.025 window_stride: 0.01 window: hann features: 80 n_fft: 512 log: true frame_splicing: 1 dither: 1.0e-05 pad_to: 0 pad_value: 0.0 spec_augment: target: nemo.collections.asr.modules.SpectrogramAugmentation freq_masks: 2 time_masks: 5 freq_width: 27 time_width: 0.05 encoder: target: nemo.collections.asr.modules.ConformerEncoder feat_in: ${model.preprocessor.features} feat_out: -1 n_layers: 17 d_model: 512 subsampling: striding subsampling_factor: 4 subsampling_conv_channels: -1 ff_expansion_factor: 4 self_attention_model: rel_pos n_heads: 8 att_context_size:

• -1
• -1 xscaling: true untie_biases: true pos_emb_max_len: 5000 conv_kernel_size: 31 conv_norm_type: batch_norm dropout: 0.1 dropout_emb: 0.0 dropout_att: 0.1 decoder: target: nemo.collections.asr.modules.ConvASRDecoder feat_in: null num_classes: -1 vocabulary: [] optim: name: adamw lr: 0.5 betas:
• 0.9
• 0.98 weight_decay: 0.001 sched: name: NoamAnnealing d_model: ${model.encoder.d_model} warmup_steps: 10000 warmup_ratio: null min_lr: 1.0e-06 trainer: devices: -1 num_nodes: 1 max_epochs: 30 max_steps: null val_check_interval: 1.0 accelerator: gpu strategy: ddp accumulate_grad_batches: 1 gradient_clip_val: 0.0 precision: 32 log_every_n_steps: 100 num_sanity_val_steps: 0 check_val_every_n_epoch: 1 sync_batchnorm: true enable_checkpointing: false logger: false benchmark: false exp_manager: exp_dir: null name: ${name} create_tensorboard_logger: true create_checkpoint_callback: true checkpoint_callback_params: monitor: val_wer mode: min save_top_k: 10 always_save_nemo: true resume_if_exists: false resume_ignore_no_checkpoint: false create_wandb_logger: true wandb_logger_kwargs: name: conformer_large_filtered_english project: nemo_conformer init_from_pretrained_model: /root/ekstep/nemo_exp/vakyansh-nemo-experimentation/scripts/finetuning/nemo_experiments/Conformer-CTC-BPE-Large/2022-06-01_19-16-48/checkpoints/Conformer-CTC-BPE-Large.nemo

Error executing job with overrides: ['model.train_ds.manifest_filepath=/content/drive/MyDrive/drive-download-20240405T032246Z-001/train.json', 'model.validation_ds.manifest_filepath=/content/drive/MyDrive/drive-download-20240405T032246Z-001/validated.json', 'model.test_ds.manifest_filepath=/content/drive/MyDrive/drive-download-20240405T032246Z-001/test.json', 'model.tokenizer.dir=/content/drive/MyDrive/1T', 'model.tokenizer.type=bpe', 'trainer.devices=-1', 'trainer.accelerator=gpu', 'trainer.strategy=ddp', 'exp_manager.create_wandb_logger=True', 'exp_manager.wandb_logger_kwargs.name=conformer_large_filtered_english', 'exp_manager.wandb_logger_kwargs.project=nemo_conformer', '+init_from_pretrained_model=/root/ekstep/nemo_exp/vakyansh-nemo-experimentation/scripts/finetuning/nemo_experiments/Conformer-CTC-BPE-Large/2022-06-01_19-16-48/checkpoints/Conformer-CTC-BPE-Large.nemo'] Traceback (most recent call last): File "/content/vakyansh-nemo-experimentation/scripts/finetuning/../../src/finetuning/speech_to_text_ctc_bpe.py", line 101, in main trainer = pl.Trainer(cfg.trainer) File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/utilities/argparse.py", line 70, in insert_env_defaults return fn(self, kwargs) File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 418, in init self.fit_loop.epoch_loop = _TrainingEpochLoop(self, min_steps=min_steps, max_steps=max_steps) File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/training_epoch_loop.py", line 59, in init if max_steps < -1: TypeError: '<' not supported between instances of 'NoneType' and 'int'

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

[nitinyadav250]

please somebody help to get this error @titu1994 sir help

[titu1994]

In your config use trainer.max_steps = -1 rather than None

[nitinyadav250]

thanks sir 🙌