weimingboya / DFT

10 stars 0 forks source link

"Error when running model: TypeError: 'generator' object is not callable" #9

Open thalapandi opened 7 months ago

thalapandi commented 7 months ago

/usr/local/lib/python3.10/dist-packages/torch/optim/lr_scheduler.py:143: UserWarning: Detected call of lr_scheduler.step() before optimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step() before lr_scheduler.step(). Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate warnings.warn("Detected call of lr_scheduler.step() before optimizer.step(). " Training starts Epoch 0 - train: 0%| | 0/35403 [00:00<?, ?it/s]

TypeError Traceback (most recent call last) in <cell line: 127>() 127 for e in range(start_epoch, start_epoch + 100): 128 if not use_rl: --> 129 train_loss = train_xe(model, dataloader_train, optim, loss_fn, text_field, e, device, scheduler, args) 130 writer.add_scalar('data/train_loss', train_loss, e) 131 else:

3 frames /content/common/utils/utils.py in train_xe(model, dataloader, optim, loss_fn, text_field, epoch, device, scheduler, args) 156 running_loss = .0 157 with tqdm(desc='Epoch %d - train' % epoch, unit='it', total=len(dataloader)) as pbar: --> 158 for it, (images, captions) in enumerate(dataloader): 159 # if it == 10: 160 # break

/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py in next(self) 629 # TODO(https://github.com/pytorch/pytorch/issues/76750) 630 self._reset() # type: ignore[call-arg] --> 631 data = self._next_data() 632 self._num_yielded += 1 633 if self._dataset_kind == _DatasetKind.Iterable and \

/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py in _next_data(self) 673 def _next_data(self): 674 index = self._next_index() # may raise StopIteration --> 675 data = self._dataset_fetcher.fetch(index) # may raise StopIteration 676 if self._pin_memory: 677 data = _utils.pin_memory.pin_memory(data, self._pin_memory_device)

/usr/local/lib/python3.10/dist-packages/torch/utils/data/_utils/fetch.py in fetch(self, possibly_batched_index) 47 if self.auto_collation: 48 if hasattr(self.dataset, "getitems") and self.dataset.getitems: ---> 49 data = self.dataset.getitems(possibly_batched_index) 50 else: 51 data = [self.dataset[idx] for idx in possibly_batched_index]

TypeError: 'generator' object is not callable

weimingboya commented 7 months ago

Please check the pytorch version, my environment is fine in 1.8.1.

thalapandi commented 7 months ago

yes I solved the issue and also model has started training and validation part is completed but when we try to get score of validation for actual and generated caption then getting like this error

a man standing in a kitchen with a white dog a teddy bear sitting next to a teddy bear path_to_jar_dirname: D:\Varsha\DFT_PROJECT_PHASE_II\common\evaluation\tokenizer gen1: {'0_0': ['']}

OSError Traceback (most recent call last) ~\AppData\Local\Temp\ipykernel_13592\2201875235.py in 142 143 # Validation scores --> 144 scores = evaluate_metrics(model, dict_dataloader_val, text_field, e, device, args) 145 print("Validation scores", scores) 146 val_cider = scores['CIDEr']

~\AppData\Local\Temp\ipykernel_13592\343776701.py in evaluate_metrics(model, dataloader, textfield, epoch, device, args) 133 gen = evaluation.PTBTokenizer.tokenize(gen) 134 print("gen1:",gen) --> 135 scores, = evaluation.compute_scores(gts, gen) 136 return scores 137

D:\Varsha\DFT_PROJECT_PHASE_II\common\evaluation__init__.py in compute_scores(gts, gen) 13 all_scores = {} 14 for metric in metrics: ---> 15 score, scores = metric.compute_score(gts, gen) 16 all_score[str(metric)] = score 17 all_scores[str(metric)] = scores

D:\Varsha\DFT_PROJECT_PHASE_II\common\evaluation\meteor\meteor.py in compute_score(self, gts, res) 47 48 self.meteor_p.stdin.write('{}\n'.format(eval_line).encode()) ---> 49 self.meteor_p.stdin.flush() 50 for i in range(0,len(imgIds)): 51 scores.append(float(self.meteor_p.stdout.readline().strip()))

OSError: [Errno 22] Invalid argument

weimingboya commented 7 months ago

Maybe one of the metrics is causing the error? Please check which one. Thx

thalapandi commented 7 months ago

Epoch 0 - train: 18%|████████▍ | 5000/28322 [1:59:27<9:17:11, 1.43s/it, loss=4] Epoch 0 - validation: 100%|█████████████████████████████████████████████| 1250/1250 [07:15<00:00, 2.87it/s, loss=3.31] Epoch 0 - evaluation: 100%|██████████████████████████████████████████████████████████| 250/250 [06:57<00:00, 1.67s/it] gen: {'0_0': ['a group of people standing in a field'], '0_1': ['a kitchen with a kitchen and a kitchen'], '0_2': ['a woman holding a dog with a dog'], '0_3': ['a bathroom with a toilet and a toilet'], '0_4': ['a bathroom with a sink and a sink'], '0_5': ['a woman riding a motorcycle on a street'], '0_6': ['a bathroom with a bathroom and a bathroom']} Calling tokenized function image_id: ['0_0', '0_0', '0_0', '0_0', '0_0', '0_1', '0_1', '0_1', '0_1'] sentences: A child holding a flowered umbrella and petting a yak. A young man holding an umbrella next to a herd of cattle. a young boy barefoot holding an umbrella touching the horn of a cow A young boy with an umbrella who is touching the horn of a cow. A boy holding an umbrella while standing next to livestock. A narrow kitchen filled with appliances and cooking utensils. A galley kitchen with cabinets and appliances on both sides path_to_jar_dirname: D:\Varsha\DFT_PROJECT_PHASE_II\common\evaluation\tokenizer gen1: {'0_0': ['']}

not sure about this why it is occurring on gen1: {'0_0': ['']}

can you tell us how to resolve the issue?

D:\Varsha\DFT_PROJECT_PHASE_II\common\evaluation\meteor\meteor.py in compute_score(self, gts, res) 47 48 self.meteor_p.stdin.write('{}\n'.format(eval_line).encode()) ---> 49 self.meteor_p.stdin.flush() 50 for i in range(0,len(imgIds)): 51 scores.append(float(self.meteor_p.stdout.readline().strip()))

OSError: [Errno 22] Invalid argument