More Tutorial Bugs: Training Error in Object Count Auto-Optimization Tutorial

[J-Fo-S]

Describe the bug Training fails on a TypeError: cannot unpack non-iterable float object. See below for details. Occurs in both local and hosted runtimes.

To Reproduce Steps to reproduce the behavior:

1. Go to https://colab.research.google.com/github/SylphAI-Inc/AdalFlow/blob/main/notebooks/qas/adalflow_object_count_auto_optimization.ipynb
2. Run through https://colab.research.google.com/github/SylphAI-Inc/AdalFlow/blob/main/notebooks/qas/adalflow_object_count_auto_optimization.ipynb#scrollTo=eqQSFnZOpfWJ

Screenshots

TypeError                                 Traceback (most recent call last)
Cell In[17], line 1
----> 1 train(debug=False, max_steps=12, strategy="constrained",
      2       raw_shots=0, bootstrap_shots=1,
      3       exclude_input_fields_from_bootstrap_demos=True
      4       )

Cell In[16], line 36, in train(train_batch_size, raw_shots, bootstrap_shots, max_steps, num_workers, strategy, optimization_order, debug, resume_from_ckpt, exclude_input_fields_from_bootstrap_demos)
     33 print(trainer)
     35 train_dataset, val_dataset, test_dataset = load_datasets()
---> 36 trainer.fit(
     37     train_dataset=train_dataset,
     38     val_dataset=val_dataset,
     39     test_dataset=test_dataset,
     40     debug=debug,
     41     resume_from_ckpt=resume_from_ckpt,
     42 )

File ~\Anaconda3\envs\ada\Lib\site-packages\adalflow\optim\trainer\trainer.py:479, in Trainer.fit(self, adaltask, train_loader, train_dataset, val_dataset, test_dataset, debug, save_traces, raw_shots, bootstrap_shots, resume_from_ckpt)
    477     starting_step += self.max_steps
    478 elif self.strategy == "constrained":
--> 479     trainer_results = self._fit_text_grad_constraint(
    480         train_loader,
    481         val_dataset,
    482         test_dataset,
    483         trainer_results=trainer_results,
    484         starting_step=starting_step,
    485     )
    486     starting_step += self.max_steps
    487 else:

File ~\Anaconda3\envs\ada\Lib\site-packages\adalflow\optim\trainer\trainer.py:1746, in Trainer._fit_text_grad_constraint(self, train_loader, val_dataset, test_dataset, trainer_results, starting_step)
   1742 from adalflow.optim.parameter import Parameter
   1744 log.info("Fitting using Textual Gradient Descent with constraints")
   1745 trainer_results = (
-> 1746     self._pre_fit(val_dataset, test_dataset)
   1747     if trainer_results is None
   1748     else trainer_results
   1749 )
   1751 print(f"save to {self.ckpt_file}")
   1753 self.adaltask.train()

File ~\Anaconda3\envs\ada\Lib\site-packages\adalflow\optim\trainer\trainer.py:619, in Trainer._pre_fit(self, val_dataset, test_dataset)
    615 def _pre_fit(self, val_dataset: Any, test_dataset: Any) -> TrainerResult:
    616     # validate first (separate into another function where we can even save the outputs so that we can highlight error predictions)
    618     trainer_state = self.gather_trainer_states()
--> 619     trainer_results: TrainerResult = self.initial_validation(
    620         val_dataset, test_dataset
    621     )
    622     self._add_history_text_optimizers(trainer_results.val_scores[-1])
    623     trainer_results.trainer_state = trainer_state

File ~\Anaconda3\envs\ada\Lib\site-packages\adalflow\optim\trainer\trainer.py:511, in Trainer.initial_validation(self, val_dataset, test_dataset)
    509 def initial_validation(self, val_dataset: Any, test_dataset: Any):
--> 511     val_output = self.adaltask.validation_step(val_dataset, 0, self.num_workers)
    512     val_score = val_output.avg_score
    513     test_score = None

File ~\Anaconda3\envs\ada\Lib\site-packages\adalflow\optim\trainer\adal.py:472, in AdalComponent.validation_step(self, batch, batch_idx, num_workers, minimum_score)
    470 # TODO: let use decide which mode to be
    471 self.task.eval()
--> 472 completed_y_preds, completed_samples, index_to_score = self.pred_step(
    473     batch, batch_idx, num_workers, running_eval=True, min_score=minimum_score
    474 )
    475 if index_to_score:
    476     # compute score from index_to_score
    477     print(
    478         f"completed_samples: {len(completed_samples)}, len: {len(list(index_to_score.values()))}"
    479     )

File ~\Anaconda3\envs\ada\Lib\site-packages\adalflow\optim\trainer\adal.py:405, in AdalComponent.pred_step(self, batch, batch_idx, num_workers, running_eval, min_score)
    401 completed_indices.add(i)  # Mark this index as completed
    403 if running_eval and not isinstance(y_pred, Parameter):
    404     # evaluate one sample
--> 405     eval_fn, kwargs = self.prepare_eval(sample, y_pred)
    406     score = eval_fn(**kwargs)
    407     index_to_score[i] = score

TypeError: cannot unpack non-iterable float object

**Desktop (please complete the following information):** - OS: Windows - Colab runtime: local connection **Smartphone (please complete the following information):** - Device: [e.g. iPhone6] - OS: [e.g. iOS8.1] - Browser [e.g. stock browser, safari] - Version [e.g. 22] **Additional context** Add any other context about the problem here.

[liyin2015]

@J-Fo-S Thanks for reporting.

This is due to a legacy api. Updating the code to

    def prepare_eval(
        self, sample: Example, y_pred: adal.GeneratorOutput
    ) -> float:
        y_label = -1
        if y_pred is not None and y_pred.data is not None:
            y_label = y_pred.data
        return self.eval_fn, {"y": y_label, "y_gt": sample.answer}

Will work fine. I will create a pr to update the notebook

[liyin2015]

https://github.com/SylphAI-Inc/AdalFlow/pull/255

[liyin2015]

close