Not found the script for testing in examples/*

[fu123456]

In "examples/*", LibMTL provides nyu, office, qm9, and xtreme, but I did not find any testing script files. With the trained models, we want to use the testing script file to output the quantitative and visual results.

[Baijiong-Lin]

The testing is called after each training epoch. https://github.com/median-research-group/LibMTL/blob/afea38bb185a410b69c703905bbe2bd5c3430647/LibMTL/trainer.py#L244

If you want to test after training, you can simply replace https://github.com/median-research-group/LibMTL/blob/afea38bb185a410b69c703905bbe2bd5c3430647/examples/nyu/train_nyu.py#L100

with NYUmodel.test(nyuv2_test_loader)

[fu123456]

Very thanks. But also with some errors:

========================================
General Configuration:
        Wighting: EW
        Architecture: HPS
        Rep_Grad: False
        Multi_Input: False
        Seed: 0
        Save Path: /home/fg/Projects/LibMTL/examples/office/result_stuff
        Load Path: /home/fg/Projects/LibMTL/examples/office/result_stuff/best.pt
        Device: cuda:0
Optimizer Configuration:
        optim: adam
        lr: 0.0001
        weight_decay: 1e-05
Load Model from - /home/fg/Projects/LibMTL/examples/office/result_stuff/best.pt
========================================
Total Params: 11487901
Trainable Params: 11487901
Non-trainable Params: 0
========================================
LOG FORMAT | amazon_LOSS Acc | dslr_LOSS Acc | webcam_LOSS Acc | TIME
Traceback (most recent call last):
  File "/home/fg/Projects/LibMTL/LibMTL/trainer.py", line 141, in _process_data
    data, label = next(loader[1]) # by Jack
ValueError: too many values to unpack (expected 2)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test_office.py", line 97, in <module>
    main(params)
  File "test_office.py", line 88, in main
    officeModel.test(test_dataloaders=test_dataloaders, epoch=params.load_path, mode='test')
  File "/home/fg/Projects/LibMTL/LibMTL/trainer.py", line 274, in test
    test_inputs, test_gts = self._process_data(test_loader)
  File "/home/fg/Projects/LibMTL/LibMTL/trainer.py", line 144, in _process_data
    data, label = next(loader[1]) # by Jack
ValueError: too many values to unpack (expected 2)

I have finished training by the following command line

python train_office.py --weighting MGDA --arch HPS --dataset_path /home/fg/Projects/LibMTL/datasets/domain_adaptation_images --gpu_id 0 --save_path /home/fg/Projects/LibMTL/examples/office/result_stuff --epochs 60 --multi_input

Then, I tried to test using the following command line

python test_office.py --load_path /home/fg/Projects/LibMTL/examples/office/result_stuff/best.pt --save_path /home/fg/Projects/LibMTL/examples/office/result_stuff --dataset_path /home/fg/Projects/LibMTL/datasets/domain_adaptation_images --gpu_id 0 

In addition, I also change "officeModel.train(... ...)" in train_office.py to "officeModel.test(test_dataloaders=test_dataloaders, epoch=params.load_path, mode='test')".

[Baijiong-Lin]

using --multi_input

[fu123456]

run the following command line for testing:

python test_office.py --dataset office-31 --load_path /home/fg/Projects/LibMTL/examples/office/result_stuff/best.pt --save_path /home/fg/Projects/LibMTL/examples/office/result_stuff --dataset_path /home/fg/Projects/LibMTL/datasets/domain_adaptation_images --gpu_id 0 --multi_input

But with errors:

========================================
General Configuration:
        Wighting: EW
        Architecture: HPS
        Rep_Grad: False
        Multi_Input: True
        Seed: 0
        Save Path: /home/fg/Projects/LibMTL/examples/office/result_stuff
        Load Path: /home/fg/Projects/LibMTL/examples/office/result_stuff/best.pt
        Device: cuda:0
Optimizer Configuration:
        optim: adam
        lr: 0.0001
        weight_decay: 1e-05
Load Model from - /home/fg/Projects/LibMTL/examples/office/result_stuff/best.pt
========================================
Total Params: 11487901
Trainable Params: 11487901
Non-trainable Params: 0
========================================
LOG FORMAT | amazon_LOSS Acc | dslr_LOSS Acc | webcam_LOSS Acc | TIME
Traceback (most recent call last):
  File "test_office.py", line 97, in <module>
    main(params)
  File "test_office.py", line 88, in main
    officeModel.test(test_dataloaders=test_dataloaders, epoch=params.load_path, mode='test')
  File "/home/fg/Projects/LibMTL/LibMTL/trainer.py", line 290, in test
    self.meter.display(epoch=epoch, mode=mode)
  File "/home/fg/Projects/LibMTL/LibMTL/_record.py", line 70, in display
    self._update_best_result(self.results, epoch)
  File "/home/fg/Projects/LibMTL/LibMTL/_record.py", line 105, in _update_best_result
    improvement = count_improvement(self.base_result, new_result, self.weight)
  File "/home/fg/Projects/LibMTL/LibMTL/utils.py", line 73, in count_improvement
    for task in list(base_result.keys()):
AttributeError: 'NoneType' object has no attribute 'keys'

[Baijiong-Lin]

officeModel.test(test_dataloaders=test_dataloaders)

[Baijiong-Lin]

Closed as no further updates.