Upload ScalePostprocessors

[kai422]

Upload ScalePostprocessors for [ICLR2024] Scaling for Training Time and Post-hoc Out-of-Distribution Detection Enhancement.

Original Codebase: https://github.com/kai422/SCALE. ICLR page: https://openreview.net/forum?id=RDSTjtnqCg.

[zjysteven]

As mentioned in the issue, we will make the merge once the training code is available. BTW, You can also update the README to include a link of your method.

[kai422]

The training code has been uploaded.

Regards, Kai

[zjysteven]

Looks pretty good. Thank you for making the contribution.

[jS5t3r]

Thanks for the contribution :)

I encounter a but when running this script:

bash scripts/ood/scale/cifar10_test_ood_scale.sh

containing:

PYTHONPATH='.':$PYTHONPATH \
# srun -p dsta --mpi=pmi2 --gres=gpu:${GPU} -n1 \
# --cpus-per-task=${CPU} --ntasks-per-node=${GPU} \
# --kill-on-bad-exit=1 --job-name=${jobname} -w SG-IDC1-10-51-2-${node} \

python main.py \
    --config configs/datasets/cifar10/cifar10.yml \
    configs/datasets/cifar10/cifar10_ood.yml \
    configs/networks/resnet18_32x32.yml \
    configs/pipelines/test/test_ood.yml \
    configs/preprocessors/base_preprocessor.yml \
    configs/postprocessors/scale.yml \
    --num_workers 8 \
    --network.checkpoint 'results/cifar10_resnet18_32x32_base_e100_lr0.1_default/s0/best.ckpt' \
    --mark 1

I want to use the main.py instead of eval_ood.py.

Error Message:

Output dir: ./results/cifar10_resnet18_32x32_test_ood_ood_scale_1/s0/ood
Exp dir already exists, merge it? (y/n)y
Output directory path: ./results/cifar10_resnet18_32x32_test_ood_ood_scale_1/s0/ood
Model Loading resnet18_32x32 Completed!
Traceback (most recent call last):
  File "main.py", line 36, in <module>
    launch(
  File "/home/user/OpenOOD/openood/utils/launch.py", line 84, in launch
    main_func(*args)
  File "main.py", line 24, in main
    pipeline.run()
  File "/home/user/OpenOOD/openood/pipelines/test_ood_pipeline.py", line 29, in run
    postprocessor = get_postprocessor(self.config)
  File "/home/user/OpenOOD/openood/postprocessors/utils.py", line 89, in get_postprocessor
    return postprocessors[config.postprocessor.name](config)
KeyError: 'scale'
Postprocessor_name is ignored because postprocessor is passed
TestStandardPreProcessor
TestStandardPreProcessor
Loaded pre-computed scores from file.
Processing near ood...
Inference has been performed on cifar100 dataset...
Computing metrics on cifar100 dataset...
FPR@95: 86.09, AUROC: 80.57 AUPR_IN: 73.52, AUPR_OUT: 83.20

[zjysteven]

@jS5t3r Did my answer to your question in the issue solve the problem?

[jS5t3r]

@zjysteven Yes.

In the result folder, there is no ood.csv showing up. How to fix this?