search

RobustBench / robustbench

RobustBench: a standardized adversarial robustness benchmark [NeurIPS 2021 Benchmarks and Datasets Track]
https://robustbench.github.io
Other
669 stars 99 forks source link

Can we improve the running speed by modifying the batch size? #186

Closed tuningManBin closed 4 months ago

tuningManBin commented 4 months ago

Hi, thank you for providing the powerful testing tool. In my project, I found that the running speed of robustness testing using RobustBench clean_acc, robust_acc = benchmark(model, model_name=model_name, n_examples=10000, dataset="cifar10", threat_model=threat_model, eps=8/255, device=device, to_disk=True) is very slow. Is there a corresponding hyperparameter that can adjust the batch size during the running process?

fra31 commented 4 months ago

Hi,

the benchmark function takes a batch_size argument https://github.com/RobustBench/robustbench/blob/b98426d7a0112f738333b321b1c3e3ebb302922f/robustbench/eval.py#L32

Depending on your use-case, e.g. development, you might also just use a subset of the test set (setting n_examples to a smaller value).

Hope this helps.

tuningManBin commented 4 months ago

Thank you for your response; it has been very helpful to me :).