twosixlabs / armory

ARMORY Adversarial Robustness Evaluation Test Bed
MIT License
176 stars 67 forks source link

CARLA overhead object detection test data #1966

Closed yusong-tan closed 1 year ago

swsuggs commented 1 year ago

@jprokos26 could you double check that you can get the new dataset when you run a config? This version has two splits "test_hallucination" and "test_disappearance" so just set the config accordingly:

"dataset": {
        . . .
        "eval_split": "test_hallucination",
        "name": "carla_over_obj_det_test"
        . . .
swsuggs commented 1 year ago

I will add baseline results to PR #1959

jprokos26 commented 1 year ago

@jprokos26 could you double check that you can get the new dataset when you run a config? This version has two splits "test_hallucination" and "test_disappearance" so just set the config accordingly:

"dataset": {
        . . .
        "eval_split": "test_hallucination",
        "name": "carla_over_obj_det_test"
        . . .

Tested working with both splits

test_hallucination

Screenshot 2023-07-26 at 10 55 02 AM

test_disappearance

Screenshot 2023-07-26 at 10 54 42 AM