twosixlabs / armory

ARMORY Adversarial Robustness Evaluation Test Bed
MIT License
174 stars 67 forks source link

infra activities for eval 7 kickoff #1961

Closed mwartell closed 11 months ago

mwartell commented 1 year ago
mwartell commented 1 year ago

Armory-upload currently routes submitted model weights into an evalX folder (with everything else). But armory only knows to look for them under model-weights . I think in eval6 I sometimes copied the weights into model-weights so armory could find them easily. Going forward, what do you prefer: updating armory to search deeper (might be tricky to tell it which eval folder to look in), just having people download the weights themselves, or something else?

swsuggs commented 1 year ago

Hey that was my question . . .

But what if we just have armory-upload put them in both spots, then we can delete model-weights when no longer needed?

mwartell commented 1 year ago

We are making AWS Lambda copy from eval?/CODE/email/model-weights/foo.pt to model-weights/foo.pt upon upload under #1962

swsuggs commented 11 months ago

Eval 7 is completed.