MadryLab / datamodels

Apache License 2.0
22 stars 3 forks source link

Import main only when necessary #1

Open bencw99 opened 2 years ago

bencw99 commented 2 years ago

Before, we imported main regardless of whether the index was already complete. Only importing after checking for completion can speed up the work for an already completed index significantly when imports are expensive (e.g., importing torch).

Before: Screen Shot 2022-10-12 at 11 44 40 AM

After: Screen Shot 2022-10-12 at 11 45 27 AM

kristian-georgiev commented 1 year ago

tried it and it works great -- much faster skipping