search

mitchellnw / micro-net-dnw

Submission for the Neurips 2019 Micro-net Challange
3 stars 2 forks source link

Which model do you plan to submit? #1

Open micronet-challenge-submissions opened 5 years ago

micronet-challenge-submissions commented 5 years ago

Hello! Thanks so much for your entry! I didn't seem to have your email so I'm writing here.

There are a number of ImageNet models listed in the table of your README. Is there one in particular that you're entering? If I'm not mistaken, it also looks like they're below the ImageNet accuracy target of 75% top-1 accuracy.

Trevor

mitchellnw commented 5 years ago

Hello! Thanks for setting up this competition.

Sorry, I didn't realize that there was 75% top-1 cutoff. We have another model that gets exactly 75% that we could upload if it's not too late?

The model uses basically the same code but uses the structure of ResNet50 instead of MobileNetV1 so it is much more accurate.

Feel free to respond here, I can also be reached at mitchellwortsman@gmail.com.

Mitchell

micronet-challenge-submissions commented 5 years ago

We're still accepting submissions until the 11th! You're welcome to upload the better model. Do note that 75% is a hard cutoff - we don't round up.

In addition to the model and description of the approach, we ask that entries calculate their score according to the rules published on our website. If you have any question about scoring, feel free to email us. We'll also review the scoring after submissions have closed and are happy to take a look if you share it with us prior.

Trevor

mitchellnw commented 5 years ago

Hi @micronet-challenge-submissions

I've updated the submission which computes the score as needed and gives a model which gets over the required 75 on Imagenet.

Thank you, Mitchell

micronet-challenge-submissions commented 5 years ago

This looks excellent! Very impressed with your sparsity numbers.

FWIW, it looks like you've actually upper bounded your score. The competition takes quantization into account, but to avoid forcing all entries to quantize we allow entries that do not perform quantization to take a "freebie" quantization discount. This entails counting your parameters as if they were 16-bits (1/2 of a parameter by our system), and counting all multiplications as if they were performed in 16-bits (additions are assumed to be 32-bits to maintain model quality). Applying these discounts to your score...

(2,558,732 / 6,900,000) / 2 + (772,846,965/ 1,170,000,000) = 0.8460

Also, we're currently posting self-reported scores on our leaderboard until after the submission deadline passes and we're able to officially review the entries. Would you like us to post your score or wait until we're able to officially review it? If you'd like us to post it, what name would you like it under?

Thanks! Trevor

mitchellnw commented 5 years ago

Thank you very much! And thanks so much for setting up this competition.

I think we would like to wait until you officially review it.

Best, Mitchell

micronet-challenge-submissions commented 5 years ago

Sounds good. Thanks!

Trevor

micronet-challenge-submissions commented 5 years ago

Hello! We're planning to release the results early next week. What name would you like your entry listed under? Thanks!

Trevor

mitchellnw commented 5 years ago

Hi Trevor!

If it's not too long, I guess the name "Discovering Neural Wirings Team (AI2, University of Washington, XNOR.AI)" is good!

micronet-challenge-submissions commented 5 years ago

It is a bit long - for formatting reasons, do you mind if we use "Discovering Neural Wirings Team"?

Thanks! Trevor

On Mon, 4 Nov 2019 at 13:18, mitchellnw notifications@github.com wrote:

Hi Trevor!

If it's not too long, I guess the name "Discovering Neural Wirings Team (AI2, University of Washington, XNOR.AI)" is good!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mitchellnw/micro-net-dnw/issues/1?email_source=notifications&email_token=AMILA6Z7FQNEKL6E77BUVILQSCGTPA5CNFSM4I4RWWYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDAX3SI#issuecomment-549551561, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMILA63JDPQXBUWYZEF5XVDQSCGTPANCNFSM4I4RWWYA .

mitchellnw commented 5 years ago

Sure, sounds good!

micronet-challenge-submissions commented 5 years ago

Thanks! Trevor

On Wed, 6 Nov 2019 at 00:51, mitchellnw notifications@github.com wrote:

Sure, sounds good!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mitchellnw/micro-net-dnw/issues/1?email_source=notifications&email_token=AMILA6ZQHGFDGZZAKGVPW3TQSKAQRA5CNFSM4I4RWWYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDFYMTY#issuecomment-550209103, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMILA6YXWJNRH62EIA6ZMHLQSKAQRANCNFSM4I4RWWYA .