search

kairos03 / ProxylessNAS-Pytorch

ProxylessNAS-Pytorch
23 stars 5 forks source link

Implement Latency Loss #2

Open xieydd opened 5 years ago

xieydd commented 5 years ago

@kairos03 Now I have full time on it , and i notice the LatencyLoss Class already implement a part of Latencyloss. I have some point confused?

kairos03 commented 5 years ago
  1. latency of every op. use https://github.com/kairos03/ProxylessNAS-Pytorch/blob/master/proxylessnas/latency.csv file. The file is official latency result in https://hanlab.mit.edu/files/proxylessNAS/LatencyTools/.
  2. no, I think we add new loss and λ in below code in train_search.py. https://github.com/kairos03/ProxylessNAS-Pytorch/blob/c87b233aaffb9e38329cbb7d4fc5f5398b1312a8/proxylessnas/train_search.py#L84
  3. no, You dont need that, just calculate loss. thx.
xieydd commented 5 years ago

Thank you for your prompt reply @kairos03 From your answer, i got some point:

  1. The latency of every op is fixed.
  2. Add the weight decay term and LatencyLoss to CrossEntropyLoss
  3. I mean the Latency loss can be differentiable, we will not make it differentiable manual.

Also Confused:

  1. This Latency implement is not complete? https://github.com/kairos03/ProxylessNAS-Pytorch/blob/master/proxylessnas/latencyloss.py#L70
kairos03 commented 5 years ago

Yes, not completed. You can change whole class.

xieydd commented 5 years ago

Thx

xieydd commented 5 years ago

what is the difference of pixel _trim and mobile_trim? @kairos03

kairos03 commented 5 years ago

I dont know that.. So, I Just choose mobile_trim. @xieydd

xieydd commented 5 years ago

@kairos03 I have writed one version of latency loss, and i am training(WIP) the model. But it is very slow, it is normal?

kairos03 commented 5 years ago

yes training is very slow

xieydd commented 5 years ago

@kairos03 One Question: Darts need Augment, and the Cell`s node number is fixed, and the Graph is stacked by the Cell. And ProxylessNAS no need Augment, What are your thoughts on this？

kairos03 commented 5 years ago

I didn't under stand your question. you mean why ProxylessNAS are not using stacking the Cells?

xieydd commented 5 years ago

@kairos03 Now, i am clear; Now the problem is accuracy is very low and train very slowly.