Closed maaft closed 4 years ago
Hi @maaft !
Feature maps of 12x12
or 24x24
are very small. In Cityscapes, a feature map of 1/32
scale have a size of 32x64
. Fluctuations in these feature maps have a small impact on the overall latency of the whole network (1000/163.9 = 6.1ms in our network v.s. ~0.05ms in your case).
As demonstrated in our Figure 3, our latency estimation achieves a very high correlation with real latency. This correlation is verified by sampling 1,000 subnetworks.
Okay, this of course makes latency-based NAS very hard for applications with smaller images.
Until I find a better solution, I will just multiply the height and width during latency measurement with a constant factor (8 in my case) which will result in stable results but will not reflect the real-latency. But at least the relative latencies between different operations are at least correctly estimated.
I noticed that the measured latencies differ substantially from run to run. For example I get following results at different program runs:
Note that in the last run the operation on quarter resolution (12x12) took longer than the operation on the base resolution (24x24).
This will substantially influence the network-architecture search as the optimizer will prefer pathes on higher resolutions.
How did you make sure that your measured latencies "make sense"? Did you set static GPU clocks or similar to get stable results?