search

tensorflow / tfjs

A WebGL accelerated JavaScript library for training and deploying ML models.
https://js.tensorflow.org
Apache License 2.0
18.26k stars 1.92k forks source link

[Optimization target] The inference time of DeepLabV3's cityscapes architecture is long #6733

Open Linchenn opened 1 year ago

Linchenn commented 1 year ago

DeepLabV3 with cityscapes takes long time to execute and we need to figure out the reason and then add it.

Linchenn commented 1 year ago

Benchmarked https://tfjs-benchmarks.web.app/local-benchmark/index.html?benchmark=custom&modelUrl=https://storage.googleapis.com/tfhub-tfjs-modules/tensorflow/tfjs-model/deeplab/cityscapes/1/quantized/2/1/model.json&ImageTensorShape=1,227,500,3:

Run on Mac pro 2019 with WebGL backend, the computer crashed. Run on gLinux 16CPU with CPU backend, it takes ~14min: image Run on Windows with high performance video card on webgl backend, it takes 543ms: image

Linchenn commented 1 year ago

@qjia7 @gyagp

As discussed with @lina128 , since the cityscape 'architecture' for DeepLabV3 model takes too much time to run inference and it is behind the times, we are considering not to add this architecture to DeepLabV3 for benchmark tools. Please let me know if you have any suggestions.

(Will also delete the row for it in the spreadsheet.)

qjia7 commented 1 year ago

@Linchenn Just tried webgpu (with #6760 ) on Intel TGL, about 700ms. Agree to not add it to DeepLabV3 in benchmarks for now. But there maybe some optimization opportunities. Can we keep this issue or file a new issue to optimize this model?

Linchenn commented 1 year ago

@Linchenn Just tried webgpu (with #6760 ) on Intel TGL, about 700ms. Agree to not add it to DeepLabV3 in benchmarks for now. But there maybe some optimization opportunities. Can we keep this issue or file a new issue to optimize this model?

Sure, we could keep it and I just updated the name of this ISSUE.