search

boostcampaitech3 / level1-image-classification-level1-nlp-10

0 stars 4 forks source link

성능 개선을 위한 Profiling #11

Closed hyoeun98 closed 2 years ago

hyoeun98 commented 2 years ago

torch.profile - https://pytorch.org/docs/master/profiler.html torch.profile tutorial - https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html snakeviz - https://jiffyclub.github.io/snakeviz/ https://tigress-web.princeton.edu/~jdh4/how_to_profile_with_dlprof_may_2021.pdf https://on-demand.gputechconf.com/ai-conference-2019/skr9110.pdf

hyoeun98 commented 2 years ago

image baseline train.py

greenare commented 2 years ago

이거 조사할 때 어떤 툴 사용하셨나요? kcachegrind 생각나네요

hyoeun98 commented 2 years ago

해당 이미지는 snakeviz를 사용했습니다.

hyoeun98 commented 2 years ago 
pip install snakeviz
python -m cProfile -o 100_percent_gpu_utilization.prof train.py --hyperparameter set
------------ training ------------------------------
python -m snakeviz 100_percent_gpu_utilization.prof --server

127.0.0.1:8080 접속 학습 중 bottleneck 부분이 어딘지 확인해 보시면 좋을 것 같습니다.