search

codertimo / BERT-pytorch

Google AI 2018 BERT pytorch implementation
Apache License 2.0
6.11k stars 1.29k forks source link

alpha-0.0.1a4 version released #14

Closed codertimo closed 5 years ago

codertimo commented 5 years ago

features

  1. Adding multi-gpu backward, forward ops with pytorch-encoding #10
  2. Adding selecting cuda device_ids
  3. Fixing requirement parsing code
daiwk commented 5 years ago

I found the imbalance of gpu usage also exists in this version:

+-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | 0 680 C ...ace/tools/python-3-tf-gpu/bin/python3.6 8743MiB | | 1 680 C ...ace/tools/python-3-tf-gpu/bin/python3.6 2439MiB | | 2 680 C ...ace/tools/python-3-tf-gpu/bin/python3.6 2439MiB | | 3 680 C ...ace/tools/python-3-tf-gpu/bin/python3.6 2439MiB | | 4 680 C ...ace/tools/python-3-tf-gpu/bin/python3.6 2439MiB | | 5 680 C ...ace/tools/python-3-tf-gpu/bin/python3.6 2439MiB | | 6 680 C ...ace/tools/python-3-tf-gpu/bin/python3.6 2439MiB | | 7 680 C ...ace/tools/python-3-tf-gpu/bin/python3.6 2439MiB | +-----------------------------------------------------------------------------+

my bert-pytorch version is as follows:

../tools/python-3-tf-gpu/bin/python3.6 ../tools/python-3-tf-gpu/bin/pip3.6 freeze| grep bert bert-pytorch==0.0.1a4