iperov / DeepFaceLab

DeepFaceLab is the leading software for creating deepfakes.
GNU General Public License v3.0
16.37k stars 28 forks source link

Training speed #5732

Open gauravlokha opened 1 year ago

gauravlokha commented 1 year ago

Using a gtx 1650 , I know not much at all but it's just for learning. So usually I leave the training overnight. Usually I get around 350ms give or take and get around 2-3 iterations per second. Now I am not using my pc since it's training but I get readings of 2900ms. But it's taking 2-3 seconds for each iteration. Can someone explain to me if I'm not getting something right?

update - when I started using my pc to post this, the training has gone back to 350ms and getting usual 2-3 iter per second.

== == == Current iteration: 0 == == == ==-------------- Model Options ---------------== == == == batch_size: 2 == == == ==---------------- Running On ----------------== == == == Device index: 0 == == Name: NVIDIA GeForce GTX 1650 == == VRAM: 2.85GB == == ==

Starting. Press "Enter" to stop training and save model.

Trying to do the first iteration. If an error occurs, reduce the model parameters.

!!! Windows 10 users IMPORTANT notice. You should set this setting in order to work correctly. https://i.imgur.com/B7cmDCB.jpg !!! [02:13:28][#000002][0394ms][3.1017][3.3906] [02:38:14][#003961][0365ms][0.6420][0.5605] [03:03:14][#007982][0387ms][0.3743][0.3337] [03:28:14][#011995][0378ms][0.3142][0.2822] [03:53:14][#016011][0398ms][0.2847][0.2543] [04:18:14][#020029][0374ms][0.2622][0.2386] [04:43:14][#024026][0388ms][0.2474][0.2247] [05:08:14][#028041][0405ms][0.2342][0.2157] [05:33:14][#032056][0418ms][0.2255][0.2064] [05:58:14][#036077][0403ms][0.2181][0.2011] [06:23:14][#040095][0408ms][0.2109][0.1937] [06:48:14][#044110][0383ms][0.2048][0.1901] [07:13:14][#048125][0390ms][0.2010][0.1845] [07:38:14][#052142][0405ms][0.1951][0.1816] [08:03:14][#056151][0381ms][0.1910][0.1780] [08:28:14][#060162][0384ms][0.1881][0.1751] [08:53:14][#064181][0403ms][0.1851][0.1714] [09:18:14][#068195][0366ms][0.1813][0.1682] [09:43:14][#072208][0388ms][0.1792][0.1676] [10:08:14][#076231][0386ms][0.1747][0.1653] [10:33:14][#080247][0400ms][0.1739][0.1626] [11:20:13][#082719][2986ms][0.1713][0.1590] [11:23:17][#082798][2602ms][0.1662][0.1685] [11:48:16][#083462][1489ms][0.1707][0.1607] [12:13:00][#085904][0360ms][0.1333][0.1455]