search

jinny960812 / SyncTalkFace

SyncTalkFace: Talking Face Generation for Precise Lip-syncing via Audio-Lip Memory
33 stars 2 forks source link

L1: 0.0412 Sync: 0.2869 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1770 #1

Open aishoot opened 2 years ago

aishoot commented 2 years ago

Thanks your nice work! My model training failed on my own dataset. Can you share your loss curve? This is my last loss value, which seems that there is something wrong.

valid:: L1: 0.0370 Sync: 0.2680 Percep: 100.0000 Mem_recon: 0.0887 Mem_ad: 0.1452
147/394 steps:: L1: 0.0411 Sync: 0.2911 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1814
148/394 steps:: L1: 0.0411 Sync: 0.2910 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1812
149/394 steps:: L1: 0.0411 Sync: 0.2909 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1811
150/394 steps:: L1: 0.0411 Sync: 0.2922 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1811
151/394 steps:: L1: 0.0411 Sync: 0.2915 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1812
152/394 steps:: L1: 0.0410 Sync: 0.2917 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1810
153/394 steps:: L1: 0.0410 Sync: 0.2916 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1810
154/394 steps:: L1: 0.0410 Sync: 0.2919 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1810
155/394 steps:: L1: 0.0410 Sync: 0.2916 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1808
156/394 steps:: L1: 0.0409 Sync: 0.2909 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1806
157/394 steps:: L1: 0.0410 Sync: 0.2906 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1804
158/394 steps:: L1: 0.0409 Sync: 0.2906 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1803
159/394 steps:: L1: 0.0409 Sync: 0.2917 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1802
160/394 steps:: L1: 0.0409 Sync: 0.2916 Percep: 200.0000 Fake: 0.0000 recon: 0.0123 add: 0.1800
161/394 steps:: L1: 0.0409 Sync: 0.2913 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1799
162/394 steps:: L1: 0.0409 Sync: 0.2922 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1798
163/394 steps:: L1: 0.0409 Sync: 0.2920 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1796
164/394 steps:: L1: 0.0410 Sync: 0.2918 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1796
165/394 steps:: L1: 0.0410 Sync: 0.2920 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1794
166/394 steps:: L1: 0.0410 Sync: 0.2927 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1795
167/394 steps:: L1: 0.0410 Sync: 0.2933 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1796
168/394 steps:: L1: 0.0410 Sync: 0.2929 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1795
169/394 steps:: L1: 0.0410 Sync: 0.2927 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1797
170/394 steps:: L1: 0.0410 Sync: 0.2921 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1797
171/394 steps:: L1: 0.0410 Sync: 0.2922 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1797
172/394 steps:: L1: 0.0409 Sync: 0.2915 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1795
173/394 steps:: L1: 0.0410 Sync: 0.2923 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1795
174/394 steps:: L1: 0.0410 Sync: 0.2929 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1794
175/394 steps:: L1: 0.0410 Sync: 0.2932 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1795
176/394 steps:: L1: 0.0410 Sync: 0.2934 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1794
177/394 steps:: L1: 0.0410 Sync: 0.2929 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1796
178/394 steps:: L1: 0.0410 Sync: 0.2934 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1794
179/394 steps:: L1: 0.0409 Sync: 0.2930 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1793
180/394 steps:: L1: 0.0409 Sync: 0.2929 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1792
181/394 steps:: L1: 0.0409 Sync: 0.2923 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1792
182/394 steps:: L1: 0.0410 Sync: 0.2928 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1792
183/394 steps:: L1: 0.0410 Sync: 0.2925 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1792
184/394 steps:: L1: 0.0410 Sync: 0.2925 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1792
185/394 steps:: L1: 0.0410 Sync: 0.2922 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1793
186/394 steps:: L1: 0.0409 Sync: 0.2922 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1794
187/394 steps:: L1: 0.0409 Sync: 0.2921 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1794
188/394 steps:: L1: 0.0410 Sync: 0.2917 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1791
189/394 steps:: L1: 0.0410 Sync: 0.2921 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1791
190/394 steps:: L1: 0.0410 Sync: 0.2927 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1790
191/394 steps:: L1: 0.0410 Sync: 0.2932 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1788
192/394 steps:: L1: 0.0410 Sync: 0.2926 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1788
193/394 steps:: L1: 0.0410 Sync: 0.2932 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1785
194/394 steps:: L1: 0.0410 Sync: 0.2928 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1783
195/394 steps:: L1: 0.0410 Sync: 0.2928 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1782
Evaluating for 98 steps
98it [02:33,  1.57s/it]
valid:: L1: 0.0361 Sync: 0.2718 Percep: 100.0000 Mem_recon: 0.4806 Mem_ad: 0.1890
196/394 steps:: L1: 0.0410 Sync: 0.2926 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1783
197/394 steps:: L1: 0.0410 Sync: 0.2925 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1784
198/394 steps:: L1: 0.0410 Sync: 0.2926 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1782
199/394 steps:: L1: 0.0410 Sync: 0.2924 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1780
200/394 steps:: L1: 0.0409 Sync: 0.2930 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1780
201/394 steps:: L1: 0.0409 Sync: 0.2926 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1780
202/394 steps:: L1: 0.0409 Sync: 0.2928 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1780
203/394 steps:: L1: 0.0409 Sync: 0.2927 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1782
204/394 steps:: L1: 0.0410 Sync: 0.2926 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1780
205/394 steps:: L1: 0.0409 Sync: 0.2919 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1778
206/394 steps:: L1: 0.0409 Sync: 0.2918 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1776
207/394 steps:: L1: 0.0409 Sync: 0.2910 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1777
208/394 steps:: L1: 0.0409 Sync: 0.2908 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1775
209/394 steps:: L1: 0.0409 Sync: 0.2903 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1775
210/394 steps:: L1: 0.0409 Sync: 0.2903 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1774
211/394 steps:: L1: 0.0409 Sync: 0.2895 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1773
212/394 steps:: L1: 0.0409 Sync: 0.2896 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1771
213/394 steps:: L1: 0.0409 Sync: 0.2897 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1771
214/394 steps:: L1: 0.0409 Sync: 0.2893 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1770
215/394 steps:: L1: 0.0410 Sync: 0.2889 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1769
216/394 steps:: L1: 0.0410 Sync: 0.2884 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1768
217/394 steps:: L1: 0.0409 Sync: 0.2881 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1766
218/394 steps:: L1: 0.0410 Sync: 0.2878 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1768
219/394 steps:: L1: 0.0410 Sync: 0.2878 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1767
220/394 steps:: L1: 0.0410 Sync: 0.2880 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1767
221/394 steps:: L1: 0.0410 Sync: 0.2877 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1766
222/394 steps:: L1: 0.0410 Sync: 0.2877 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1766
223/394 steps:: L1: 0.0410 Sync: 0.2881 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1767
224/394 steps:: L1: 0.0411 Sync: 0.2881 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1766
225/394 steps:: L1: 0.0411 Sync: 0.2878 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1765
226/394 steps:: L1: 0.0411 Sync: 0.2878 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1763
227/394 steps:: L1: 0.0411 Sync: 0.2873 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1764
228/394 steps:: L1: 0.0411 Sync: 0.2872 Percep: 200.0000 Fake: 0.0000 recon: 0.0122 add: 0.1765
229/394 steps:: L1: 0.0411 Sync: 0.2868 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1764
230/394 steps:: L1: 0.0411 Sync: 0.2871 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1766
231/394 steps:: L1: 0.0411 Sync: 0.2869 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1767
232/394 steps:: L1: 0.0412 Sync: 0.2868 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1765
233/394 steps:: L1: 0.0412 Sync: 0.2870 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1765
234/394 steps:: L1: 0.0412 Sync: 0.2876 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1766
235/394 steps:: L1: 0.0412 Sync: 0.2878 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1765
236/394 steps:: L1: 0.0412 Sync: 0.2874 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1765
237/394 steps:: L1: 0.0412 Sync: 0.2873 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1767
238/394 steps:: L1: 0.0412 Sync: 0.2872 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1767
239/394 steps:: L1: 0.0412 Sync: 0.2868 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1767
240/394 steps:: L1: 0.0412 Sync: 0.2871 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1768
241/394 steps:: L1: 0.0412 Sync: 0.2871 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1768
242/394 steps:: L1: 0.0412 Sync: 0.2868 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1771
243/394 steps:: L1: 0.0412 Sync: 0.2868 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1770
244/394 steps:: L1: 0.0412 Sync: 0.2869 Percep: 200.0000 Fake: 0.0000 recon: 0.0121 add: 0.1770
jinny960812 commented 2 years ago

Hi, thanks for your interest in our work. You need to either decrease the learning rate of the discriminator or perform less number of update on the discriminator relative to the generator. Also, keep in mind that we are planning to update the code very soon.