b0noI / dialog_converter

MIT License
38 stars 21 forks source link

Stack 2 and 3 sentence training data #8

Closed goswamig closed 6 years ago

goswamig commented 6 years ago

Results: Original: -rw-r--r-- 1 gauta 1896053708 5.9M Jun 15 17:23 train.b -rw-r--r-- 1 gauta 1896053708 5.9M Jun 15 17:23 train.a -rw-r--r-- 1 gauta 1896053708 326K Jun 15 17:23 test.b -rw-r--r-- 1 gauta 1896053708 320K Jun 15 17:23 test.a 101312 101312

After stacking one sentence -rw-r--r-- 1 gauta 1896053708 44M Jun 15 17:29 train.b -rw-r--r-- 1 gauta 1896053708 68M Jun 15 17:29 train.a -rw-r--r-- 1 gauta 1896053708 2.3M Jun 15 17:29 test.b -rw-r--r-- 1 gauta 1896053708 3.6M Jun 15 17:29 test.a 231644 231644

After stacking 2 sentences -rw-r--r-- 1 gauta 1896053708 1.6G Jun 15 17:35 train.b -rw-r--r-- 1 gauta 1896053708 3.1G Jun 15 17:35 train.a -rw-r--r-- 1 gauta 1896053708 83M Jun 15 17:35 test.b -rw-r--r-- 1 gauta 1896053708 165M Jun 15 17:35 test.a 361976 361976

goswamig commented 6 years ago

@b0noI @kasavine

b0noI commented 6 years ago

@gautamkmr please resolve conflicts and I will merge

goswamig commented 6 years ago

done