explain how unsup data pre-process works

[huangjinsuzhou]

Hi, I tried to use your code to pre-process my data but keeps getting error for the un-labeled txt. I tried to read your code but could not fully understand it. How the ori and aug data were generated.

In your IMDB class, there is an API, could you explain what is the meaning of the returned line[1] and line[2]?

def get_unsup(self, lines): for line in itertools.islice(lines, 0, None): yield (None, line[1], []), (None, line[2], []) # ko, en

[SanghunYun]

line : (id) (ori) (aug)

line[0] = id line[1] = ori line[2] = aug

So, you have to modify your dataset columns structure or modify the code