search

AntNLP / antu

Universal data IO and neural network modules in NLP tasks.
Apache License 2.0
17 stars 10 forks source link

Any description about example？ #5

Open Airuibadi opened 5 years ago

Airuibadi commented 5 years ago

Can't figure out such highly abstract code in your example, some description would help ,at least some brief comment. Please.

JT-Ushio commented 5 years ago

This PR provides a Code Example of what you need. SingleTaskDataset class is a simple abstract of single-task dataset. It has three parts: data_reader, Vocabulary and Counter Just look at the following two files to understand how to use data IO. Reader Dataset builder If you have a complicated way to build a batch, you can override the SingleTaskDataset.get_batches method.

Hope the code example can help you solve the problem. ps. A more detailed documentation example is in our plan.