Few questions on training data

[EdenHazard10]

• Is the dataset used for training available in raw format(text - human understandable) ? Or can we get a sampled version of the dataset ? This can help us create our own custom dataset.
• How to structure a custom dataset for this Model ? I believe many people have asked the same question before. Would be incredibly helpful if someone can guide.
• This is tied to the previous questions. How to create new training data on top of a trained model ? I want to utilise the existing capabilities as well as add new training data.

There are many cases in which we are getting only 40-50% accuracy for our own testing data. My understanding is that we can improve it beyond 90% if good training data is added.

Please let me know if you need any clarifications.

[loretoparisi]

@embreinhardt +1 definitively need to answer to this questions.

[Newmu]

• Data is available for research purposes at Julian McAuley's site. We used the aggressively deduplicated version of the dataset on that site. The only preprocessing applied was to unescape html entities.
• For an example of using the feature pipeline discussed in the paper with custom (labeled) training data we've added a demo that replicates the SST binary result from the paper https://github.com/openai/generating-reviews-discovering-sentiment/commit/83d940d132fbd585deb63b8e3fcb523ea9557683.