search

enzoampil / tito-joker

A humorous AI that uses state-of-the-art deep learning to tell jokes
http://35.225.94.177:8501/
GNU General Public License v3.0
45 stars 5 forks source link

Refactor `run_generation` module #29

Open enzoampil opened 4 years ago

enzoampil commented 4 years ago

The current module still has a lot of unnecessary code coming from the CLI implementation from transformers. These are better as standalone functions that take default arguments from a separate config file (potentially config.yaml).

Potential output:

Create new GenerationPipeline under HuggingFace's transormers.pipeline module.

enzoampil commented 4 years ago

Apparently, no need to create a transformers.pipeline, all you need is to do model.generate() to generate text from a language generation model.

Amazing!

https://github.com/huggingface/transformers/issues/3728#issuecomment-611797988

enzoampil commented 4 years ago

However, there is still value add since transformers.pipeline adds the seamless model download experience + allows access to different models from the same API

enzoampil commented 4 years ago

Note: make sure to use GPT2LMHeadModel instead of GPTModel. It's the former that has the generate method.

enzoampil commented 4 years ago

Had a "no padding_id" warning from doing model.generate(), but learned from the issue response below that this is expected behaviour for GPT2 https://github.com/huggingface/transformers/issues/2630

enzoampil commented 4 years ago

Generation method + detailed docs found in this module