search

jcjohnson / torch-rnn

Efficient, reusable RNNs and LSTMs for torch
MIT License
2.5k stars 507 forks source link

Make the model "forget" or modify vocabulary it has been trained on? #213

Closed hackily closed 6 years ago

hackily commented 7 years ago

Is it possible to make the model forget, or modify a word that it has been trained on?

I left the RNN to train on a dataset for a week, but clearly should have removed expletives. So now my model output swears like a sailor.

Is there any way of removing those words out of its vocabularly, or even just modify the words?

For example -> shit -> shoot

... I'm going to hazard a guess that this isn't possible, but on the off chance it can be done, this would be very much appreciated.

turbotas commented 6 years ago

I'm going to say nope - the model builds it's output character by character and not word by word, so the swearing is deeply embedded in the net. The idea of needing to beep out the output is kind of funny - what on earth did you train it on?

hackily commented 6 years ago

I let it train on a bunch of comments from reddit, and I wanted to be able to show it live in class. It's funny, because swearing is embedded deeply in this net, and the (inter)net.