search

ryankiros / neural-storyteller

A recurrent neural network for generating little stories about images
2.96k stars 543 forks source link

Generating biases #14

Open ncoronges opened 8 years ago

ncoronges commented 8 years ago

There are two files used for generation referenced in config.py: caption_style.npy' romance_style.npy'

these style vectors are used by generate.py bneg = numpy.load(config.paths['negbias']) bpos = numpy.load(config.paths['posbias'])

While supposedly you can create your decoder using the skip-thoughts decoder there does not seem to be a step where the style vectors are generated. If you have trained on your own corpus, and have dictionary and model, how do you then generate the style vectors and biases?

naotokui commented 7 years ago

Any info on this topic?? I'd love to know how to generate the style vectors.

koga73 commented 7 years ago

I would like to know the same. How are the .npy bias files generated?

Pantsworth commented 7 years ago

Based on code in generate.py, in the story function, the sentences are encoded, then the mean is taken using numpy.mean before applying the pos and neg biases.

svecs = skipthoughts.encode(z['stv'], sentences, verbose=False) shift = svecs.mean(0) - z['bneg'] + z['bpos']

To train new biases, encode your sentences and then save the mean of the encoded vector – numpy.save("path", <encoded_vector_array.mean(0)>). This will create a compatible bias file, which will be a simple numpy array.

The romance_style bias file was "the mean of the skip-thought vectors for romance novel passages that are of length > 100". Based on my limited testing of new bias vectors, this line culling is important to getting good results (although the overall effect may be more subtle than expected). Apologies in advance if any of the above is incorrect, I am not an authoritative source. Best of luck!