Open FurkanGozukara opened 5 years ago
By the way I have tested sentences as well and the outputs are pretty bad
It is also pretty slow even on my computer
Core i7 2600K @4.5 GHZ, 32 GB ram
Settings
def interact_model(
model_name='345M',
seed=None,
nsamples=2,
batch_size=2,
length=None,
temperature=1,
top_k=40,
models_dir='models',
):
Model prompt >>> Online Pokemon Game
======================================== SAMPLE 1 ========================================
" as seen on TV with the rest of Pokémon.
Mewtwo can evolve into a Mega Mewtwo from the Pokémon Center after collecting 100 EVs (500 if Mewtwo is caught in the Stadium).
Mewtwo in "Pikachu's Pachislot."
Mewtwo in "Million Shots!"
Mewtwo from "Mii-chan Gets Touched by a Monster."
Mewtwo from "Dewkinesis".
Pikachu's Meowth from "Shopping Cart."
Mewtwo from "Wobbuffet" (Japanese: 南 つけよろ) is an evolutive of Mega Gardevoir.
Mewtwo's design is based on its appearance in the anime Pikachu and the Pokémon Music Squad. Mewtwo, along with Mega Sceptile, and its alternate color scheme Mega Kangaskhan, were featured in a sketch, and Mega Sceptile was later featured in an issue of the same name.[1] In the Japanese dub, Mega Mewtwo is called "Mewtwo Mewtwo" instead and not "Mr. Mewtwo Mewtwo". The artwork for Mega Mewtwo depicts two Pokémon with two Pokémon in their mouths as well as the silhouette of a man in a hoodie holding a large baseball bat.
A promotional artwork of Mewtwo is seen on a card from the Pokémon Card Game series.
Mega Mewtwo does not have the ability to Mega Evolve, unlike other Pokémon with the ability to Mega Evolve. In the Super Smash Bros. series, Mewtwo has the same stat requirements as its Mega version.
Pikachu and other Pokémon in the Super Smash Bros. series have the ability to Mega-Evolve via training.
Wobbuffet, from "Mega-Dewpoint's Story", is the first boss to Mega Evolve.
In the Japanese dub, Mega Mewtwo is called "Dewkinesis."
A promotional artwork of Mewtwo appears in the first issue of Pokemon Diamond and Pearl.
Pikachu is the first to evolve into an alternate art Mewtwo, from the episode "Pokemon Rumble: Battle for Zekrom."
Anime
Mewtwo was one of the Pokémon introduced in Sun & Moon, which was originally featured in Pokémon X and Y. It is featured in the Diamond & Pearl chapter of the Diamond & Pearl series when Brock
======================================== SAMPLE 2 ========================================
" in all of the following formats:
Game Card – Game Card Format
– Game Card Format Special Character Card – Special Character Card Format
– Special Character Card Format Game Card (Awards Style)
– Game Card (Awards Style) Player's Information Box and Promotional Card – In-game Card Format
– In-game Card Format Game Cards
Special Items and Coin Chests have been replaced with a completely new type of Card. Some of them have different names or the same type with the same number of cards as the other cards, but there are no differences in appearance aside from different images and different text.
Each item can be obtained in the following formats:
1st Card: Pokemon
2nd Card: Pokemon
3rd Card: Pokemon with Ability
4th Card: Any Pokemon with Ability
Card
"Player's Information Box" – In-game Card Format
– In-game Card Format Disc 1 - Disc 2 (Japanese version only)
– Disc 1 - Disc 2 (Japanese version only) Disc 3 (Japanese version only)
– Disc 3 (Japanese version only) Disc 4 (Japanese version only)
– Disc 4 (Japanese version only) Disc 5 (Japanese version only)
– Disc 5 (Japanese version only) Player's Information Box and Promotional Card – In-game Card Format
In-game cards
There are several type of cards and in-game cards with the special qualities of each card (for example, there is only one "Energy" type card; that is, it can't be Energy-based). Some of these items are exclusive.
If a card possesses one of the following special qualities, it will be available only after the game ends by entering the "Item Access Code" or by redeeming a prize (but not both: you can obtain multiple prizes in the same playthrough).
The player's information box will usually show something similar to the following, but you can see additional information about the type of card above the box:
Level
The character's level
The nature of the attack it uses
The type of attack it executes
The level the enemy can attack you
Additional data
Certain cards, such as Special Items and Coin Chests, will have stats, movesets, battle abilities, and attacks with additional movesets and/or special attributes or abilities for other cards as well
================================================================================
Model prompt >>> Pikachu is a short, chubby rodent Pokémon. It is covered in yellow fur with two horizontal brown stripes on its back. It has a small mouth, long, pointed ears with black tips, and brown eyes. A each cheek is a red circle that contains a pouch for electricity storage. It has short forearms with five fingers on each paw, and its feet each have three toes. At the base of its lightning bolt-shaped tail is a patch of brown fur. A female will have a V-shaped notch at the end of its tail, which looks like the top of a heart. It is classified as a quadruped, but it has been known to stand and walk on its hind legs.
======================================== SAMPLE 1 ========================================
Ash's Pikachu appeared in Ash and Pikachu after a wild Ash caught it while riding his bicycle.
Major appearances
Pikachu debuted in A PokéRoulette Adventure as one of the Pokémon seen in Ash's Pokémon Pal Joey's Pikachu and the Mystery of Pikachu and the HeartGold and SoulSilver. It reappeared in I Have a Personal Pokémon and its evolutionary line appeared in the opening movie of the Ruby and Sapphire chapter, Pokémon Heroes: Latios and Latias.
Other
An Italian Pikachu was owned by the character Andrea De Nardo. He used it in a prank in the Italian film A Grand Prix For Pikachu.
Two Pikachu models were displayed at The Pokémon Box. One was obtained by Ash on a promotion, and the other Ash's Pikachu in his box of Pokémon in the Kalos region.
A Japanese Pikachu, nicknamed "Neko," sold in the retail department of the Pokémon Trading Card Game in Japan. A Nihon Gion shop in Hoenn, where the same Pikachu model can be played, has the exact silhouette of the previous Japanese Pikachu model.
A Japanese Pikachu model was first seen in a promotional video for Pokémon Center Japan and debuted in a Pikachu and the Clash of X's Wrath special. It appears in Ash's Pokémon Ranger Roost in The Search for the Legendary Pokémon!. In Ash's final battle against Cilan's Pinsir in Let's Choose His Own Adventure!, he defeats it using Pikachu to stop it from using a Stone.
Pokémon
Pikachu A female Japanese Pokémon resembling Pikachu was used by Team Rocket in Ash's first encounter with them. It was later acquired by Ash in his Pokémon Ranger Roost and was later seen in Ash's Pikachu and the Clash of X's Wrath special.
Jirachi A Japanese Pikachu was used by Ash to defeat the Kirlia in Pokémon Ranger Roost. Ash's Nidoqueen stole one from its Trainer. It was later seen being sold in the Kalos region in To Let Go of the Heroes!. It appeared briefly again in Let's Pick a Dream for the Day of the Sun, using it in an Ash's Pikachu.
Rouxels Pikachu Rouxels was one of the Pokémon Ash caught from a Vileplume egg before being attacked by it. A third Rouxels was used by the Johto League Leader when he came up with an idea for one of his Pokémon contests to celebrate Ash's birthday.
Dub
======================================== SAMPLE 2 ========================================
In the anime
In the main series
Major appearances
Cherche, a Yellow Pikachu, first appeared in The First Battle with a Mega-Pikachu. She was caught by Ash and his Pikachu, who later evolved into Mega Gardevoir and Mega Gardevine.
Pikachu was briefly seen being carried around by Pikachu.
The same Cherche appeared in The Last Battle XIII.
Multiple others were shown to exist which were later erased from history by Team Rocket.
A Pikachu appeared throughout Pikachu Adventures.
A male Pikachu's fur was seen in A Battle to the Top!, under the ownership of the Pokéwalker.
A Trainer named Bruno was seen wearing a Pikachu's fur mask in A Battle for Two Americas!. He eventually lost that mask and replaced it with a new one during A Tricky Friendship!, in which he also revealed his identity to Team Rocket.
Minor appearances
Pikachu debuted in Celebi: The Voice of the Forest as one of the Pokémon residing in a forest in Celadon City.
A Trainer's Caterpie appeared in The Final Battle VIII.
A Pikachu appeared in Pika Breeding!, under the ownership of Pidgey.
Professor Oak's Pikachu and his Trainer appeared in The First Battle XIII.
Two Pichu appeared in a fantasy in A Day Break!, under the ownership of the Pokéwalker.
Several Ash's Pikachu made their Pokémon dreams in The Power of Us...!
Three Pikachu were among the Pokémon seen at Professor Oak's Laboratory in A Day in the Life of Arceus.
Several Pachirisu appeared in Volcanion and the Mechanical Marvel. They reappeared in All About Bugsy!.
A Trainer's Pichu appeared in A Challenge in Pokémon!
Pokédex entries
Episode Pokémon Source Entry EP060 Cherche Dawn's Pokédex Cherche, the Yellow Pikachu, is a cute little Pokémon that looks just like a Pichu at first. However, it is actually a type of Pikachu. This concludes the entries from the original series.
Episode Pokémon Source Entry DP077 Cherche Dawn's Pokédex Cherche, the Yellow Pikachu, is a cute-looking Pokémon with a yellow hue. Its fur is black, and there are five clawed feet on its hind legs. Cherche can grow to be an enormous Pokémon. This concludes the entries from the Diamond & Pearl
================================================================================
6) When I change the length parameter to 1000, I get this error
Model prompt >>> Pikachu is a short, chubby rodent Pokémon. It is covered in yellow fur with two horizontal brown stripes on its back. It has a small mouth, long, pointed ears with black tips, and brown eyes. A each cheek is a red circle that contains a pouch for electricity storage. It has short forearms with five fingers on each paw, and its feet each have three toes. At the base of its lightning bolt-shaped tail is a patch of brown fur. A female will have a V-shaped notch at the end of its tail, which looks like the top of a heart. It is classified as a quadruped, but it has been known to stand and walk on its hind legs. Traceback (most recent call last): File "C:\Python37\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call return fn(*args) File "C:\Python37\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn options, feed_dict, fetch_list, target_list, run_metadata) File "C:\Python37\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun run_metadata) tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,0] = 1024 is not in [0, 1024) [[{{node sample_sequence/while/model/GatherV2_1}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "src/interactive_conditional_samples.py", line 90, in
fire.Fire(interact_model) File "C:\Python37\lib\site-packages\fire\core.py", line 127, in Fire component_trace = _Fire(component, args, context, name) File "C:\Python37\lib\site-packages\fire\core.py", line 366, in _Fire component, remaining_args) File "C:\Python37\lib\site-packages\fire\core.py", line 542, in _CallCallable result = fn(*varargs, **kwargs) File "src/interactive_conditional_samples.py", line 80, in interact_model context: [contexttokens for in range(batch_size)] File "C:\Python37\lib\site-packages\tensorflow\python\client\session.py", line 929, in run run_metadata_ptr) File "C:\Python37\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run feed_dict_tensor, options, run_metadata) File "C:\Python37\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run run_metadata) File "C:\Python37\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,0] = 1024 is not in [0, 1024) [[node sample_sequence/while/model/GatherV2_1 (defined at C:\gpt-2\src\model.py:157) ]] Caused by op 'sample_sequence/while/model/GatherV2_1', defined at: File "src/interactive_conditional_samples.py", line 90, in
fire.Fire(interact_model) File "C:\Python37\lib\site-packages\fire\core.py", line 127, in Fire component_trace = _Fire(component, args, context, name) File "C:\Python37\lib\site-packages\fire\core.py", line 366, in _Fire component, remaining_args) File "C:\Python37\lib\site-packages\fire\core.py", line 542, in _CallCallable result = fn(*varargs, kwargs) File "src/interactive_conditional_samples.py", line 64, in interact_model temperature=temperature, top_k=top_k File "C:\gpt-2\src\sample.py", line 73, in sample_sequence back_prop=False, File "C:\Python37\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 3556, in while_loop return_same_structure) File "C:\Python37\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 3087, in BuildLoop pred, body, original_loop_vars, loop_vars, shape_invariants) File "C:\Python37\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 3022, in _BuildLoop body_result = body(packed_vars_for_body) File "C:\Python37\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 3525, in body = lambda i, lv: (i + 1, orig_body( lv)) File "C:\gpt-2\src\sample.py", line 45, in body next_outputs = step(hparams, prev, past=past) File "C:\gpt-2\src\sample.py", line 33, in step lm_output = model.model(hparams=hparams, X=tokens, past=past, reuse=tf.AUTO_REUSE) File "C:\gpt-2\src\model.py", line 157, in model h = tf.gather(wte, X) + tf.gather(wpe, positions_for(X, past_length)) File "C:\Python37\lib\site-packages\tensorflow\python\util\dispatch.py", line 180, in wrapper return target(*args, *kwargs) File "C:\Python37\lib\site-packages\tensorflow\python\ops\array_ops.py", line 3273, in gather return gen_array_ops.gather_v2(params, indices, axis, name=name) File "C:\Python37\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 4390, in gather_v2 "GatherV2", params=params, indices=indices, axis=axis, name=name) File "C:\Python37\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper op_def=op_def) File "C:\Python37\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func return func(args, kwargs) File "C:\Python37\lib\site-packages\tensorflow\python\framework\ops.py", line 3300, in create_op op_def=op_def) File "C:\Python37\lib\site-packages\tensorflow\python\framework\ops.py", line 1801, in init self._traceback = tf_stack.extract_stack()InvalidArgumentError (see above for traceback): indices[0,0] = 1024 is not in [0, 1024) [[node sample_sequence/while/model/GatherV2_1 (defined at C:\gpt-2\src\model.py:157) ]]
Same issue with (6), if u have solution right now, please inform me, thank you :)
1) You need to play with python a bit. File reading (with control string splitting, JSON or other method) or parsing multiple files in directory. My preferred way is database with web interface, however, I do NOT run automation. 2) I guess not, but if you program your own functionality then you can have more. Again, my parameters are controlled by database. 3) Nvidia only (well it's possible to work on AMD but afaik Linux only and head scratching a lot). 117M will work on 2GB GPU, 345M I guess 4GB GPU. I run mine on Nvidia Gefore 1660 6GB and I can confirm that it can run up to 7 samples in a batch but can crash. 6 samples are OK but crashes when looped. 5 samples are safe. 4) I guess keeping exactly the same grammar and style the data it learned on had? However, as said in developers readme, a lot of is not yet understood about the AI behavior (hence the research). 5) Install GPU version of tensorflow. It's written in readme. For correct version of cudnn and cuda refer to google search for the version of tensorflow you are willing to install. There is also a guide on this that will tell you exactly how to update PATH variable (in case of windows).
It is slow on CPU, yes. It takes 87 seconds to generate 5 samples in a batch from the word "Suddenly", using maximum output length possible, on Gefore 1660 6GB:
39. Waiting for DB data
T: 1.0 | K: 0.0 | P: 0.1, 0.9 | P_M: 0.0, 0.5 | P_S: 0 | S: 5 | B: 5 | ID: s08k6ih5s5oqjlcc55tcandlg0 |>>> Suddenly
>> Response Saved as: D:\!OpenAI_stories\OpenAI-20190704220718.txt
!! Time taken (sec): 87.18098640441895
6)
length = hparams.n_ctx - context_length
is the possible maximum length per sample, so with one token it can output up to 1023 afaik. If you want to get 1000, you CANNOT use more than 24 tokens as input
@FurkanGozukara I've created PR for this. You can take a look at #174
1050ti? forget about GPU. You need 2080ti
Hey, how do i make the GPT-2 to create its output to a specifig text file?
Here's the script I use to interface to the GPT-2 models. I have it outputting the text to my speech synthesis module, but you could just as easily use a "with file as..." and write the variable ("text" in the script below - scroll down to the while True block near the end) content to the file...
#!/usr/bin/env python3
import fire
import json
import os
import numpy as np
import tensorflow as tf
import zmq
import time
context = zmq.Context()
print("Connecting to Speech Center")
socket = context.socket(zmq.REQ)
socket.connect("tcp://localhost:5555") #5556 is visual, 5558 is language, 555$
import model, sample, encoder
def interact_model(
model_name='345M', #345M on Pi4B4 or 8 only (memory allocation issue) 774 & 1558 too big for Pi4b8G too.
seed=None,
nsamples=1,
batch_size=1,
length=140,
temperature=1.2,
top_k=48,
top_p=0.7,
models_dir='models',
):
models_dir = os.path.expanduser(os.path.expandvars(models_dir))
if batch_size is None:
batch_size = 1
assert nsamples % batch_size == 0
enc = encoder.get_encoder(model_name, models_dir)
hparams = model.default_hparams()
with open(os.path.join(models_dir, model_name, 'hparams.json')) as f:
hparams.override_from_dict(json.load(f))
if length is None:
length = hparams.n_ctx // 2
elif length > hparams.n_ctx:
raise ValueError("Can't get samples longer than window size: %s" % hparams.n_ctx)
with tf.Session(graph=tf.Graph()) as sess:
context = tf.placeholder(tf.int32, [batch_size, None])
np.random.seed(seed)
tf.set_random_seed(seed)
output = sample.sample_sequence(
hparams=hparams, length=length,
context=context,
batch_size=batch_size,
temperature=temperature, top_k=top_k, top_p=top_p
)
saver = tf.train.Saver()
ckpt = tf.train.latest_checkpoint(os.path.join(models_dir, model_name))
saver.restore(sess, ckpt)
while True:
raw_text = input("\n\nModel prompt >>> ")
while not raw_text:
print('Prompt should not be empty!')
raw_text = input("\n\nModel prompt >>> ")
context_tokens = enc.encode(raw_text)
generated = 0
for _ in range(nsamples // batch_size):
out = sess.run(output, feed_dict={
context: [context_tokens for _ in range(batch_size)]
})[:, len(context_tokens):]
for i in range(batch_size):
generated += 1
text = enc.decode(out[i])
stripFrag = text.rsplit(".", 1)
text = stripFrag[0] + "." #Truncates after last "." to strip sent. frags.
print("=" * 40 + " SAMPLE " + str(generated) + " " + "=" * 40)
print(text)
socket.send_string(text)
message = socket.recv()
message = message.decode('utf-8')
if message==text:
print("1")
else:
print("0")
print("=" * 80)
if __name__ == '__main__':
fire.Fire(interact_model)
I did setup the system on windows 10, x64, 32 gb ram, core i7 2600k and GTX 1050Ti
When I run this
python src/interactive_conditional_samples.py
It asks Model prompt >>Then I type my word such as Pikachu and output is as below. This output obviously seeems incorrect and unrelated
So my first questions are as below
1 ) How can i give input.txt as input which will have few sentences and write output to a text file?
2 ) Also are there any other parameters than these?
interactive_conditional_samples.py
hparams.json
3) How can i make it work on GPU instead of CPU?
4) Can I make it return more logical output for given sentences?
5) How can I make it run on GPU instead of CPU? I have GTX 1050 Ti, 4 GB ram
Here is the full input and output of my CMD