huggingface / transfer-learning-conv-ai

🦄 State-of-the-Art Conversational AI with Transfer Learning
MIT License
1.73k stars 431 forks source link

Logical error in train.py #81

Open monisha08041998 opened 4 years ago

monisha08041998 commented 4 years ago

Hi Iam facing error for dataset_name, dataset in personachat.items():

num_candidates = len(dataset[0]["utterances"][0]["candidates"])

    num_candidates=2
    if args.num_candidates > 0 and dataset_name == 'train':
        num_candidates = min(args.num_candidates, num_candidates)
    for dialog in dataset:
        persona = dialog["personality"].copy()​
        for _ in range(args.personality_permutations):
            for utterance in dialog["utterances"]:
                history = utterance["history"][-(2*args.max_history+1):]
                for j, candidate in enumerate(utterance["candidates"][-num_candidates:]):
                    lm_labels = bool(j == num_candidates-1)
                    instance = build_input_from_segments(persona, history, candidate, tokenizer, lm_labels)
                    for input_name, input_array in instance.items():
                        datasets[dataset_name][input_name].append(input_array)
                datasets[dataset_name]["mc_labels"].append(num_candidates - 1)
                datasets[dataset_name]["n_candidates"] = num_candidates
            persona = [persona[-1]] + persona[:-1]  # permuted personalities

in this part of the train.py and also have anyone tried to train it with your own data.Please guide the steps

steveluo01 commented 1 year ago

for utterance in dialog["utterances"]: history = utterance["history"][-(2*args.max_history+1):] for j, candidate in enumerate(utterance["candidates"][-num_candidates:]): lm_labels = bool(j == num_candidates-1) instance = build_input_from_segments(persona, history, candidate, tokenizer, lm_labels)

Hello, can you teach me how to understand these sentences?