Train a chatbot model: error: one of the arguments --datasetdir/-d --checkpointfile/-c is required An exception has occurred, use %tb to see the full traceback.

[irdanish11]

The exception occurs in general_utils.py file at line 31, when we try to parse all the arguments in training mode. args = parser.parse_args()

[AbrahamSanders]

Hi @irdanish11, can you share the command you are using that generates this error?

[irdanish11]

when I run the train.py file to train, on 18th line of train.py it creates the model using 'general_utils.initialize_session' then the above described exception occurs. when I checked the 'general_utils.py' file line by line, it turned out that it is the 'args = parser.parse_args()' is line that generating the exception.

`def initialize_session(mode): """Helper method for initializing a chatbot training session by loading the model dir from command line args and reading the hparams in

Args:
    mode: "train" or "chat"
"""
parser = argparse.ArgumentParser("Train a chatbot model" if mode == "train" else "Chat with a trained chatbot model")
if mode == "train":
    ex_group = parser.add_mutually_exclusive_group(required=True)
    ex_group.add_argument("--datasetdir", "-d", help="Path structured as datasets/dataset_name. A new model will be trained using the dataset contained in this directory.")
    ex_group.add_argument("--checkpointfile", "-c", help="Path structured as 'models/dataset_name/model_name/checkpoint_name.ckpt'. Training will resume from the selected checkpoint. The hparams.json file should exist in the same directory as the checkpoint.")
    em_group = parser.add_argument_group()
    em_group.add_argument("--encoderembeddingsdir", "--embeddingsdir", "-e", help="Path structured as embeddings/embeddings_name. Encoder (& Decoder if shared) vocabulary and embeddings will be initialized from the checkpoint file and tokens file contained in this directory.")
    em_group.add_argument("--decoderembeddingsdir", help="Path structured as embeddings/embeddings_name. Decoder vocabulary and embeddings will be initialized from the checkpoint file and tokens file contained in this directory.")
elif mode == "chat":
    parser.add_argument("checkpointfile", help="Path structured as 'models/dataset_name/model_name/checkpoint_name.ckpt'. The hparams.json file and the vocabulary file(s) should exist in the same directory as the checkpoint.")
else:
    raise ValueError("Unsupported session mode. Choose 'train' or 'chat'.")
args = parser.parse_args()`

[irdanish11]

The last line in the above code is causing exception and the entire trace message is:

`usage: Train a chatbot model [-h] (--datasetdir DATASETDIR | --checkpointfile CHECKPOINTFILE) [--encoderembeddingsdir ENCODEREMBEDDINGSDIR] [--decoderembeddingsdir DECODEREMBEDDINGSDIR] Train a chatbot model: error: one of the arguments --datasetdir/-d --checkpointfile/-c is required An exception has occurred, use %tb to see the full traceback.

SystemExit: 2

/root/anaconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py:2969: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D. warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)`

[irdanish11]

I'm using spyder on linux to run this code

[AbrahamSanders]

As the error is saying, you need either --datasetdir or --checkpointfile parameters to train.

In spyder go to Run -> Configuration per file You can set Command Line options. Try setting --datasetdir=datasets\cornell_movie_dialog

Let me know if this works. If not, you can always just train from an Anaconda console: run train.py --datasetdir=datasets\cornell_movie_dialog

[irdanish11]

I've changed the setting in spyder that you have recommended, the same error remains

[irdanish11]

Actually I want to build the chatbot in urdu language so I have totally different corpus, so can you please guide me about your code and where you have done the preprocessing and overview of your code that how it works

[AbrahamSanders]

That is strange that spyder is not sending the commands properly, You can try running the training directly in an Anaconda console. I recommend doing it that way anyway since training may take a long time and you won't be able to close Spyder.

Can you share the format of your Urdu dataset? You will need to format it using the CSV, Cornell, or DailyDialog formats (see the readme files under these directories: datasets

Also the clean_text function in vocabulary.py is doing the text normalization. You may need to modify it to work with the specifics of your language if it does not work out of the box. You can check the cleaned_dataset.txt file that is generated when you train a model to make sure that the text normalization is working the way you expect. For languages that use an English or similar european alphabet, the normalization makes everything lowercase, strips out special characters, treats punctuation as its own word, and can optionally expand contractions like "don't" --> "do not".

[irdanish11]

Here is the example some of the sentences in our corpus, now can you please tell in which file I have to change the corpus name so that it would start training on my corpus instead of cornell. And thanks for helping me the method you have told is just working fine now in spyder.

[irdanish11]

And one thing more can I have to split m dataset in question and answers files separately

[irdanish11]

my corpus is .txt format what changes I have to make to my corpus , and please explain me about movie_lines.txt movie_conversations.txt do I have to change my corpus according to these 2 files, currently my corpus is in plain text file

[tathagatankit]

Can we train these models on Google Cloud Platform?? If yes, then please let me know how to do it?