Handling several slots in the same sentence

[Asma-droid]

Hi all,

Thanks to this great library :-)

I would like to use several slots within a same sentence but the produced json file does not pick up the right start and end of the slots.

Below a simple example:

**** txt_file *** %&ask_toilet where the @[toilet#singular] is @[please]?

@[toilet#singular] toilet loo @[please] please plz

* json result ***

{ "rasa_nlu_data": { "common_examples": [ { "entities": [ { "end": 13, "entity": "toilet", "start": 10, "value": "loo" }, { "end": 42, "entity": "please", "start": 36, "value": "please" } ], "intent": "ask_toilet", "text": "Where the loo is please?" }, { "entities": [ { "end": 13, "entity": "toilet", "start": 10, "value": "loo" }, { "end": 39, "entity": "please", "start": 36, "value": "plz" } ], "intent": "ask_toilet", "text": "where the loo is plz?" }, { "entities": [ { "end": 16, "entity": "toilet", "start": 10, "value": "toilet" }, { "end": 39, "entity": "please", "start": 36, "value": "plz" } ], "intent": "ask_toilet", "text": "where the toilet is plz?" } ], "entity_synonyms": [], "lookup_tables": [], "regex_features": [] } }

Any idea please ???

[SimGus]

Hi,

What you report seems related to issue #22 which was fixed in v1.6.1. Using the template you provided, I cannot reproduce the results you get.

Would you mind giving a little more information about the environment you run chatette in? Namely, I'd need to know:

• What version of Chatette are you running? You can find that by running python -m chatette --version.
• Which version of python do you use?
• What OS are you running?

Thanks in advance :)

[Asma-droid]

Thanks a lot. It works fine!! I have just one other problème. I use the french language however the actual code does not support UTF8 encoding. Any help please!!!

[SimGus]

You're welcome!

For Unicode encoding, I guess you are running Windows, which is likely to encode your files using Windows-1252 encoding. Try using a file editor that allows you to save your template files in UTF-8 and feeding them to Chatette should work without a problem. Using a recent version of python (>= 3.5 I would say) could also help.

Just so you know, I speak French and I have no problem running Chatette to produce French datasets by running Linux (whatever the Python version) and encoding all my files in UTF-8.

Hope this helps! Feel free to ask if you need help again!

[Asma-droid]

Thanks a lot. It works fine for me.

Just another question. How to avoid redendency. In fact, if i put 100 as the augmentation parameter, the algorithm generate redandant sentences.

[SimGus]

What exactly do you mean by "redundancy"? The generated data should never contain duplicates, so if you get the same generated sentence twice (or more times), this is a bug.

If you mean your sentences are too close to each other, this simply depends on your templates. A good way to have variation in the generated sentences is to have a lot of rules in your aliases, slots and templates. You can take a look at different examples on the repo if you want to see how to make good templates.

Cheers!

[Asma-droid]

Hello @SimGus

I am so sory for my late response.

I have tried to use the simple example of the directory toilets and i have set the number of generated sentences to 1000 instead of 3. The algorithme generate duplicated. Have you a solution to avoid that ??

[SimGus]

Hey @Asma-droid,

The two sentences are actually not duplicates: one of them starts with an uppercase letter, while the other one starts with a lowercase letter. If you want to avoid generating sentences with different cases, simply remove the ampersand & that are at the beginning of the unit declaration (so for the toilet example, remove the ampersand in %[&ask_toilet](100)).

I hope this helps.

[Asma-droid]

Thank you for this response !

[SimGus]

You're welcome! I will close this issue as it seems fixed.