Documentation for predictions incorrect.

[AdamMiltonBarker]

[Enter feedback here]

When we send a request to predictions using the specified parameters in full it tells us that we need to provide all parameters, however there are no more parameters provided in the documentation.

{"kind":"Conversation","analysisInput":{"conversationItem":{"id":1,"participantId":1,"text":"Text 1"}},"parameters":{"projectName":"intellistore","deploymentName":"isa-7","stringIndexType":"TextElement_V8"}}

We receive in the body error:

{"InvalidArgument! The input request body is invalid. Please check proper values are specified for parameters like \"kind\", \"projectKind\", and so on.."}

---

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

• ID: 6ceb42a1-2153-154c-3169-e8c29bd40770
• Version Independent ID: 1af19c98-4142-eb11-e77c-6cd177ddf5ae
• Content: Send prediction requests to a conversational language understanding deployment - Azure AI services
• Content Source: articles/ai-services/language-service/conversational-language-understanding/how-to/call-api.md
• Service: azure-ai-language
• GitHub Login: @aahill
• Microsoft Alias: aahi

[SaibabaBalapur-MSFT]

@AdamMiltonBarker Thanks for your feedback! We will investigate and update as appropriate.

[AdamMiltonBarker]

@AdamMiltonBarker Thanks for your feedback! We will investigate and update as appropriate.

Thanks, any update on this as our development is blocked.

[RamanathanChinnappan-MSFT]

@AdamMiltonBarker I've delegated this to @aahill, a content author, to review and share their valuable insights.

[AdamMiltonBarker]

Hi @aahill our development has been blocked for 5 days please would it be possible to fix the documentation so we can continue?

[AdamMiltonBarker]

Hi @aahill please can we have an update?

[AdamMiltonBarker]

Hi @aahill please can we have an update? Our development has been blocked for over a week now.

[AdamMiltonBarker]

@aahill @RamanathanChinnappan-MSFT @SaibabaBalapur-MSFT please could someone respond to this, we are part of Startups for Founders and we have deadlines with this project to both progress to next stage of the program and to apply for an event in the US. This issue has completely blocked our development for over a week. Please would it possible to fix your documentation?

[SaibabaBalapur-MSFT]

@aahill please review it.

[AdamMiltonBarker]

@aahill please can we have an update?

[aahill]

Hi @AdamMiltonBarker , sorry for the delayed response, I've been out due for extended medical leave and am just now back.

Looking at the JSON snippet you provided, I think the issue may be that your id and participantId fields are lacking double quotes (") to specify them as text, instead of numbers when sending your request to the API. For example, I'm able to successfully send a prediction request to a project I set up.

{
  "kind": "Conversation",
  "analysisInput": {
    "conversationItem": {
      "id": "1",
      "participantId": "1",
      "text": "text1"
    }
  },
  "parameters": {
    "projectName": "myProject",
    "deploymentName": "v1",
    "stringIndexType": "TextElement_V8"
  }
}

When I change id and participantId to number values (that is to say, set them to):

• "id": 1 and
• "participantId": 1

instead of:

• "id": "1",
• "participantId": "1",

I get the same error that you are getting.

{
    "error": {
        "code": "InvalidArgument",
        "message": "The input request body is invalid. Please check proper values are specified for parameters like \"kind\", \"projectKind\", and so on.."
    }
}

Hope this helps.

[AdamMiltonBarker]

@aahill ok thanks, this has fixed the issue. I feel that the error response should be more specific on this one, and that your documentation needs to be updated to specify that string is a requirement for those values, naturally with the name ID you would usually expect an integer.

We have another issue. We have implemented code that trains the model and deploys, this has been working well until we hit 60 utterances. our current dataset has 110 utterances. However, only 60 of these will make it into the training data. We have checked on studio and it is completely ignoring 50 of the examples sent through the API.

In addition, despite setting the data split parameters when sending the request to train, there is no testing data in studio.

, "evaluationOptions" => [ "kind" => "percentage", "testingSplitPercentage" => 20, "trainingSplitPercentage" => 80, ],

[aahill]

Thanks, I'll make a note about this feedback for the docs, and pass the suggestion to the product team.

I'm having trouble replicating your issue with utterances not being uploaded, as I've been able to train a model using the example project file we have for the quickstart, which has a large number of utterances. Perhaps make sure you aren't hitting any of the limits for the service? (for example, utterance length)

https://learn.microsoft.com/en-us/azure/ai-services/language-service/conversational-language-understanding/service-limits#api-limits. I may also suggest Microsoft Q&A which might be able to provide insights for your implementation.

In regards to test data in the studio, is the issue that you aren't seeing the testing data in the Data labeling section of the studio?

if so, only data that's labeled as in your JSON as "dataset": "Test" will appear specifically in that section

{
    "text": "delete all emails from December",
    "language": "en-us",
    "intent": "Delete",
    "entities": [],
    "dataset": "Test"
}

Doing so will enable these utterances to show up there

They'll also be used if you use the manual setting for training.

  "evaluationOptions": {
    "kind": "manual"
  }

Otherwise if you use the percentage option and don't label your utterances explicitly as either test or train, you can still see the utterances being used to test your model in the model performance > test set details section in the studio.

[AdamMiltonBarker]

Hi @aahill I already checked the limits in the docs you have sent and we are no where near. We have 110 utterances but only 60 exist once sent to the API (6 pages of 10 utterances).

As I said everything is working correctly, we can upload data, train, and inference, the only issue is that 50 utterances are not making it into the dataset despite being sent correctly.

Regarding the model data split, we followed this documentation.

Using training mode standard, no test data exists.

The model is trained and deployed, the last of the utterances are completely lost somewhere between leaving our server and being imported into the data on your platform.

{"projectFileVersion":"2022-05-01","stringIndexType":"Utf16CodeUnit","metadata":{"projectKind":"Conversation","settings":{"confidenceThreshold":0.6999999999999999555910790149937383830547332763671875},"projectName":"intellistore","description":"ISA the IntelliStore Intelligent Store Assistant","multilingual":false,"language":"en"},"assets":{"projectKind":"Conversation","intents":[{"category":"SearchCategory"},{"category":"SearchBrand"},{"category":"SearchProduct"}],"entities":[{"category":"Category","compositionSetting":"returnLongestOverlap"},{"category":"Brand","compositionSetting":"returnLongestOverlap"},{"category":"Product","compositionSetting":"returnLongestOverlap"}],"utterances":[{"text":"Where can I find shoes","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":17,"length":5}]},{"text":"Where can I find footwear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":17,"length":8}]},{"text":"Where can I find footgear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":17,"length":8}]},{"text":"Where can I find trainers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":17,"length":8}]},{"text":"Where can I find sneakers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":17,"length":8}]},{"text":"Where can I find runners?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":17,"length":7}]},{"text":"What would you recommend for trainers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":29,"length":8}]},{"text":"What would you recommend for sneakers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":29,"length":8}]},{"text":"What would you recommend for runners?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":29,"length":7}]},{"text":"Take me to your trainers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"Take me to your sneakers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"Take me to your runners","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":7}]},{"text":"Show me trainers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":8,"length":8}]},{"text":"Show me sneakers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":8,"length":8}]},{"text":"Show me runners","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":8,"length":7}]},{"text":"I want to see your trainers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":19,"length":8}]},{"text":"I want to see your sneakers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":19,"length":8}]},{"text":"I want to see your runners","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":19,"length":7}]},{"text":"How many trainers do you have?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":8}]},{"text":"How many sneakers do you have?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":8}]},{"text":"How many runners do you have?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":7}]},{"text":"Does this store carry trainers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":22,"length":8}]},{"text":"Does this store carry sneakers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":22,"length":8}]},{"text":"Does this store carry runners?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":22,"length":7}]},{"text":"Do you have trainers in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":12,"length":8}]},{"text":"Do you have sneakers in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":12,"length":8}]},{"text":"Do you have runners in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":12,"length":7}]},{"text":"Do you have any trainers in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"Do you have any sneakers in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"Do you have any runners in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":7}]},{"text":"Can I buy trainers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":10,"length":8}]},{"text":"Can I buy sneakers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":10,"length":8}]},{"text":"Can I buy runners?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":10,"length":7}]},{"text":"What would you recommend for shoes","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":29,"length":5}]},{"text":"What would you recommend for footwear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":29,"length":8}]},{"text":"What would you recommend for footgear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":29,"length":8}]},{"text":"Take me to your shoes","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":5}]},{"text":"Take me to your footwear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"Take me to your footgear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"Show me shoes","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":8,"length":5}]},{"text":"Show me footwear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":8,"length":8}]},{"text":"Show me footgear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":8,"length":8}]},{"text":"I want to see your shoes","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":19,"length":5}]},{"text":"I want to see your footwear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":19,"length":8}]},{"text":"I want to see your footgear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":19,"length":8}]},{"text":"How many shoes do you have","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":5}]},{"text":"How many footwear do you have","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":8}]},{"text":"How many footgear do you have","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":8}]},{"text":"Does this store carry shoes","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":22,"length":5}]},{"text":"Does this store carry footwear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":22,"length":8}]},{"text":"Does this store carry footgear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":22,"length":8}]},{"text":"Do you have shoes in stock","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":12,"length":5}]},{"text":"Do you have footwear in stock","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":12,"length":8}]},{"text":"Do you have footgear in stock","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":12,"length":8}]},{"text":"Do you have any shoes in stock","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":5}]},{"text":"Do you have any footwear in stock","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"Do you have any footgear in stock","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"Can I buy shoes","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":10,"length":5}]},{"text":"Can I buy footwear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":10,"length":8}]},{"text":"Can I buy footgear","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":10,"length":8}]},{"text":"Where can I find flip flops?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":17,"length":10}]},{"text":"Where can I find pluggers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":17,"length":8}]},{"text":"Where can I find jaggers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":17,"length":7}]},{"text":"What would you recommend for flip flops?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":29,"length":10}]},{"text":"What would you recommend for pluggers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":29,"length":8}]},{"text":"What would you recommend for jaggers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":29,"length":7}]},{"text":"Take me to your flip flops","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":10}]},{"text":"Take me to your pluggers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"Take me to your jaggers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":7}]},{"text":"Show me flip flops","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":8,"length":10}]},{"text":"Show me pluggers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":8,"length":8}]},{"text":"Show me jaggers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":8,"length":7}]},{"text":"How many flip flops do you have?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":10}]},{"text":"How many pluggers do you have?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":8}]},{"text":"How many jaggers do you have?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":7}]},{"text":"I want to see your flip flops","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":19,"length":10}]},{"text":"I want to see your pluggers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":19,"length":8}]},{"text":"I want to see your jaggers","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":19,"length":7}]},{"text":"Does this store carry flip flops?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":22,"length":10}]},{"text":"Does this store carry pluggers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":22,"length":8}]},{"text":"Does this store carry jaggers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":22,"length":7}]},{"text":"Do you have flip flops in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":12,"length":10}]},{"text":"Do you have pluggers in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":12,"length":8}]},{"text":"Do you have jaggers in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":12,"length":7}]},{"text":"Do you have any flip flops in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":10}]},{"text":"Do you have any pluggers in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"Do you have any jaggers in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":7}]},{"text":"Can I buy flip flops?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":10,"length":10}]},{"text":"Can I buy pluggers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":10,"length":8}]},{"text":"Can I buy jaggers?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":10,"length":7}]},{"text":"Where can I find sandles?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":17,"length":7}]},{"text":"Where can I find moccasin?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":17,"length":8}]},{"text":"What would you recommend for sandles?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":29,"length":7}]},{"text":"What would you recommend for moccasin?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":29,"length":8}]},{"text":"Show me sandles","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":8,"length":7}]},{"text":"Show me moccasin","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":8,"length":8}]},{"text":"How many trainers do you have?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":8}]},{"text":"How many sneakers do you have?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":8}]},{"text":"How many runners do you have?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":9,"length":7}]},{"text":"Take me to your sandles","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":7}]},{"text":"Take me to your moccasin","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"I want to see your sandles","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":19,"length":7}]},{"text":"I want to see your moccasin","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":19,"length":8}]},{"text":"Does this store carry sandles?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":22,"length":7}]},{"text":"Does this store carry moccasin?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":22,"length":8}]},{"text":"Do you have sandles in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":12,"length":7}]},{"text":"Do you have moccasin in stock?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":12,"length":8}]},{"text":"Do you have any sandles?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":7}]},{"text":"Do you have any moccasin?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":16,"length":8}]},{"text":"Can I buy sandles?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":10,"length":7}]},{"text":"Can I buy moccasin?","language":"en","intent":"SearchCategory","entities":[{"category":"Category","offset":10,"length":8}]}]}}

[aahill]

I see, because there are no utterances labeled "dataset": "Test" in your project, all the utterances are added automatically to the "training set" label, and appear in the training set panel by default in the data labeling section of the studio UI (because nothing was labeled "test". That's fine, and I'll explain where you can see your test data further down)

(note: the number of utterances here might be slightly different than yours as I had to remove a few duplicate utterances as they were giving me an error, like this one for example).

      {
          "code": "InvalidArgument",
          "message": "Your utterance `How many trainers do you have?` is duplicated in the dataset `Train`. No duplicate utterances are allowed in the same dataset."
      }

One thing here is that if you choose to have the system train a model by automatically splitting your data into a training and testing set (using the "kind":"percentage" training kind, like you mention you're using from the documentation), the system only splits the data into the sets when you send the training command.

Earlier I mentioned it's fine to have your data all labeled training, and in the training set tab of the data labeling section, it's because the system will split only automatically split data that's designated/labeled as training (which is the default label). The training documentation mentions this in the phrasing used by the studio experience, so I can understand it's confusing when using the service programmatically.

In order to see the data that was used to actually train and test your model, you can go to the model performance section in the studio, and then test set details

Hopefully that helps explains why your test data isn't appearing in the labeling tab.

Now, going back to the data labeling section of the studio, I am seeing 10 pages of utterances, which still leaves the question of why all of your utterances aren't appearing on your side, and I'm not quite sure about.

I'm wondering if when you're uploading your project JSON, the earlier errors I mentioned are preventing the project from being updated, and leaving only the project version with a smaller number of utterances? I'm unsure.

[AdamMiltonBarker]

Hi @aahill ok thanks for looking into this. I will check how we create the dataset, that should be an error on our side, in our dataset there are no duplicates so it must be how our script creates the dataset from the database. Again if this is the case then an error stating that would be very helpful, as it is the data uploads (without error, I will check that) and training completes via the API, we can then deploy the trained model and infer against it. I will check the errors on our side now and get back to you. Thanks again.

[AdamMiltonBarker]

Hi @aahill I am still investingating but we don't get an error response from the initial upload.

[{"http_code":"HTTP\/2 202 ","content-length":"0","access-control-expose-headers":"*","request-id":"21a322bb-ec92-4336-b11d-c94eaebe4f4c","apim-request-id":"21a322bb-ec92-4336-b11d-c94eaebe4f4c","operation-location":"https:\/\/intellistore.cognitiveservices.azure.com\/language\/authoring\/analyze-conversations\/projects\/intellistore\/import\/jobs\/304155f7-9bbd-4ff9-a9a1-d5dd1523ff37_638400096000000000?api-version=2023-04-01","x-envoy-upstream-service-time":"838","strict-transport-security":"max-age=31536000; includeSubDomains; preload","x-content-type-options":"nosniff","x-ms-region":"West Europe","date":"Fri, 05 Jan 2024 21:28:42 GMT"},null,202]

UPDATE. This was the issue, I found the duplicate in our database, once replaced it has the correct amount now. This should return an error.

[aahill]

Great to hear, and thanks for that feedback as well, I'll pass that to the product team as well.

please-close