search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.26k stars 21.43k forks source link

Actual API response for Computer Vision Product Recognition is different from docs. #120464

Closed macieyng closed 4 months ago

macieyng commented 7 months ago

Endpoint GET https://***.cognitiveservices.azure.com/computervision/productrecognition/ms-pretrained-product-detection/runs/***?api-version=2023-04-01-preview returns something different from what was described in article https://learn.microsoft.com/en-us/azure/ai-services/computer-vision/how-to/shelf-analyze

API Reference has basically no description of Schema https://eastus.dev.cognitive.microsoft.com/docs/services/unified-vision-apis-public-preview-2023-04-01-preview/operations/644aba14fb42681ae06f1b0c

Diff TL;DR by ChatGPT

  1. Structure of "result" Property: 1.1. First Schema: Uses "tags" for both "products" and "gaps" under the "result" property. 1.2. Second Schema: Uses "classifications" for both "products" and "gaps" under the "result" property.
  2. Root Level Properties: 2.1. First Schema: Includes additional properties at the root level - "runName," "modelName," "createdDateTime," "updatedDateTime," "status." 2.2. Second Schema: Does not include the additional root level properties present in the first schema.
  3. Different Naming for "tags" and "classifications": 3.1. First Schema: Uses "name" and "confidence" for "tags" properties. 3.2. Second Schema: Uses "label" and "confidence" for "classifications" properties.
  4. Simplification in Second Schema: 4.2 The second schema simplifies the structure by using consistent "classifications" for both "products" and "gaps" instead of having different structures for "tags" under "products" and "gaps" as in the first schema.

Here are JSON Schemas for both.

Actual:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Generated schema for Root",
  "type": "object",
  "properties": {
    "runName": {
      "type": "string"
    },
    "modelName": {
      "type": "string"
    },
    "createdDateTime": {
      "type": "string"
    },
    "updatedDateTime": {
      "type": "string"
    },
    "status": {
      "type": "string"
    },
    "result": {
      "type": "object",
      "properties": {
        "imageMetadata": {
          "type": "object",
          "properties": {
            "width": {
              "type": "number"
            },
            "height": {
              "type": "number"
            }
          },
          "required": [
            "width",
            "height"
          ]
        },
        "products": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "id": {
                "type": "string"
              },
              "boundingBox": {
                "type": "object",
                "properties": {
                  "x": {
                    "type": "number"
                  },
                  "y": {
                    "type": "number"
                  },
                  "w": {
                    "type": "number"
                  },
                  "h": {
                    "type": "number"
                  }
                },
                "required": [
                  "x",
                  "y",
                  "w",
                  "h"
                ]
              },
              "tags": {
                "type": "array",
                "items": {
                  "type": "object",
                  "properties": {
                    "name": {
                      "type": "string"
                    },
                    "confidence": {
                      "type": "number"
                    }
                  },
                  "required": [
                    "name",
                    "confidence"
                  ]
                }
              }
            },
            "required": [
              "id",
              "boundingBox",
              "tags"
            ]
          }
        },
        "gaps": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "id": {
                "type": "string"
              },
              "boundingBox": {
                "type": "object",
                "properties": {
                  "x": {
                    "type": "number"
                  },
                  "y": {
                    "type": "number"
                  },
                  "w": {
                    "type": "number"
                  },
                  "h": {
                    "type": "number"
                  }
                },
                "required": [
                  "x",
                  "y",
                  "w",
                  "h"
                ]
              },
              "tags": {
                "type": "array",
                "items": {
                  "type": "object",
                  "properties": {
                    "name": {
                      "type": "string"
                    },
                    "confidence": {
                      "type": "number"
                    }
                  },
                  "required": [
                    "name",
                    "confidence"
                  ]
                }
              }
            },
            "required": [
              "id",
              "boundingBox",
              "tags"
            ]
          }
        }
      },
      "required": [
        "imageMetadata",
        "products",
        "gaps"
      ]
    }
  },
  "required": [
    "runName",
    "modelName",
    "createdDateTime",
    "updatedDateTime",
    "status",
    "result"
  ]
}

Declared:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Generated schema for Root",
  "type": "object",
  "properties": {
    "imageMetadata": {
      "type": "object",
      "properties": {
        "width": {
          "type": "number"
        },
        "height": {
          "type": "number"
        }
      },
      "required": [
        "width",
        "height"
      ]
    },
    "products": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "id": {
            "type": "string"
          },
          "boundingBox": {
            "type": "object",
            "properties": {
              "x": {
                "type": "number"
              },
              "y": {
                "type": "number"
              },
              "w": {
                "type": "number"
              },
              "h": {
                "type": "number"
              }
            },
            "required": [
              "x",
              "y",
              "w",
              "h"
            ]
          },
          "classifications": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "confidence": {
                  "type": "number"
                },
                "label": {
                  "type": "string"
                }
              },
              "required": [
                "confidence",
                "label"
              ]
            }
          }
        },
        "required": [
          "id",
          "boundingBox",
          "classifications"
        ]
      }
    },
    "gaps": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "id": {
            "type": "string"
          },
          "boundingBox": {
            "type": "object",
            "properties": {
              "x": {
                "type": "number"
              },
              "y": {
                "type": "number"
              },
              "w": {
                "type": "number"
              },
              "h": {
                "type": "number"
              }
            },
            "required": [
              "x",
              "y",
              "w",
              "h"
            ]
          },
          "classifications": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "confidence": {
                  "type": "number"
                },
                "label": {
                  "type": "string"
                }
              },
              "required": [
                "confidence",
                "label"
              ]
            }
          }
        },
        "required": [
          "id",
          "boundingBox",
          "classifications"
        ]
      }
    }
  },
  "required": [
    "imageMetadata",
    "products",
    "gaps"
  ]
}

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

SaibabaBalapur-MSFT commented 7 months ago

@macieyng Thanks for your feedback! We will investigate and update as appropriate.

Naveenommi-MSFT commented 7 months ago

@macieyng Thank you for bringing this to our attention. I've delegated this to content author @PatrickFarley, who will review it and offer their insightful opinions.

Naveenommi-MSFT commented 7 months ago

@PatrickFarley Could you please review add comments on this, update as appropriate.

PatrickFarley commented 4 months ago

Thank you @macieyng for spotting this. It appears we had old content from a preview version of the API. I'm updating the doc now. The reference doc that you linked is accurate, and you can view the full schema in this repo: https://github.com/Azure/azure-rest-api-specs/blob/main/specification/cognitiveservices/data-plane/UnifiedVision/preview/2023-04-01-preview/UnifiedVision.json

PatrickFarley commented 4 months ago

please-close