Kaggle / kaggle-api

Official Kaggle API
Apache License 2.0
6.01k stars 1.06k forks source link

listing model instances doesn't return previous versions #586

Open waseem-mend opened 1 month ago

waseem-mend commented 1 month ago

When requesting info about model instances using Kaggle API, it returns a single result of the latest (current) version. I would expect that the API response will contain all versions.

For example:

Model request:

cURL request: curl https://www.kaggle.com/api/v1/models/tensorflow/ssd-mobilenet-v1/get -u *****:*****

response:

{
  "id": 299,
  "ref": "tensorflow/ssd-mobilenet-v1",
  "title": "ssd_mobilenet_v1",
  "subtitle": "Object detection model trained on the COCO dataset.",
  "author": "TensorFlow",
  "slug": "ssd-mobilenet-v1",
  "isPrivate": false,
  "description": "## Overview\n\nAn object detection model that has been released in the\n[Tensorflow detection model zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1_detection_zoo.md)\nand trained on the [COCO dataset](http://cocodataset.org).\n\nThis model is available at http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_2018_01_28.tar.gz.\n\nThe model is useful for out-of-the-box inference, but also useful for\ninitializing the models when training on novel datasets.",
  "instances": [
    {
      "hasBaseModelInstanceInformation": false,
      "id": 2478,
      "slug": "fpn-640x640",
      "framework": "tensorFlow2",
      "fineTunable": false,
      "overview": "SSD Mobilenet V1 Object detection model with FPN feature extractor, shared box\npredictor and focal loss, trained on COCO 2017 dataset with trainning images\nscaled to 640x640.",
      "usage": "## Overview\n\nSSD with Mobilenet V1 FPN feature extractor, shared box predictor and focal loss\n(a mobile version of [Retinanet in Lin et al](https://arxiv.org/abs/1708.02002))\ninitialized from Imagenet classification checkpoint.\n\nTrained on [COCO 2017](https://cocodataset.org/) dataset (images scaled to\n640x640 resolution).\n\nModel created using the\n[TensorFlow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection)\n\nAn example detection result is shown below.\n\n![Object detector output](https://www.gstatic.com/aihub/tfhub/detection/od_no_keypoints.png)\n\n#### Example use\n\n```\n# Apply image detector on a single image.\ndetector = hub.load(\u0022${URL}\u0022)\ndetector_output = detector(image_tensor)\nclass_ids = detector_output[\u0022detection_classes\u0022]\n```\n\n### Inputs\n\nA three-channel image of variable size - the model does **NOT** support\nbatching. The input tensor is a `tf.uint8` tensor with shape `[1, height, width,\n3]` with values in `[0, 255]`.\n\n### Outputs\n\nThe output dictionary contains:\n\n*   `num_detections`: a `tf.int` tensor with only one value, the number of\n    detections `[N]`.\n*   `detection_boxes`: a `tf.float32` tensor of shape `[N, 4]` containing\n    bounding box coordinates in the following order: `[ymin, xmin, ymax, xmax]`.\n*   `detection_classes`: a `tf.int` tensor of shape `[N]` containing detection\n    class index from the label file.\n*   `detection_scores`: a `tf.float32` tensor of shape `[N]` containing\n    detection scores.\n*   `raw_detection_boxes`: a `tf.float32` tensor of shape `[1, M, 4]` containing\n    decoded detection boxes without Non-Max suppression. `M` is the number of\n    raw detections.\n*   `raw_detection_scores`: a `tf.float32` tensor of shape `[1, M, 90]` and\n    contains class score logits for raw detection boxes. `M` is the number of\n    raw detections.\n*   `detection_anchor_indices`: a `tf.float32` tensor of shape `[N]` and\n    contains the anchor indices of the detections after NMS.\n*   `detection_multiclass_scores`: a `tf.float32` tensor of shape `[1, N, 91]`\n    and contains class score distribution (including background) for detection\n    boxes in the image including background class.\n\n#### Source\n\nThe model\u0027s checkpoints are\n[publicly available](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md)\nas a part of the\n[TensorFlow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection).\n\n#### Metrics\n\nMetric                    | Value | Outputs\n------------------------- | ----- | -------\nmAP on COCO 2017 test set | 29.1  | Boxes",
      "downloadUrl": "/models/tensorflow/ssd-mobilenet-v1/TensorFlow2/fpn-640x640/1/download",
      "versionId": 3324,
      "versionNumber": 1,
      "trainingData": [

      ],
      "url": "https://www.kaggle.com/models/tensorflow/ssd-mobilenet-v1/TensorFlow2/fpn-640x640",
      "licenseName": "Apache 2.0",
      "modelInstanceType": "unspecified",
      "baseModelInstanceInformation": null,
      "externalBaseModelUrl": ""
    },
    {
      "hasBaseModelInstanceInformation": false,
      "id": 2475,
      "slug": "default",
      "framework": "tfLite",
      "fineTunable": false,
      "overview": "TF Lite deployment of tensorflow/ssd_mobilenet_v1/1.",
      "usage": "## Description\nPre-trained model optimized to work with TensorFlow Lite for Object detection.\n\nSee\n[Object detection overview](https://www.tensorflow.org/lite/examples/object_detection/overview)\npage for documentation and examples.",
      "downloadUrl": "/models/tensorflow/ssd-mobilenet-v1/TfLite/default/1/download",
      "versionId": 3320,
      "versionNumber": 1,
      "trainingData": [

      ],
      "url": "https://www.kaggle.com/models/tensorflow/ssd-mobilenet-v1/TfLite/default",
      "licenseName": "Apache 2.0",
      "modelInstanceType": "unspecified",
      "baseModelInstanceInformation": null,
      "externalBaseModelUrl": ""
    },
    {
      "hasBaseModelInstanceInformation": false,
      "id": 2477,
      "slug": "metadata",
      "framework": "tfLite",
      "fineTunable": false,
      "overview": "TF Lite deployment of tensorflow/ssd_mobilenet_v1/2.",
      "usage": "## Description\n\nPre-trained model optimized to work with TensorFlow Lite for Object detection.\nThis model contains both TFLite model metadata and the label file.\n[TFLite metadata](https://www.tensorflow.org/lite/convert/metadata) is a rich\nmodel description including both human and machine readable information.\n\nSee\n[Object detection overview](https://www.tensorflow.org/lite/examples/object_detection/overview)\npage for documentation and examples.\n\n### Release Notes\n\nv2 - updated the models with better metadata information",
      "downloadUrl": "/models/tensorflow/ssd-mobilenet-v1/TfLite/metadata/2/download",
      "versionId": 3323,
      "versionNumber": 2,
      "trainingData": [

      ],
      "url": "https://www.kaggle.com/models/tensorflow/ssd-mobilenet-v1/TfLite/metadata",
      "licenseName": "Apache 2.0",
      "modelInstanceType": "unspecified",
      "baseModelInstanceInformation": null,
      "externalBaseModelUrl": ""
    },
    {
      "hasBaseModelInstanceInformation": false,
      "id": 2476,
      "slug": "default",
      "framework": "tfJs",
      "fineTunable": false,
      "overview": "TF.js deployment of tensorflow/ssd_mobilenet_v1/1.",
      "usage": "## Origin\n\nThis model is published on NPM as `@tensorflow-models/coco-ssd`.\n\nSee the [NPM documentation](https://www.npmjs.com/package/@tensorflow-models/coco-ssd)\nfor how to load and use the model in JavaScript.",
      "downloadUrl": "/models/tensorflow/ssd-mobilenet-v1/TfJs/default/1/download",
      "versionId": 3321,
      "versionNumber": 1,
      "trainingData": [

      ],
      "url": "https://www.kaggle.com/models/tensorflow/ssd-mobilenet-v1/TfJs/default",
      "licenseName": "Apache 2.0",
      "modelInstanceType": "unspecified",
      "baseModelInstanceInformation": null,
      "externalBaseModelUrl": ""
    }
  ],
  "tags": [
    {
      "nameNullable": "image",
      "descriptionNullable": null,
      "fullPathNullable": "data type \u003e image",
      "ref": "image",
      "name": "image",
      "hasName": true,
      "description": "",
      "hasDescription": false,
      "fullPath": "data type \u003e image",
      "hasFullPath": true,
      "competitionCount": 379,
      "datasetCount": 7233,
      "scriptCount": 5590,
      "totalCount": 13202
    },
    {
      "nameNullable": "object detection",
      "descriptionNullable": "",
      "fullPathNullable": "task \u003e object-detection",
      "ref": "object detection",
      "name": "object detection",
      "hasName": true,
      "description": "",
      "hasDescription": true,
      "fullPath": "task \u003e object-detection",
      "hasFullPath": true,
      "competitionCount": 5,
      "datasetCount": 480,
      "scriptCount": 231,
      "totalCount": 716
    }
  ],
  "publishTime": "2020-10-06T00:00:00Z",
  "provenanceSources": "https://github.com/tensorflow/tfhub.dev/tree/master/assets/docs/tensorflow",
  "url": "https://www.kaggle.com/models/tensorflow/ssd-mobilenet-v1"
}

Model instance request:

cURL request: curl https://www.kaggle.com/api/v1/models/tensorflow/ssd-mobilenet-v1/tfLite/metadata/get -u *****:****

response:

{
  "hasBaseModelInstanceInformation": false,
  "id": 2477,
  "slug": "metadata",
  "framework": "tfLite",
  "fineTunable": false,
  "overview": "TF Lite deployment of tensorflow/ssd_mobilenet_v1/2.",
  "usage": "## Description\n\nPre-trained model optimized to work with TensorFlow Lite for Object detection.\nThis model contains both TFLite model metadata and the label file.\n[TFLite metadata](https://www.tensorflow.org/lite/convert/metadata) is a rich\nmodel description including both human and machine readable information.\n\nSee\n[Object detection overview](https://www.tensorflow.org/lite/examples/object_detection/overview)\npage for documentation and examples.\n\n### Release Notes\n\nv2 - updated the models with better metadata information",
  "downloadUrl": "/models/tensorflow/ssd-mobilenet-v1/TfLite/metadata/2/download",
  "versionId": 3323,
  "versionNumber": 2,
  "trainingData": [

  ],
  "url": "https://www.kaggle.com/models/tensorflow/ssd-mobilenet-v1/TfLite/metadata",
  "licenseName": "Apache 2.0",
  "modelInstanceType": "unspecified",
  "baseModelInstanceInformation": null,
  "externalBaseModelUrl": ""
}

As you can see, the response contains only the latest version "versionNumber":2

stevemessick commented 3 weeks ago

I don't think we officially support using curl to download anything. It certainly is not part of the Kaggle API (this GitHub project).

However, listing model instance versions seems like a reasonable feature. I added a feature request to our internal issue tracker.

http://b/346568526