search

Azure / azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.
MIT License
4.59k stars 2.8k forks source link

Export and download custom vision model error;- is already queued for export #23922

Closed agribot2 closed 2 years ago

agribot2 commented 2 years ago

Describe the bug I am using an existing iteration in an existing custom vision project that has already correctly trained , classified images, etc I am trying download the azure customer vision model using python.

But the response is an error message that says ;- eaf41 redacted-3675c is already queued for export.

To Reproduce Steps to reproduce the behaviour:

1) Run the following python code (Taken from the Microsoft site ;- [https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/export-programmatically](https://github.com/MicrosoftDocs/azure-docs/issues/url)

from azure.cognitiveservices.vision.customvision.training import CustomVisionTrainingClient
from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient
from azure.cognitiveservices.vision.customvision.training.models import ImageFileCreateBatch, ImageFileCreateEntry, Region
from msrest.authentication import ApiKeyCredentials
import time
import requests

ENDPOINT ="https://redacted.cognitiveservices.azure.com/"  # redacted
training_key = "b82b6redacted44e"
prediction_key = "cb66credactedc38b58"
credentials = ApiKeyCredentials(in_headers={"Training-key": training_key})
trainer = CustomVisionTrainingClient(ENDPOINT, credentials)

project_id = 'bd9b1397-redacted-7fc4968c31d6'
iteration_id = '2c398dd6-redacted-c65fb0e939ab'
platform="TensorFlow"
flavour="TensorFlowNormal"

export = trainer.export_iteration(project_id, iteration_id, platform, flavour, raw=False) # This does NOT work . The error says ...already queued for exporting 

export_status = trainer.ExportStatus()
while (export.status == "Exporting"):
    print ("Waiting 10 seconds...")
    time.sleep(10)

exports = trainer.get_exports(project_id, iteration_id)  
    # Locate the export for this iteration and check its status  
for e in exports:
    if e.platform == export.platform and e.flavor == export.flavor:
        export = e
        break
print("Export status is: ", export.status)
if export.status == "Done":
    # Success, now we can download it
    export_file = requests.get(export.download_uri)
    with open("export.zip", "wb") as file:
        file.write(export_file.content)

Actual behaviour / result;- The cloud based system causes the following error message ;- eaf41-redacted-3675c is already queued for export. No download is started.

Also, if the following different code is run MORE THAN ONCE, it only works as required the first time, but fails when run again. Taken from https://stackoverflow.com/questions/61097420/question-model-dowloading-with-custom-vision-python-api

#Always takes the newest project and its newest iteration
iterations = c_plat.get_iterations(projects[0].id)
response = c_plat.export_iteration(project_id=projects[0].id, iteration_id=iterations[0].id, platform = "DockerFile", raw=False, flavor="ARM")

import webbrowser
webbrowser.open(c_plat.get_exports(project_id=projects[0].id, iteration_id=iterations[0].id)[0].download_uri)

Expected behaviour The system should download the iteration model to the user.

Screenshots N/A

Additional context The other method of downloading a model;- Manually via the web portal, works correctly. That interface works correctly even when tested using the same endpoint, keys, id's, project, training, publish and export functions that the python code targets.

mccoyp commented 2 years ago

Hi @agribot2, thank you for opening an issue! I'm tagging some folks who should be able to help. For more context about the issue: https://github.com/MicrosoftDocs/azure-docs/issues/91074

agribot2 commented 2 years ago

Hi, Has this defect been considered by the team ? Has any decision been made about what to do next ? Thanks

SatishBoddu-MSFT commented 2 years ago

@agribot2 Thanks for the feedback! We are currently investigating and will update you shortly.

SaurabhSharma-MSFT commented 2 years ago

@agribot2 I could see the same and it only runs for the first time. I am checking internally and get back to you.

shonohs commented 2 years ago

I think the behavior is by design. If export_iteration was called twice, it will throw error for the second call. I guess we need two fixes; 1) fix the document so that the sample code calls get_export first to check if the iteration was already exported. 2) Fix the API error message. "...is already queued for export..." is not very clear.

PatrickFarley commented 2 years ago

@shonohs what steps are needed to re-export an iteration? If we check, and it was already exported, how do we export again?

shonohs commented 2 years ago

You can use get_exports() to get a link to an exported model. We don't have API to force re-generate it. (Even if we had the API, new exported model will be exactly same as the old one.)

ghost commented 2 years ago

Hi @agribot2. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

ghost commented 2 years ago

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @areddish, @tburns10.

Issue Details
- **Package Name**: azure.cognitiveservices.vision.customvision - **Package Version**: Here is a list of the python packages being used;- adal==1.2.6 applicationinsights==0.11.9 argon2-cffi==20.1.0 async-generator==1.10 attrs==20.3.0 azure-batch==10.0.0 azure-cognitiveservices-vision-customvision==3.1.0 azure-common==1.1.26 azure-core==1.10.0 azure-graphrbac==0.61.1 azure-mgmt-authorization==0.61.0 azure-mgmt-containerregistry==2.8.0 azure-mgmt-keyvault==2.2.0 azure-mgmt-resource==12.0.0 azure-mgmt-storage==11.2.0 azureml==0.2.7 azureml-core==1.22.0 azureml-telemetry==1.22.0 azureml-widgets==1.22.0 azure-nspkg==3.0.2 azure-storage-blob==12.7.1 backcall==0.2.0 backports.tempfile==1.0 backports.weakref==1.0.post1 bleach==3.3.0 certifi==2020.12.5 cffi==1.14.4 chardet==4.0.0 colorama==0.4.4 contextlib2==0.6.0.post1 cryptography==3.3.1 cycler==0.10.0 decorator==4.4.2 defusedxml==0.6.0 docker==4.4.1 entrypoints==0.3 fsspec==0.8.5 idna==2.10 importlib-metadata==3.4.0 install==1.3.4 ipykernel==5.4.3 ipython==7.20.0 ipython-genutils==0.2.0 ipywidgets==7.6.3 isodate==0.6.0 jedi==0.18.0 jeepney==0.6.0 Jinja2==2.11.3 jmespath==0.10.0 jsonpickle==1.5.0 jsonschema==3.2.0 jupyter-client==6.1.11 jupyter-core==4.7.1 jupyterlab-pygments==0.1.2 jupyterlab-widgets==1.0.0 kaggle==1.5.10 kiwisolver==1.3.1 MarkupSafe==1.1.1 matplotlib==3.3.4 mistune==0.8.4 msrest==0.6.21 msrestazure==0.6.4 nbclient==0.5.2 nbconvert==6.0.7 nbformat==5.1.2 ndg-httpsclient==0.5.1 nest-asyncio==1.5.1 notebook==6.2.0 numpy==1.19.5 oauthlib==3.1.0 opencv-contrib-python==4.5.1.48 packaging==20.9 pandas==1.2.1 pandocfilters==1.4.3 parso==0.8.1 pathspec==0.8.1 pickleshare==0.7.5 Pillow==8.1.0 pip==21.0 prometheus-client==0.9.0 prompt-toolkit==3.0.16 pyasn1==0.4.8 pycparser==2.20 Pygments==2.8.0 PyJWT==1.7.1 pyOpenSSL==20.0.1 pyparsing==2.4.7 pyrsistent==0.17.3 python-dateutil==2.8.1 python-slugify==4.0.1 pytz==2020.5 pywin32==227 pywinpty==0.5.7 pyzmq==22.0.3 requests==2.25.1 requests-oauthlib==1.3.0 ruamel.yaml.clib==0.2.2 ruamel.yaml==0.16.12 SecretStorage==3.3.0 Send2Trash==1.5.0 setuptools==47.1.0 six==1.15.0 terminado==0.9.2 testpath==0.4.4 text-unidecode==1.3 torch==1.7.1 torchvision==0.4.1 tornado==6.1 tqdm==4.56.2 traitlets==5.0.5 typing-extensions==3.7.4.3 urllib3==1.26.3 wcwidth==0.2.5 webencodings==0.5.1 websocket-client==0.57.0 widgetsnbextension==3.5.1 zipp==3.4.0 - **Operating System**: Using windows 10 on a intel based laptop. Using Visual Studio 2019, version 16.11.10 - **Python Version**: Python 3.7 **Describe the bug** I am using an existing iteration in an existing custom vision project that has already correctly trained , classified images, etc I am trying download the azure customer vision model using python. **But the response is an error message that says ;- eaf41 redacted-3675c is already queued for export.** **To Reproduce** Steps to reproduce the behaviour: 1) Run the following python code (Taken from the Microsoft site ;- [[https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/export-programmatically](https://github.com/MicrosoftDocs/azure-docs/issues/url)](url) ``` from azure.cognitiveservices.vision.customvision.training import CustomVisionTrainingClient from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient from azure.cognitiveservices.vision.customvision.training.models import ImageFileCreateBatch, ImageFileCreateEntry, Region from msrest.authentication import ApiKeyCredentials import time import requests ENDPOINT ="https://redacted.cognitiveservices.azure.com/" # redacted training_key = "b82b6redacted44e" prediction_key = "cb66credactedc38b58" credentials = ApiKeyCredentials(in_headers={"Training-key": training_key}) trainer = CustomVisionTrainingClient(ENDPOINT, credentials) project_id = 'bd9b1397-redacted-7fc4968c31d6' iteration_id = '2c398dd6-redacted-c65fb0e939ab' platform="TensorFlow" flavour="TensorFlowNormal" export = trainer.export_iteration(project_id, iteration_id, platform, flavour, raw=False) # This does NOT work . The error says ...already queued for exporting export_status = trainer.ExportStatus() while (export.status == "Exporting"): print ("Waiting 10 seconds...") time.sleep(10) exports = trainer.get_exports(project_id, iteration_id) # Locate the export for this iteration and check its status for e in exports: if e.platform == export.platform and e.flavor == export.flavor: export = e break print("Export status is: ", export.status) if export.status == "Done": # Success, now we can download it export_file = requests.get(export.download_uri) with open("export.zip", "wb") as file: file.write(export_file.content) ``` Actual behaviour / result;- **The cloud based system causes the following error message ;- eaf41-redacted-3675c is already queued for export.** No download is started. Also, if the following different code is run MORE THAN ONCE, it only works as required the first time, but fails when run again. Taken from [https://stackoverflow.com/questions/61097420/question-model-dowloading-with-custom-vision-python-api](url) ``` #Always takes the newest project and its newest iteration iterations = c_plat.get_iterations(projects[0].id) response = c_plat.export_iteration(project_id=projects[0].id, iteration_id=iterations[0].id, platform = "DockerFile", raw=False, flavor="ARM") import webbrowser webbrowser.open(c_plat.get_exports(project_id=projects[0].id, iteration_id=iterations[0].id)[0].download_uri) ``` **Expected behaviour** The system should download the iteration model to the user. **Screenshots** N/A **Additional context** The other method of downloading a model;- Manually via the web portal, works correctly. That interface works correctly even when tested using the same endpoint, keys, id's, project, training, publish and export functions that the python code targets.
Author: agribot2
Assignees: lmazuel, xiangyan99
Labels: `question`, `Service Attention`, `Client`, `customer-reported`, `Cognitive - Custom Vision`, `issue-addressed`
Milestone: -
agribot2 commented 2 years ago

Thank you for the suggestion that users must use the get_exports() method as a work-around for this defect. Allow me to say...

Having an API that work only once and fails if run a second time is poor quality.

However, given that there are work-arounds available I can understand the decision. I do hope that the team that manages the tutorial and example code at [https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/export-programmatically](https://github.com/MicrosoftDocs/azure-docs/issues/url) now adds information about the defect and also provides example code that uses the get_exports() method to avoid the defect.

Thank you for clarifying the work-around.

ghost commented 2 years ago

Hi @agribot2. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

ghost commented 2 years ago

Hi @agribot2, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.