MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.25k stars 21.42k forks source link

Training error #20939

Closed danieltanasec closed 5 years ago

danieltanasec commented 5 years ago

When I run the script for my own classes I get an error. I made sure that the training key and images url are correct but could't solve the problem. I should also specify that the images are local

File "C:\Users\danbe\AppData\Local\Programs\Python\Python36\lib\site-packages\azure\cognitiveservices\vision\customvision\training\custom_vision_training_client.py", line 1018, in create_images_from_urls raise HttpOperationError(self._deserialize, response) msrest.exceptions.HttpOperationError: Operation returned an invalid status code 'Bad Request'


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

YutongTie-MSFT commented 5 years ago

@Tzuya14 Thanks for your feedback, I will investigate and get back to you soon.

areddish commented 5 years ago

Hi @Tzuya14, this error message usually indicates you've hit the project limit. If you navigate to https://customvision.ai and click on the gear icon in the upper right it should take you to a page that lists your limits.

Towards the bottom left under the training key you should see text of the form "XX projects created; YY remain"

danieltanasec commented 5 years ago

@areddish Hi, I checked the limits and I have "1 projects created; 99 remain" and Unlimited predictions

areddish commented 5 years ago

@Tzuya14 sorry, I see now that the bad request was from create_images_from_urls so you were successful in creating the project. Were any of the images uploaded to the project in the portal? Or did it fail on the first one?

You said "the images are local". Does that mean you are passing "file://" urls? Or how are you specifying the url?

For local files we there is create_images_from_files. I can provide you a sample for that if it helps.

danieltanasec commented 5 years ago

@areddish When I check the web page the project is created but no pictures are uploaded. I just declared the url as "data/images/image.jpg". I was thinking maybe here is the problem. The code I used is:

trainer = CustomVisionTrainingClient(training_key, endpoint=ENDPOINT)

# Create a new project
print("Creating project...")
project = trainer.create_project("Man_vs_Woman")

# Make two tags in the new project
man_tag = trainer.create_tag(project.id, "Man")
woman_tag = trainer.create_tag(project.id, "Woman")

man_images_list = os.listdir("data/train/man/")
woman_images_list = os.listdir("data/train/woman/")

print("Adding images...")
for image_num in man_images_list:
    image_url = "data/train/man/"+image_num
    trainer.create_images_from_urls(project.id, [ImageUrlCreateEntry(url=image_url, tag_ids=[man_tag.id])])

for image_num in woman_images_list:
    image_url = "data/train/woman/"+image_num
    trainer.create_images_from_urls(project.id, [ImageUrlCreateEntry(url=image_url, tag_ids=[woman_tag.id])])

print("Training...")
iteration = trainer.train_project(project.id)
while (iteration.status != "Completed"):
    iteration = trainer.get_iteration(project.id, iteration.id)
    print("Training status: " + iteration.status)
    time.sleep(1)

# The iteration is now trained. Make it the default project endpoint
trainer.update_iteration(project.id, iteration.id, is_default=True)
print ("Done!")
areddish commented 5 years ago

Ah yes, that will result in an invalid url to the service, hence the bad request. You want to use create_images_from_files. Something like this:

trainer = CustomVisionTrainingClient(training_key, endpoint=ENDPOINT)

# Create a new project
print("Creating project...")
project = trainer.create_project("Man_vs_Woman")

# Make two tags in the new project
man_tag = trainer.create_tag(project.id, "Man")
woman_tag = trainer.create_tag(project.id, "Woman")

man_images_list = os.listdir("data/train/man/")
woman_images_list = os.listdir("data/train/woman/")

print("Adding images...")
for image_num in man_images_list:
    file_name = "data/train/man/"+image_num
    with open(file_name, mode="rb") as image_contents:
        trainer.create_images_from_files(project.id, images=[ImageFileCreateEntry(name=file_name, contents=image_contents.read()], tag_ids=[man_tag.id])])

for image_num in woman_images_list:
    file_name = "data/train/woman/"+image_num
    with open(file_name, mode="rb") as image_contents:
        trainer.create_images_from_files(project.id, images=[ImageFileCreateEntry(name=file_name, contents=image_contents.read()], tag_ids=[woman_tag.id])])

print("Training...")
iteration = trainer.train_project(project.id)
while (iteration.status != "Completed"):
    iteration = trainer.get_iteration(project.id, iteration.id)
    print("Training status: " + iteration.status)
    time.sleep(1)

# The iteration is now trained. Make it the default project endpoint
trainer.update_iteration(project.id, iteration.id, is_default=True)
print ("Done!")
danieltanasec commented 5 years ago

@areddish Thank a lot! It works now

areddish commented 5 years ago

@Tzuya14 when you train a model you get back an iteration. You can pass that iteration in the predict image call to target that specific model iteration. The above code will set the latest trained iteration to be the default which is what predict_image will use if you don't pass an explicit iteration.

If you want to re-use an iteration in a separate program (or without retraining) you just need to save off the project id and iteration id and then re-use them in the predict_image call:

predictor = CustomVisionPredictionClient(prediction_key, endpoint=ENDPOINT)

man_images_list = os.listdir("data/validation/man/")

print("Adding images...")
# Fill this in with your project id returned from create_project. Alternatively, you can create a 
# CustomVisionTrainingClient and then call get_projects and filtered that by the project if
# only have the name.
my_project_id = "<your project id>"
# Fill this in with your iteration id returned from train_project or get_iteration. You can get a list of your 
# iterations by calling trainer.get_iterations(my_project_id).
my_iteration_id = "<your iteration id>"
for image_num in man_images_list:
    file_name = "data/validation/man/"+image_num
    with open(file_name, mode="rb") as test_data:
        results = predictor.predict_image(my_project_id, test_data, iteration_id=my_iteration_id)
        print(results)