Image Classification ML service with SAPUI5

[voicedmendor]

I am having issue while uploading a goose image file. Getting the below error. I followed the needed steps outlined in this link tutorial.

https://www.sap.com/developer/tutorials/ml-fs-sapui5-img-classification.html

Appreciate your help.

Thanks.

[adadouche]

Hi,

I think you have missed to add the following section to your manifest file as described in step 9: "demo": { "type": "sap.ui.model.json.JSONModel", "preload": true, "uri": "model/demo.json" } Can yo check and let me know?

Thanks

@bdel

[voicedmendor]

Now I am getting the below error. I created another new application.

[voicedmendor]

I am uploading the below file for test.

[adadouche]

Hi,

Sorry for the late response, but with TechEd season running now I was pretty busy.

I tried your image and it worked with my code.

I have pushed the all project code in the repo now: https://github.com/SAPDocuments/Tutorials/tree/master/tutorials/ml-fs-sapui5-img-classification/sapui5ml-imageclassifier

Can you check and compare to yours?

Thanks

@bdel

[GauravSardana]

Hi Abdel,

Even i am facing the same issue which voicedmendor is facing. Error says: "Invalid request", "error_description": "This service requires at least 1 file. Please put your file(s) into the files field of the POST request", "processed_time": "Wed, 25 Oct 2017 17:28:54 GMT",

I have copied the exact code from your tutorial: https://www.sap.com/developer/tutorials/ml-fs-sapui5-img-classification.html

Please help.

Warm Regards,

Gaurav

[ericsolberg]

I'm having the same error, and have validated that I have the "demo" section in my models as described in step 9.

[adadouche]

Can you both provide the link of the original image you have been using? Git may have tempered with it when uploading it.

Can you start your browser developer tool (pressing F12), then switch to the "Network" tab? Click on XHR to filter the other request type Once there try to upload the picture again. in the console you should see an entry for the "inference_sync" call. Select it. Then on the right you can select "Headers".

The Request Payload should look like this:

------WebKitFormBoundaryBDoVmWHrIY0KYEMA
Content-Disposition: form-data; name="files"; filename="30873712-c8c22790-a2b3-11e7-8c9e-2bce5d239d92.jpg"
Content-Type: image/jpeg

------WebKitFormBoundaryBDoVmWHrIY0KYEMA
Content-Disposition: form-data; name="_charset_"

UTF-8
------WebKitFormBoundaryBDoVmWHrIY0KYEMA
Content-Disposition: form-data; name="files-data"

------WebKitFormBoundaryBDoVmWHrIY0KYEMA--

And on the Response tab you should have

{
  "_id": "d0b8a6d9-c036-42d1-a857-388e4a72c6f9", 
  "predictions": [
    {
      "name": "30873712-c8c22790-a2b3-11e7-8c9e-2bce5d239d92.jpg", 
      "results": [
        {
          "label": "goose", 
          "score": 0.999639
        }, 
        {
          "label": "crane", 
          "score": 0.000152
        }, 
        {
          "label": "black swan", 
          "score": 8.3e-05
        }, 
        {
          "label": "drake", 
          "score": 6.4e-05
        }, 
        {
          "label": "prairie chicken", 
          "score": 2.6e-05
        }
      ]
    }
  ], 
  "processed_time": "Thu, 26 Oct 2017 18:37:28 GMT", 
  "request": {
    "files": [
      "30873712-c8c22790-a2b3-11e7-8c9e-2bce5d239d92.jpg"
    ], 
    "options": {}, 
    "tenantName": "imgclassif-tech-user", 
    "texts": []
  }, 
  "status": "DONE", 
  "tenantName": "imgclassif-tech-user"
}

Can you share yours?

[ericsolberg]

As the request payload, I simply have: (image below) [object Object]

I was trying to figure out how to make the file upload control properly stringify the object into json, but haven't figured it out yet.

Here's a link to the image that I'm using: https://www.dropbox.com/s/6fn4ncz61adzm1i/image.jpeg?dl=0

[ericsolberg]

Oh and the response tab is: { "_id": "c25a7073-dfd2-4a3d-b80d-ad47a13f0bf0", "error": "Invalid request", "error_description": "This service requires at least 1 file. Please put your file(s) into the files field of the POST request", "processed_time": "Thu, 26 Oct 2017 20:54:50 GMT", "request": { "files": null, "options": {}, "tenantName": "imgclassif-tech-user", "texts": [] }, "status": "FAILED", "status_code": 400, "tenantName": "imgclassif-tech-user" }

[ericsolberg]

I get the same result with my project built through the tutorial, and from your project download from the link you provided.

[GauravSardana]

Me too getting nothing in Request Payload. Screenshot below:

[adadouche]

Hi @GauravSardana & @voicedmendor & @ericsolberg,

This is really weird!

I recreated the project from scratch and used your gif, but it still works for me.

In the browser development tools, can you check the "Disable cache" and try again?

Also I notice that you have 6 Request headers and 12 Response headers.

Can you provide the full request?

To do so, use a right click on the "inference_sync" call then "Save as HAR with Content". You can obfuscate your APIKey and other information from there before sharing.

Thanks

@bdel

[adadouche]

Few more question:

• which browser and browser are you using?
• what is your client operating system?

I'm using Chrome 61 on Windows 10.

Thanks

[GauravSardana]

Hi Abdel,

Below are the responses:

Response Header (6): Content-Length:477 Content-Type:application/json Date:Mon, 30 Oct 2017 11:23:27 GMT Server:SAP Strict-Transport-Security:max-age=31536000; includeSubDomains; preload X-Cnection: close

Request Headers (14): Accept:application/json Accept-Encoding:gzip, deflate, br Accept-Language:en-US,en;q=0.9 APIKey: --removed-- Cache-Control:no-cache Connection:keep-alive Content-Length:15 Content-Type:text/plain;charset=UTF-8 Cookie:oucrsqwegjpmigzontthwkrzk=bOxszNmu%2FPeya7MBUAjkc3sOO0AvIj0B3tKXO7pu752bNicxICRwTV5FOn8yGPE2y7TPlkKZ4BrNUdHey9Oxjc0GVkT8uN7caIoh%2BSY0JwiG9Z1RZWp2yBkl6ukodqkZ4h9NYrAZOHza9NHZBbOqhED7RXW9R2zrWw3521p%2FQxgCXubtHVVv3H0JWvKuxF7n87L18%2Brh%2FDTQuUmNMLiMlstQrVpPu5S9nMXRYK%2BSh1lIxAJ7pjsC7WuUSrjqYL7%2FKJfBoh3jvJuFgzezkpbIF%2FSk20Thie56wZy3RzCrpwoq%2FMy7r5xzlj94H1CbcqxUb%2F8jtkpos1OrUY%2BZBYktBnGC6X9mIOX8XGBTH0b76vjdBe9fw%2F3heeAh20iGuG4AY1n85FWjE4ou%2BNVGJGWcAA%3D%3D; JTENANTSESSIONID_p1941796292trial=KQmmRupLFXHAlbathznQSYn%2BMwcGu9EA96wgctwnm7Q%3D; BIGipServer~jpaas_folder~dispatcher.hanatrial.ondemand.com=!VTRzqTsSxxGv+q+wDhtcRsHHmTA76BB2Ybr58cAvtiuEkiXWR3K1wEnvfLCYIVrTB34EaOLHYwNkMg==; oucrsdyppieqilfdzexnmwhtx=0rnZar07Y0frCzFLhSGncLCjXXlRE6Qpb%2BGGgndJxSsFrVh3jkBYRmpdyB%2FD5MAoAQj5Wqcc20boyBB5UiDgQ0R5DIw2%2BlVD6QY2i2ja3sJRA2Ho0KAhrk2f6rSSEhzPxWG%2BshofTt%2FDKwv0K3%2BnPz7h4aFZKPou2aYxXlcbzTIjxRJi%2FcFE32UQgXLmTUH0x4wL3ctQFw%2FFm1YWCGrg2voZep0KC8a4VHN5UsLo5BtDQFJOYXGyzZs2MByvLkycw8DltVkYVJk5gTkpJQdO%2Fsrk%2Fid%2BGbDGOwCSbfNRNqKiIcjviRNXlzSEeCHCl7WlFAY8nTGj8zPJF8paBcY69nKpB4IAV6jdPRLOu1TtKIrVo4Bhlm%2FKaBuuDoLp2BJFBVm3NJFD9b2OMnKgtX3%2FpJGiEEU0pcy1npwnBv0qCprxB9W3Mrb9snjDuM1%2BrrXG; JSESSIONID=5B2D460031EF918D97BE08352BEA632548896DD624C2FAE396129975D7FC78B3 Host:webidetesting4844192-p1941796292trial.dispatcher.hanatrial.ondemand.com Origin:https://webidetesting4844192-p1941796292trial.dispatcher.hanatrial.ondemand.com Pragma:no-cache Referer:https://webidetesting4844192-p1941796292trial.dispatcher.hanatrial.ondemand.com/webapp/extended_runnable_file.html?hc_orionpath=%2Fp1941796292trial%24P1941796292-OrionContent%2Fsapui5ml-imageclassifier&origional-url=index.html&sap-ui-appCacheBuster=..%2F&sap-ui-xx-componentPreload=off User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.75 Safari/537.36

Browser Details: Google Chrome Version 62.0.3202.75 (Official Build) (64-bit)

Windows 10 Enterprise

Checked by selecting "Disable Cache" in developer tools, but no success :(

Regards,

Gaurav

[GauravSardana]

Hi Guys,

Anyone got the breakthrough in this ??

Gaurav

[adadouche]

Hi guys, sorry for the back and forth but this is a really strange issue related to SAPUI5.

Moreover, this works perfectly on 2 of my environments. Are you using any special chrome extensions? Anti virus that is a bit picky? Have you tried on a different machine?

I'll see if I can provide a different method to upload the file (not using the built in xhr function but via ajax instead).

Can you in the mean time try with one of your colleague or home machine to test the application and try a different browser?

I'll do my best to code something up next week, but we have SAP TechEd in Barcelona the week after so I might need to postpone a possible to the last week of November.

Regards

[swapnilspande]

Hi guys, Were you able to resolve the error? I'm getting the same error. I observed that Content-Type is defaulted to plaintext for our requests. I think it should be multipart/form-data instead.

[adadouche]

I'm back from Teched

[adadouche]

Hi guys, I'm back from TechEd (and a bit tired to be honest).

I have continued the investigation on this issue but really can't figure out why this works fine for me and plenty of people and not for you.

The FileUploader form generated source is like this:

enctype="multipart/form-data" 
method="post"
id="__xmlview0--idFileUpload-fu_form"
action="/ml/imageclassifier/inference_sync" 
target="__xmlview0--idFileUpload-frame"

Can you confirm that the source of your generated page uses enctype="multipart/form-data" ? You will have to use the "Elements" tab from the Developer tool to get the "real" source generated.

Can you tell me which version of SAPUI5 you are running? To do so, in the developer tools, type in the console tab: sap.ui.version

I'll work on an alternate piece of code using AJAX but this issue is making me crazy.

Regards

[swapnilspande]

Thanks, Abdel. I can only imagine how exhaustive the teched would have been. You will need to take some rest :)

About the strange issue, we are going through... I can also see the similar html code.

However, the **Request Header** always contains **Content-Type:text/plain;charset=UTF-8**. Can you please check how it looks for you? Our SAPUI5 version is "1.48.10". I'll try to share a simple index.html that is also giving similar error, albeit in different manner... Thanks, Swapnil.

[swapnilspande]

Hi Abdel,

I created a simple index.html file with below code and it gives expected output. I didn't provide any value for content-type and it is shown as Content-Type:multipart/form-data; boundary=----WebKitFormBoundaryDPSRNOnBSQ70U8x0.

However, if I uncomment the line xhr.setRequestHeader("Content-Type", "multipart/form-data"); which is trying to default content-type, it gives me the same error(as mentioned in above messages). The content-type is shown as Content-Type:multipart/form-data.

As mentioned in previous message, content-type is getting defaulted to text/plain;charset=UTF-8 in my Web IDE application. I think this is the parameter we'll have to concentrate on.

Do let me know your thoughts. Thanks very much.

Below is the sample code for index.html. `<!DOCTYPE html>

`

[adadouche]

Hi,

The Content-Type contains both the "multipart/form-data" but also the boundary information which is used in the request payload.

This is why is you set Content-Type to only "multipart/form-data", then the request will be missing the boundary piece.

I started debugging the FileUploader (sap.ui.unified.FileUploader) class where the _sendFilesWithXHR function is defined, but I cannot figure out why the headers get modified in your environment and not mine.

Can you add a few break points there and check what is happening?

Also can you confirm whether or not the mock server is enable on your run configuration?

I noticed that people are running extended_runnable_file.html instead of the index.html.

What I also noticed is that if you have the Mock Server enabled, all xhr request are intercepted and passed thru the MockServer (sap.ui.core.util.MockServer). And there is plenty of changes on the content type.

@voicedmendor, @GauravSardana, @ericsolberg,

• Can you share / re-share your current machine operating system and browser and you physical location (EMEA, NA, APAC)?
• Can verify that the mock data feature is currently disabled in your run configuration (this should allow you to run the index.html, and not the extended_runnable_file.html)

The good news is that one of my colleague in the US was having the issue yesterday.

Thanks

@bdel

[soumyanandi]

Hi Abdel,

I am facing the same issue too. The payload is going as [object Object] and Content-Type is taking as text/plain;charset=UTF-8. I am running the application in the system as mentioned below,

a) Operating System: Windows 7 b) Browser: Chrome v62.0.3202.94 c) SAPUI5 webide version: 1.48.10 d) Mock data feature is not enabled. e) Physical Location: APAC

Can you please help regarding this issue. Let me know if you need any further details from my side.

Regards, Soumya

[adadouche]

hi @voicedmendor, @GauravSardana, @ericsolberg, @soumyanandi, @swapnilspande,

I just uploaded an alternate piece of code where I directly use XHR to upload the picture to the service. Can you guys give it a try?

Thanks again for your patience

[adadouche]

hi @voicedmendor, @GauravSardana, @ericsolberg, @soumyanandi, @swapnilspande,

Did you have a chance to test the new code?

[swapnilspande]

Hi Abdel, Yes, in my case, the issue was due to extended_runnable_file.html file. When I manually replaced it with index.html, even the Built-in XHR worked. Thanks a lot for that pointer. It might be helpful to add another footnote in your tutorial, regarding the use of index.html for testing.

Thanks for the custom XHR code as well. It is useful while trying to use other services which need multiple formdata objects(like OCR).

Regards, Swapnil.

[rosshigh]

I seem to be having the same problem. I get a [Object object] for the Request Header and I get this in the response: { "_id": "f25eb46b-d9f9-4682-b3b9-fc3801114d26", "error": "Invalid request", "error_description": "This service requires at least 1 file. Please put your file(s) into thefilesfield of the POST request", "processed_time": "Mon, 11 Dec 2017 18:22:28 GMT", "request": { "files": null, "options": {}, "tenantName": "imgclassif-tech-user", "texts": [] }, "status": "FAILED", "status_code": 400, "tenantName": "imgclassif-tech-user" }

I'm running index.html Chrome: 62.0.3202.94 Windows 10 UI5 version: 1.50.6 Content-Type:multipart/form-data

I copied the view and controller code from the repo.

I'm not sure how to tell if the Mock Server is being used.

[rosshigh]

I replaced the upload code with the following: ` var oView = this.getView();

        var file = oControlEvent.getParameters().files[0];
        var data = new FormData();

        data.append("files", file, "myimage.png");

        $.ajax({
            type: "POST",
            url: oView.getModel("demo").getProperty("/url"),
            beforeSend: function(request) {
                request.setRequestHeader("APIKey", oView.getModel("demo").getProperty("/APIKey"));
              },
            success: function(data) {
                console.log('it worked');
                // your callback here
            },
            error: function(error) {
                console.log(error);
                // handle error
            },
            async: true,
            data: data,
            cache: false,
            contentType: "multipart/form-data",
            processData: false,
            timeout: 60000
        });`

Unfortunately, I still get a bad request error and the Request Payload is still [object Object]. I would love to show this to the faculty who come to the SAP University Alliance workshops in January. Any ideas on how to proceed?

Thanks, Ross

[rosshigh]

Got it to work on our onprem HANA appliance.

[adadouche]

Hi @rosshigh,

Does that mean that you are using the SAPUI5 library from your HANA appliance?

@bdel

[swapnilspande]

Hi @rosshigh,

In my case, when I would run index.html, the application URL would look like this. https://webidetesting8428063-mytrialtrial.dispatcher.hanatrial.ondemand.com/webapp/extended_runnable_file.html?hc_orionpath=%2Fmytrial%24acc-OrionContent%2Fsapui5ml&origional-url=index.html&sap-ui-appCacheBuster=..%2F&sap-ui-xx-componentPreload=off This is when I get the [Object object] error. I think this means that the Mock Server is being used.

After I manually replace everything starting from "extended_runnable_file.html" by index.html and refresh the application, it works as expected.

Can you check if the app URL is similar for you as well?

[rosshigh]

I'm using the OpenSAP CDN on prem.

I checked the URL used from the web ide and it is using the extended_runnable_file.html, which is odd because I created a run configuration for the index.html and made sure the mock server was turned off. When I replace extended_runnable_file.html with index.html, it worked! Excellent!

Thank you both for your help.

Ross

[rosshigh]

One last, perhaps picky, request. If you load a second (or third, etc.) image, the image control doesn't refresh with the new image. Is there a way to force it to refresh?

Thanks, Ross

[adadouche]

@rosshigh, working on it.

I will push a new version with some additional features in place, along with the "similarity scoring" tutorial.

I'll keep you posted.

[adadouche]

hi @voicedmendor, @GauravSardana, @ericsolberg, @soumyanandi, @swapnilspande, @rosshigh

I have pushed a new version and will close this issue for now.

I hope everyone was able to find a way to leverage the service.

Let me know if you didn't

[srilaxmidonakanti]

Hi adadouche,

i am trying to follow the tutorial : https://www.sap.com/developer/tutorials/ml-fs-sapui5-img-classification.html

I followed all the steps and getting the below error:

Error 400 : { "error": "Invalid request", "error_description": "This service requires at least 1 file. Please put your file(s) into the files field of the POST request", "status": "FAILED", "status_code": 400 }

Please help on this.

Thanks in advance.

[rosshigh]

You have to replace extended_runnable_file with index in the URL.

[srilaxmidonakanti]

Hi rosshigh,

Thanks for the solution .

Given solution (replace extended_runnable_file with index in the URL) is working if i create a SAP UI5 application. But my requirement is to create a SAP Fiori application and used oData service to get & post the data. On top of that applying Machine learning concept also as per following blog.

https://blogs.sap.com/2017/12/27/evaluating-sap-leonardo-machine-learning-image-classification-api-for-a-simple-inventory-management-use-case/

If i replace extended_runnable_file with index in the URL, getting HTTP Status 404 - Not Found.

Given solution will only work for SAP UI5 app? Kindly confirm me. So that i can change the template.

Thanks in advance.

[adadouche]

Hi @srilaxmidonakanti ,

Can I kindly ask you to create a separate issue for your problem as apparently it is not same symptom.

In the meantime here are some hints.

This error is usually triggered for 2 possible reasons:

• you are not using a destination for the API endpoint, which will cause an XSRF problem
• you haven't whitelisted the API Key in the neo-app.json

Can you confirm both are properly configured?

I just answered a really similar question here.

Thanks

[srilaxmidonakanti]

you are not using a destination for the API endpoint, which will cause an XSRF problem:

Maintaining the destination HCP Cockpit destinations as below

you haven't whitelisted the API Key in the neo-app.json:

Written the the code in neo-app.json file as below.

[adadouche]

Hi @srilaxmidonakanti ,

Can you create a new issue with all these details?

Thanks

[Madhusudan0291]

Hi Abdel

I have tried using the Fileuploader control in sapui5 and it works perfectly for me. But when i m trying to do the same thing by taking Picture from camera(i.e Hybrid app) and try to upload it directly i get the error. Can you please provide some example where by using phone camera we can upload the file. Can you please help me here to solve the issue.

Thanks and Regards Madhu

[adadouche]

Hi @Madhusudan0291

This issue is closed as the original problem was fixed. I recommend you to either open a new issue or ask the SAP Community at https://answers.sap.com.

In this issue, the SAPUI5 FileUploader didn't work for certain people in certain region but this is more a SAPUI5 issue than a tutorial issue.

This is why I provided 2 alternate option using direct XHR or AJAX calls.

Thanks

@bdel

[ptharso]

I wish to copy and test your code in my sap web ide....is possible? how to download?

[adozgec]

Hi,

"Step 16: Validation" does not accept my input.

The message that appears after running the application after the image classification completed is Process completed! Target URL: /ml-dest/imageclassification/classification for me but step 16 does not accept that.

Thanks

[aj-aayushijain]

Hi @adadouche,

My Request payload contains - ------WebKitFormBoundaryBDoVmWHrIY0KYEMA Content-Disposition: form-data; name="files"; filename="30873712-c8c22790-a2b3-11e7-8c9e-2bce5d239d92.jpg" Content-Type: image/jpeg

but i am still getting that bad request Error - "error":{"code":"400","message":"Invalid request: This service requires at least 1 file. Please put your file(s) into the files field of the POST request","requestId":"a5d8f8b2-52bc-4d57-5df0-7f705cb4c82f"}"

I tired all the above mentioned things like my url contains index.html and payload is also looking same as mentioned.

can you please help me to solve this issue?

Thanks.

[adadouche]

Hi @aayushi22jain

Can you open a new issue?

Thanks