examples/upload_scan.py fails with 2022.x Black Duck servers

[ghost]

As of 2022.x (?), uploading dryrun bdio files to Black Duck server /api/scans/data using this hub-rest-api-python examples/upload_scan.py fails in various ways, including 400 "The application has encountered an unknown error." errors client-side (visible as 415 UNSUPPORTED_MEDIA_TYPE "Unsupported media type" errors on Black Duck scan container logs--yeah, apparently a bug in there).

curl and postman can be tweaked to work fine relatively easily. But tweaking this script took a little longer.

Our understanding is that the /api/scans/data api endpoint now requires Content-Type multipart/form-data, with the "file" key. See the Black Duck api online doc.

But different clients handle Content-Type differently.

examples/upload_scan.py, via Scans.py, doesn't handle that properly for Black Duck 2022.x.

The python requests library, which is imported into this tool, handles the Content-Type and boundary properly--if you don't confuse it by manually adding your own Content-Type header.

The "headers=headers" in Scans.py, includes a somewhat hidden Content-Type "headers" dictionary key, so that needs to be removed, to avoid confusing the requests library.

To workaround, edit Scans.py. Use the header.pop command to remove that hidden Content-Type value from the "headers" variable, and tweak the requests.post call, like so.

    elif filename.endswith('.bdio'):
        #headers['Content-Type'] = 'application/vnd.blackducksoftware.bdio+zip'
        headers.pop('Content-Type')
        #with open(filename,"rb") as f:
            #response = requests.post(url, headers=headers, data=f, verify=not self.config['insecure'])
        f = {'file': open(filename,'rb')}
        response = requests.post(url, headers=headers, files=f, verify=not self.config['insecure'])

h/t Jeff C.

[gsnyder2007]

@pjsnps thanks for letting me/us know about the issue. The scripts in this folder (.../examples) use the old interface. Examples using the new interface (Client) are in .../examples/client just FYI.

Do you have a fix you want to submit via PR? If not, I'll see how soon I can get around to providing one (using the new Client interface as I'm trying to urge people to move off the old one). Thanks again for the heads up about the issue.

[ghost]

Hi @gsnyder2007, thanks for your reply, and encouragement. I thought about submitting a simple 2-line PR for this "fix" for the old interface, but had no idea how badly changing the handling of *.bdio files for all use-cases back through time would break things.

I'm not a python dev, but I swear I did spend a few minutes trying to tweak the quick_start.py new Client example to POST a .bdio up to the server, but I failed in the time I allotted myself. I'm game to try again if someone could provide some hints. Thanks!

[ghost]

Funny, same customer asked about another "project" api endpoint, I checked examples/client/crud_project.py, and can see how the POST is done. I think I might be able to handle that over the next many days...wish me luck.

(Side note to others: crud_project.py will whack your project (but not your scans), if the project name you provide already exists. Obvious in retrospect...)

[OffBy0x01]

@gsnyder2007 perhaps we should split useable scripts from examples?