search

whitesource-ps / ws-sbom-generator

WS SBOM Report Generator in SPDX or CycloneDX format
Apache License 2.0
32 stars 7 forks source link

[Question] [ws-sbom-generator] While executing I get 'Invalid organization token' #68

Open adubey8 opened 2 years ago

adubey8 commented 2 years ago

Question

My command execution:

export USER_KEY=VALUE
export ORG_WS_TOKEN=VALUE
export WS_SCOPE_TOKEN=VALUE
export WS_URL=https://app-eu.whitesourcesoftware.com
export REPORT_OUTPUT_DIR=VALUE

docker run --name ws-sbom-generator -v /working-folder-02/digital-signatures/sbom/sbom_extra.json:/opt/ws-sbom-generator/sbom-generator/resources -v $REPORT_OUTPUT_DIR:/opt/ws-sbom-generator/sbom-generator/output -e WS_USER_KEY=$USER_KEY -e WS_TOKEN=$ORG_WS_TOKEN -e WS_URL=$WS_URL -y STIC-cloud whitesourcetools/ws-sbom-generator

I get

WARNING 2022-02-10 00:30:39,148 140201480283968 sbom_generator: /opt/ws-sbom-generator/sbom_generator/resources/sbom_extra.json configuration file was not found. Be sure to create a file in the following structure (-e/--extra):
            {
                "namespace": "http://CreatorWebsite/pathToSpdx/DocumentName-UUID",
                "org_email": "org@email.address",
                "person": "person name",
                "person_email": "person@email.address"
            }

INFO 2022-02-10 00:30:39,148 140201480283968 sbom_generator: Creating SBOM reports on all Organization Projects
Traceback (most recent call last):
  File "/usr/local/bin/ws_sbom_generator", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/ws_sbom_generator/sbom_generator.py", line 397, in main
    scopes = args.ws_conn.get_projects()
  File "/usr/local/lib/python3.9/site-packages/ws_sdk/web.py", line 751, in get_projects
    products = self._generic_get(get_type="ProductVitals")['productVitals']
  File "/usr/local/lib/python3.9/site-packages/ws_sdk/web.py", line 259, in _generic_get
    return self.call_ws_api(request_type=f"get{token_type.capitalize()}{get_type}", kv_dict=kv_dict)
  File "/usr/local/lib/python3.9/site-packages/ws_sdk/web.py", line 227, in call_ws_api
    __handle_ws_server_errors(resp.text)
  File "/usr/local/lib/python3.9/site-packages/ws_sdk/web.py", line 201, in __handle_ws_server_errors
    raise WsSdkServerGenericError(body[token], error)
ws_sdk.ws_errors.WsSdkServerGenericError: Generic error on token: VALUE. Error: {"errorCode":1002,"errorMessage":"Invalid organization token"}

I get my orgToken from the url: https://app-eu.whitesourcesoftware.com/Wss/WSS.html#!product;id=VALUE;orgToken=VALUE And I update my SCOPE_TOKEN based on a project as I am a project Admin. Why I am still getting mapped to org scope?

Environment Details

NatalyaDalid commented 2 years ago

Hi @adubey8, Unfortunately, the URL you have provided is not clickable to me. The WS orgToken (or API Key) can be found under the 'Integrate' tab in your WhiteSource organization.

In addition, we have fixed several issues and released a new version. Could you please pull the latest version and try running again using the following cmd lines: if you'd like to run with the Org Admin Token: docker run --name ws-sbom-generator -v /working-folder-02/digital-signatures/sbom/sbom_extra.json:/opt/ws-sbom-generator/sbom-generator/resources -v $REPORT_OUTPUT_DIR:/opt/ws-sbom-generator/sbom-generator/output -e WS_URL=https://app-eu.whitesourcesoftware.com -e WS_USER_KEY=<USER_KEY> -e WS_TOKEN=<WS_ORG_TOKEN> -e WS_SCOPE_TOKEN=<WS_PROJECT_TOKEN> whitesourcetools/ws-sbom-generator if you'd like to run with the Product Admin Token: docker run --name ws-sbom-generator -v /working-folder-02/digital-signatures/sbom/sbom_extra.json:/opt/ws-sbom-generator/sbom-generator/resources -v $REPORT_OUTPUT_DIR:/opt/ws-sbom-generator/sbom-generator/output -e WS_URL=https://app-eu.whitesourcesoftware.com -e WS_USER_KEY=<USER_KEY> -e WS_TOKEN=<WS_PRODUCT_TOKEN> -e WS_SCOPE_TOKEN=<WS_PROJECT_TOKEN> -e WS_TOKEN_TYPE=product whitesourcetools/ws-sbom-generator

Thanks, WS PS Team

adubey8 commented 2 years ago

hey @NatalyaDalid thanks for the reply. The steps mentioned worked to execute the sbom generator. But I do not see the output file generated?

White Source Unified Agent does not exist in path: '/tmp/sdk/wss-unified-agent.jar'
WARNING 2022-02-14 22:01:05,285 140002195912512 sbom_generator: /opt/ws-sbom-generator/sbom_generator/resources/sbom_extra.json configuration file was not found. Be sure to create a file in the following structure (-e/--extra):
            {
                "namespace": "http://CreatorWebsite/pathToSpdx/DocumentName-UUID",
                "org_name": "Organization Name",
                "org_email": "org@email.address",
                "person": "person name",
                "person_email": "person@email.address"
            }

INFO 2022-02-14 22:01:09,995 140002195912512 sbom_generator: Creating SBOM Document from WhiteSource project: 'gcr.io_IMAGE'
ERROR 2022-02-14 22:01:12,392 140002195912512 sbom_generator: Unable to get Organization name. Using: 'ORG_NAME'
Unable to fix spdxName of GPL
License with identifier: GPL was not found
.
.
.
WARNING 2022-02-14 22:01:22,858 140002195912512 sbom_generator: Found 3 licenses on library: libgssrpc4. Using the first one
WARNING 2022-02-14 22:01:22,858 140002195912512 sbom_generator: Found 3 licenses on library: krb5-multidev. Using the first one
WARNING 2022-02-14 22:01:22,858 140002195912512 sbom_generator: Found 2 licenses on library: libperl5.32. Using the first one
WARNING 2022-02-14 22:01:22,859 140002195912512 sbom_generator: No copyright info found for library: websocket-client
WARNING 2022-02-14 22:01:22,859 140002195912512 sbom_generator: No copyright info found for library: google-auth
INFO 2022-02-14 22:01:22,861 140002195912512 sbom_generator: Writing file: /opt/ws-sbom-generator/sbom_generator/output/WhiteSource gcr.io_IMAGE SBOM report-SPDX-2.2.tv in format: tv
INFO 2022-02-14 22:01:22,871 140002195912512 sbom_generator: Report saved at ['/**opt/ws-sbom-generator/sbom_generator/output**/WhiteSource gcr.io_IMAGE SBOM report-SPDX-2.2.tv']

The output folder in the container is empty:

root@f960eecf69d3:/opt/ws-sbom-generator/sbom_generator/output# ls -lrt 
total 0
root@f960eecf69d3:/opt/ws-sbom-generator/sbom_generator/output#
NatalyaDalid commented 2 years ago

Hi @adubey8 ,

Can you please double-check the $REPORT_OUTPUT_DIR environment variable is being set? if not you can also set your local report output path in the following way: -v <your_local_path>:/opt/ws-sbom-generator/sbom-generator/output

The full cmd line will be like this: docker run --name ws-sbom-generator -v /working-folder-02/digital-signatures/sbom/sbom_extra.json:/opt/ws-sbom-generator/sbom-generator/resources -v <your_local_path>:/opt/ws-sbom-generator/sbom-generator/output -e WS_URL=https://app-eu.whitesourcesoftware.com -e WS_USER_KEY=<USER_KEY> -e WS_TOKEN=<WS_ORG_TOKEN> -e WS_SCOPE_TOKEN=<WS_PROJECT_TOKEN> whitesourcetools/ws-sbom-generator

You will be able to find the report in your local output path.

Thanks, Natalya

adubey8 commented 2 years ago

Hey Natalya, Thanks for the reply.

I repeated what you described above.

docker run --name ws-sbom-generator -v /working-folder-02/:/opt/ws-sbom-generator/sbom-generator/output -e WS_URL=$WS_URL -e WS_USER_KEY=$USER_KEY -e WS_TOKEN=$WS_ORG_TOKEN -e WS_SCOPE_TOKEN=$WS_PROJECT_TOKEN whitesourcetools/ws-sbom-generator

and I get a stacktrace just like mentioned before, with last two lines as if the output is written:

WARNING 2022-02-16 15:04:03,286 140655840515904 sbom_generator: No copyright info found for library: google-auth
INFO 2022-02-16 15:04:03,288 140655840515904 sbom_generator: Writing file: /opt/ws-sbom-generator/sbom_generator/output/WhiteSource gcr.io_slb-datalake-lakefront_pod-scheduler 1.1.0.202108263 (56c1806e977f) SBOM report-SPDX-2.2.tv in format: tv
INFO 2022-02-16 15:04:03,298 140655840515904 sbom_generator: Report saved at ['/opt/ws-sbom-generator/sbom_generator/output/WhiteSource gcr.io_slb-datalake-lakefront_pod-scheduler 1.1.0.202108263 (56c1806e977f) SBOM report-SPDX-2.2.tv']

But my output folder remains empty. It doesnt complain about permissions but says report saved. So not sure what must be causing not to write the final file. Any help is appreciated.

Thanks, Aady

UPDATE: I tried the same example with local python script:

python3 -m ws_sbom_generator.sbom_generator -u $USER_KEY -k $WS_ORG_TOKEN -s $WS_PROJECT_TOKEN -a $WS_URL -t json -o $REPORT_OUTPUT_DIR

and it generated the file: cat 'WhiteSource XXX 1.1.0.XXX (56c1806e977f) SBOM report-SPDX-2.2.json' Not sure but for some reason docker isn't able to save the .tv file. I shall try changing the format of file next. I will keep posted.

NatalyaDalid commented 2 years ago

Hi @adubey8,

Thank you for the update

Could you please also try running with the full local path, like C:/Users.... instead of /working-folder-02/

Thanks, Natalya