Unified Agent does not take -projectName into account

jomora commented 2 years ago

We're trying to scan a docker image and upload the results to Whitesource.

The docker image is present as a TAR file and has a generic name, i.e. image.tar.

When running the unified agent we use the parameter -projectName to target a specific project.

However, the project name is set to the name of the file without the extension, i.e. image.

So, it results in a project named image being created in the Whitesource user interface.

The same happens if we manually create the project via the UI and then pass the projectToken via the -projectToken parameter.

In effect, we cannot scan our docker images.

Example: Scan ingress-nginx-controller

We run the following command:

java -jar /wss-unified-agent.jar -d . -c wss_configs/wss_config_defaultbackend.cfg -project ingress-nginx-controller -product 'OUR PRODUCT NAME' -productVersion 1 -wss.url 
https://saas.whitesourcesoftware.com/agent -apiKey ((redacted)) -userKey ((redacted))

We would expect a project of name "ingress-nginx-controller" to be created, but this does not happen. Instead the project "image" is created.

Below you will find our config file with for the first case (use projectName instead of projectToken):

[INFO] [2021-07-14 12:15:10,647 +0000] - UnifiedAgent version (pluginVersion) : 21.6.2.2
[INFO] [2021-07-14 12:15:10,674 +0000] - 
apiKey=******
archiveExtractionDepth=10
case.sensitive.glob=false
checkPolicies=true
configFilePath=wss_configs/wss_config_defaultbackend.cfg
docker.includes=.*.tar
docker.scanImages=true
docker.scanTarFiles=true
failErrorLevel=ALL
fileSystemScan=true
followSymbolicLinks=true
forceCheckAllDependencies=true
forceUpdate.failBuildOnPolicyViolation=false
forceUpdate=true
generateProjectDetailsJson=true
generateScanReport=true
includes=**
offline=false
productName=OUR PRODUCT NAME
productVersion=1
projectName=ingress-nginx-controller
projectTag=
python.resolveGlobalPackages=true
resolveAllDependencies=false
updateType=OVERRIDE
userKey=******
wss.url=https://saas.whitesourcesoftware.com/agent

[INFO] [2021-07-14 12:15:50,845 +0000] - Newly created projects:
[INFO] [2021-07-14 12:15:50,845 +0000] - # image - 1
[INFO] [2021-07-14 12:15:50,845 +0000] - No projects were updated.
[INFO] [2021-07-14 12:15:50,845 +0000] - Project name: image - 1, URL: https://saas.whitesourcesoftware.com/Wss/WSS.html#!project;id=...
[INFO] [2021-07-14 12:15:50,845 +0000] - Support Token: ...
[INFO] [2021-07-14 12:15:50,847 +0000] - generating scan report
[INFO] [2021-07-14 12:16:08,539 +0000] - scan report created successfully at /tmp/build/e55deab7/repository/./whitesource/image - 1-2021-07-14T121608+0000-scan_report.json
[INFO] [2021-07-14 12:16:08,542 +0000] - Log files are found at: ./whitesource/Wed-Jul-14-2021-12.15.10
[INFO] [2021-07-14 12:16:08,548 +0000] - 
------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------- WhiteSource Scan Summary: --------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------------------------------------
======================================================================================================================================================
Scan Origin: Local Docker Images
======================================================================================================================================================
Step                                              Completion Status               Elapsed                  Comments
======================================================================================================================================================
Fetch Configuration                                  COMPLETED                  00:00:00.238               --------
Docker Resolver Scan                                 COMPLETED                  00:00:18.484               --------
   tar-name                                          COMPLETED                  00:00:18.484               image.tar
      Scan Files Matching Includes Pattern           COMPLETED                  00:00:00.896               3753 source/binary files
         post-upgrade                                COMPLETED                    --------                 11 source/binary files
         11                                          COMPLETED                    --------                 1 source/binary files
                                                     COMPLETED                    --------                 1703 source/binary files
         tar                                         COMPLETED                    --------                 4 source/binary files
         crt                                         COMPLETED                    --------                 140 source/binary files
         dist                                        COMPLETED                    --------                 4 source/binary files
         conf                                        COMPLETED                    --------                 50 source/binary files
         trigger                                     COMPLETED                    --------                 7 source/binary files
         post-install                                COMPLETED                    --------                 11 source/binary files
         script                                      COMPLETED                    --------                 2 source/binary files
         0                                           COMPLETED                    --------                 151 source/binary files
         1                                           COMPLETED                    --------                 25 source/binary files
         pre-install                                 COMPLETED                    --------                 4 source/binary files
         vfat                                        COMPLETED                    --------                 1 source/binary files
         sh                                          COMPLETED                    --------                 4 source/binary files
         pem                                         COMPLETED                    --------                 140 source/binary files
         cnf                                         COMPLETED                    --------                 4 source/binary files
         so                                          COMPLETED                    --------                 25 source/binary files
         pl                                          COMPLETED                    --------                 9 source/binary files
         pub                                         COMPLETED                    --------                 22 source/binary files
         pre-upgrade                                 COMPLETED                    --------                 4 source/binary files
      Alpine                                         COMPLETED                    --------                 40 packages
Check Policies                                       COMPLETED                  00:00:10.548               --------
Update Inventory                                     COMPLETED                  00:00:11.030               0 updated projects

======================================================================================================================================================
Elapsed running time:                                                           00:00:40.300

LenaKleyner commented 2 years ago

Hi @jomora,

This is actually by design. Specifically for the Docker scanning mode of the Unified Agent, the project name is not being taken into consideration. The reasoning is that in this mode, several images can be scanned in parallel, resulting in several projects being created/updated in the WhiteSource application. Thus, the project name, created per each image, is derived from the name of the scanned image.

When scanning images, the docker.projectNameFormat can be used to adjust the name of the project. We currently support 3 options:

default for comprising the project name from the image name, the image tag and the image id
repositoryNameAndTag for comprising the project name from the image name and the image tag
repositoryName for comprising the project name from the image name only

However, in your case, in which the image tar file is scanned, the project name will be derived from the tar name.

I hope it clarifies the observed behavior, please let me know if you have additional questions.

Thanks,

Lena

jomora commented 2 years ago

@LenaKleyner, thanks a lot for the detailed response. This clarified the situation.

We think it would be valuable if the documentation would state how tar files are handled.

https://whitesource.atlassian.net/wiki/spaces/WD/pages/710673221/Docker+Image+Integration#Example

Thanks again!

@bgoerzig and @jomora

LenaKleyner commented 2 years ago

Thanks @jomora , we've added some clarifications to the docs.

whitesource / unified-agent-distribution

Unified Agent does not take -projectName into account #15

Example: Scan ingress-nginx-controller