Unable to find repo details from the local repository

[jacace]

Hello,

I am getting the error "Unable to find repo details from the local repository" but there is in fact a file .sastscanrc in /app.

I can see that a file findsecbugs-report.xml was generated but there is just one finding.

**1. Why there is hust one finding? it seems the scanners stopped prematurely.

2. Also, how can I get findsecbugs-report.xml exported to a SARIF?**

docker run --rm -e "WORKSPACE=%cd%" -v "C:\maven\apache-maven-3.6.3\bin" -v "C:\source_code\app":/app appthreat/sast-scan scan --src /app --type java

---

/ _ \					/ /\ \ _ __				_ _ _		_	_	'_ \	'_ \			'_ \	'_/ \/ _`	__						_)		_)								_/ (			\		_/ .__/	._/\/	_		_	_	___	_,	__
_		_

INFO [2020-04-01 13:26:44,486] Scanning /app using scan plugins ['credscan', 'java'] INFO [2020-04-01 13:26:44,494] ================================================================================ INFO [2020-04-01 13:26:44,496] ⚡︎ Executing "gitleaks --depth=2 --repo-path=/app --redact --timeout=2m --report=/app/reports/credscan-report.json --report-format=json" INFO[2020-04-01T13:26:44Z] no leaks found, skipping writing report INFO[2020-04-01T13:26:44Z] No leaks detected. 0 commits audited in 353 milliseconds 67 microseconds INFO [2020-04-01 13:26:44,896] ================================================================================ INFO [2020-04-01 13:26:44,897] ⚡︎ Executing "java -jar /opt/spotbugs/lib/spotbugs.jar -textui -include /usr/local/src/spotbugs/include.xml -exclude /usr/local/src/spotbugs/exclude.xml -noClassOk -auxclasspathFromFile /tmp/tmpl891l6xd -sourcepath /app -quiet -medium -xml:withMessages -effort:max -nested:false -output /app/reports/findsecbugs-report.xml /app"WARNING [2020-04-01 13:27:02,662] Unable to find repo details from the local repository. Consider adding a local .sastscanrc file with the url details. WARNING [2020-04-01 13:27:02,680] Project type is not supported: java

Thanks,

[jacace]

Hello @prabhu

I figured it out myself how to continue on error using the flag: --no-error, so I now have a SARIF file. Still not sure why there is only one finding from find-sec-bugs and nothing else. So I still believe the scan process is prematurely.

On a side note, I noticed 3 files are created, incl a JSON, a XML, a HTML and a SARIF file. But I did not see anything with regards to Grafeas. I know sasts-scan uses internally scan-reports which generates data in Grafeas format. I can see in requirements.txt that scan-reports gets installed but I can't figure it where (dir) is located in the container filesystem to use it. I inspected the container file system and inside lib there are only python files, while one level up there are system files (incl. cpp debug, eclipse, games, gcc gems, golang, java, jvm, jvm-commmon, jvm-private, locale, modprobe.d, modules node_modules, python3.6, rpm, sse2, swidtag, sysimage, system, tmpfiles.d and udev) but I could not see scan-reports.

Let me re-cap the questions as per update: **1. Why there is just one finding? it seems the scanners stopped prematurely.

2. How to get data in Grafeas using scan-reports?**

Thanks,

Javier Caceres

[prabhu]

Hi @jacace,

For java projects, we must compile the project first before invoking the scan. The warning message you are seeing is to indicate the fact that the tool could not find the repository related information possibly because the folder is not a git repository? You can ignore the warning.

Grafeas is part of dependency scan. To enable it simply pass depscan along with type

Eg: --type credscan,java,depscan . You can also ignore the type parameter to enable automatic detection. With depscan the vulnerability information will appear in grafeas format. To improve vulnerability scanning you can pass GITHUB_TOKEN as an environment variable.

You can refer to some examples such as: https://github.com/AppThreat/WebGoat/blob/develop/azure-pipelines.yml#L23

https://github.com/AppThreat/sast-scan/blob/master/.github/workflows/pythonapp.yml#L34

Hope this helps.

[prabhu]

To answer your other question regarding scan-reports it is installed via pip install to /usr/local/lib/python3.6/site-packages/

docker run --rm -e "WORKSPACE=${PWD}" -v $PWD:/app appthreat/sast-scan python3 -c "import sys; print(sys.path)"
['', '/usr/lib64/python36.zip', '/usr/lib64/python3.6', '/usr/lib64/python3.6/lib-dynload', '/usr/local/lib64/python3.6/site-packages', '/usr/local/lib/python3.6/site-packages', '/usr/lib64/python3.6/site-packages', '/usr/lib/python3.6/site-packages']

docker run --rm -e "WORKSPACE=${PWD}" -v $PWD:/app appthreat/sast-scan python3 -c "import reporter; print(reporter.__file__)"
/usr/local/lib/python3.6/site-packages/reporter/__init__.py

[prabhu]

Also, you might be having quite an old build for sast-scan. The new build no longer has that particular message - https://github.com/AppThreat/sast-scan/blob/master/lib/context.py#L85

A simple docker pull should resolve the issue.

[jacace]

thanks @prabhu !

I just pulled latest version but still seeing same error and unable to get a grafeas file as per detailed error log below:

Please note:

1. This is a indeed a (private) GitHub repo
2. The compiled source code is in: app/target/webapi-1.3.jar (i.e.: C:\app_source_code\target\webapi-1.3.jar).
3. The latest docker image version is reading correctly the file .sastscanrc

This is my comman dto run the image: docker run --rm -e "WORKSPACE=%cd%" -v "C:\apache-maven-3.6.3\bin" -v "C:\app_source_code":/app appthreat/sast-scan scan --src /app --type credscan,java,depscan

Am I doing anything wrong? Thanks.

Scan log Below.

---

---

/ _ \					/ /\ \ _ __				_ _ _		_	_	'_ \	'_ \			'_ \	'_/ \/ _`	__						_)		_)								_/ (			\		_/ .__/	._/\/	_		_	_	___	_,	__
_		_

INFO [2020-04-01 17:50:46,735] Scanning /app using scan plugins ['credscan', 'java', 'depscan']

INFO [2020-04-01 17:50:46,738] ================================================================================ INFO [2020-04-01 17:50:46,739] ⚡︎ Executing "gitleaks --depth=2 --repo-path=/app --redact --timeout=2m --report=/app/reports/credscan-report.json --report-format=json" INFO[2020-04-01T17:50:47Z] no leaks found, skipping writing report INFO[2020-04-01T17:50:47Z] No leaks detected. 0 commits audited in 265 milliseconds 279 microseconds INFO [2020-04-01 17:50:47,045] ================================================================================ INFO [2020-04-01 17:50:47,045] ⚡︎ Executing "java -jar /opt/spotbugs/lib/spotbugs.jar -textui -include /usr/local/src/spotbugs/include.xml -exclude /usr/local/src/spotbugs/exclude.xml -noClassOk -auxclasspathFromFile /tmp/tmpjdfyjkyw -sourcepath /app -quiet -medium -xml:withMessages -effort:max -nested:false -output /app/reports/findsecbugs-report.xml /app" WARNING [2020-04-01 17:51:01,967] Unable to find repo details from the local repository. Consider adding a local .sastscanrc file with the url details. WARNING [2020-04-01 17:51:01,974] Project type is not supported: java INFO [2020-04-01 17:51:01,975] ================================================================================ INFO [2020-04-01 17:51:01,975] ⚡︎ Executing "/usr/local/bin/depscan --no-banner --src /app --report_file /app/reports/depscan-report.json" INFO [2020-04-01 17:51:02,880] ================================================================================ INFO [2020-04-01 17:51:02,893] ⚡︎ Executing "cdxgen -r -t java -o /app/reports/bom-java.xml /app" WARNING [2020-04-01 17:51:04,506] Unable to parse /app/reports/bom-java.xml no element found: line 1, column 0 WARNING [2020-04-01 17:51:04,507] No packages found in the project! INFO [2020-04-01 17:51:04,845] To use GitHub advisory source please set the environment variable GITHUB_TOKEN!

Thanks,

[prabhu]

Can we do a zoom meeting since something is not looking right on windows? Please email me: prabhu @ appthreat.com

[jacace]

thanks @prabhu

Just FYI: I tried using the repository URL and github token too but got same result.

Assuming no git clone is required (since the code is already in C:\app_source_code) then the question is: What's the structure expected in the source code local folder? In my case I have pretty standard folders e.g.: C:\app_source_code\ .github\workflows\main.yml C:\app_source_code\ .mvn\wrapper C:\app_source_code\ .settings C:\app_source_code\target -> here the jar C:\app_source_code\src\main\java\com -> here folders mapping the package structure C:\app_source_code\ .sastscanrc C:\app_source_code\pom.xml etc...

docker run -v "C:\apache-maven-3.6.3\bin" -v "C:\app_source_code":/app -e "REPOSITORY_URL=https://github.com/myuser/myreoi.git" -e "GITHUB_TOKEN=" appthreat/sast-scan:latest scan --src /app

[prabhu]

I have so far tested with .class files so invoking mvn compile before running the tool definitely works. The command you are using is fine (although mapping apache maven is not required).

Can you try with this image quay.io/appthreat/sast-scan:feature_mvn instead of appthreat/sast-scan:latest. I have added some debugs now.

docker run -v "C:\app_source_code":/app -e "REPOSITORY_URL=https://github.com/myuser/myrepo" -e "GITHUB_TOKEN=" quay.io/appthreat/sast-scan:feature_mvn scan --src /app

[jacace]

Hello @prabhu

I just tried the with the image suggested but deepscan neither BOM worked - In can see it in the log. fyi: I am on a Windows 10 laptop running Linux containers, not sue if this makes a difference. IMHO, seems like the compiled .jar/classes can be found dynamically.

docker run -v "C:\app_source_code_root":/app -e "REPOSITORY_URL=https://github.com/myuser/myrepo" -e "GITHUB_TOKEN=" quay.io/appthreat/sast-scan:feature_mvn scan --src /app --no-error

---

/ _ \					/ /\ \ _ __				_ _ _		_	_	'_ \	'_ \			'_ \	'_/ \/ _`	__						_)		_)								_/ (			\		_/ .__/	._/\/	_		_	_	___	_,	__
_		_

INFO [2020-04-01 19:07:54,690] Scanning /app using scan plugins ['credscan', 'java', 'depscan'] INFO [2020-04-01 19:07:54,702] ================================================================================ INFO [2020-04-01 19:07:54,702] ⚡︎ Executing "gitleaks --depth=2 --repo-path=/app --redact --timeout=2m --report=/app/reports/credscan-report.json --report-format=json" INFO[2020-04-01T19:07:54Z] no leaks found, skipping writing report INFO[2020-04-01T19:07:54Z] No leaks detected. 0 commits audited in 81 milliseconds 733 microseconds INFO [2020-04-01 19:07:54,999] ================================================================================ INFO [2020-04-01 19:07:55,000] ⚡︎ Executing "java -jar /opt/spotbugs/lib/spotbugs.jar -textui -include /usr/local/src/spotbugs/include.xml -exclude /usr/local/src/spotbugs/exclude.xml -noClassOk -auxclasspathFromFile /tmp/tmpj6kfxca2 -sourcepath /app -quiet -medium -xml:withMessages -effort:max -nested:false -output /app/reports/findsecbugs-report.xml /app" INFO [2020-04-01 19:08:16,775] ================================================================================ INFO [2020-04-01 19:08:16,776] ⚡︎ Executing "/usr/local/bin/depscan --no-banner --src /app --report_file /app/reports/depscan-report.json" INFO [2020-04-01 19:08:18,394] ================================================================================ Traceback (most recent call last): File "/usr/local/bin/depscan", line 11, in sys.exit(main()) File "/usr/local/lib/python3.6/site-packages/depscan/cli.py", line 134, in main create_bom(project_type, bom_file, args.src_dir) File "/usr/local/lib/python3.6/site-packages/depscan/lib/bom.py", line 132, in create_bom with open(bom_file, mode="w") as fp: FileNotFoundError: [Errno 2] No such file or directory: '/app/reports/bom-java.xml'

tool description critical high medium low status

---

findsecbugs Security audit by Find Security Bugs 1 0 0 0 ❌

[jacace]

Hello @prabhu

I just tested in an Ubuntu VM (previously I was running Linux containers in Windows 10) and got a bit further. deepscan still failing. Now BOM scan partially working. Do you see anything missing? Also, still surprised to get just one finding.

$ docker run -v "/home/ubuntu/app_source_code":/app -e "REPOSITORY_URL=https://github.com/myuser/myrepo.git" -e "GITHUB_TOKEN=" quay.io/appthreat/sast-scan:feature_mvn scan --src /app

---

/ _ \					/ /\ \ _ __				_ _ _		_	_	'_ \	'_ \			'_ \	'_/ \/ _`	__						_)		_)								_/ (			\		_/ .__/	._/\/	_		_	_	___	_,	__
_		_

INFO [2020-04-02 08:29:06,464] Scanning /app using scan plugins ['credscan', 'java', 'depscan'] INFO [2020-04-02 08:29:06,465] ================================================================================ INFO [2020-04-02 08:29:06,465] ⚡︎ Executing "gitleaks --depth=2 --repo-path=/app --redact --timeout=2m --report=/app/reports/credscan-report.json --report-format=json"

WARN[2020-04-02T08:29:06Z] Exceeded depth limit (2) INFO[2020-04-02T08:29:06Z] no leaks found, skipping writing report INFO[2020-04-02T08:29:06Z] No leaks detected. 2 commits audited in 24 milliseconds 510 microseconds INFO [2020-04-02 08:29:06,549] ================================================================================ INFO [2020-04-02 08:29:06,549] ⚡︎ Executing "java -jar /opt/spotbugs/lib/spotbugs.jar -textui -include /usr/local/src/spotbugs/include.xml -exclude /usr/local/src/spotbugs/exclude.xml -noClassOk -auxclasspathFromFile /tmp/tmp33oqp9w6 -sourcepath /app -quiet -medium -xml:withMessages -effort:max -nested:false -output /app/reports/findsecbugs-report.xml /app" INFO [2020-04-02 08:29:17,595] ================================================================================ INFO [2020-04-02 08:29:17,595] ⚡︎ Executing "/usr/local/bin/depscan --no-banner --src /app --report_file /app/reports/depscan-report.json" INFO [2020-04-02 08:29:17,815] ================================================================================ INFO [2020-04-02 08:29:17,816] ⚡︎ Executing "cdxgen -r -t java -o /app/reports/bom-java.xml /app"

===License scan findings===

+-------------------------------------------+-----------+--------------+--------------------------------------------------+ | Package | Version | License Id | License conditions | +===========================================+===========+==============+==================================================+ | ch.qos.logback:logback-classic | 1.2.3 | EPL-1.0 | disclose-source, include-copyright, same-license | +-------------------------------------------+-----------+--------------+--------------------------------------------------+ | ch.qos.logback:logback-core | 1.2.3 | EPL-1.0 | disclose-source, include-copyright, same-license | +-------------------------------------------+-----------+--------------+--------------------------------------------------+ | jakarta.annotation:jakarta.annotation-api | 1.3.5 | EPL-2.0 | disclose-source, include-copyright, same-license | +-------------------------------------------+-----------+--------------+--------------------------------------------------+ Traceback (most recent call last): File "/usr/local/bin/depscan", line 11, in sys.exit(main()) File "/usr/local/lib/python3.6/site-packages/depscan/cli.py", line 171, in main s.refresh() File "/usr/local/lib/python3.6/site-packages/vdb/lib/nvd.py", line 93, in refresh return self.download_all() File "/usr/local/lib/python3.6/site-packages/vdb/lib/nvd.py", line 50, in download_all self.store(data) File "/usr/local/lib/python3.6/site-packages/vdb/lib/nvd.py", line 98, in store docs = dbLib.store(db, data) File "/usr/local/lib/python3.6/site-packages/vdb/lib/db.py", line 51, in store docs = storage.store(datas, db_file=db["db_file"], index_file=db["index_file"]) File "/usr/local/lib/python3.6/site-packages/vdb/lib/storage.py", line 29, in store packed_obj = msgpack.packb(data_list, use_bin_type=True) File "/usr/local/lib64/python3.6/site-packages/msgpack/init.py", line 35, in packb return Packer(kwargs).pack(o) File "msgpack/_packer.pyx", line 286, in msgpack._cmsgpack.Packer.pack File "msgpack/_packer.pyx", line 296, in msgpack._cmsgpack.Packer.pack MemoryError**

tool description critical high medium low status

---

findsecbugs Security audit by Find Security Bugs 1 0 0 0 ❌

$ cd reports $ ls -l -rw-r--r-- 1 root root 2823 Apr 2 08:31 all-b3c5c8f8-417f-4c55-a349-b27036bae31f-report.json -rw-r--r-- 1 root root 3051167 Apr 2 08:30 bom-java.xml -rw-r--r-- 1 root root 9393 Apr 2 08:29 findsecbugs-report.html -rw-r--r-- 1 root root 4486 Apr 2 08:29 findsecbugs-report.sarif -rw-r--r-- 1 root root 28046 Apr 2 08:29 findsecbugs-report.xml -rw-r--r-- 1 root root 503 Apr 2 08:30 license-java.json

Thanks,

Javier caceres

[prabhu]

On Linux, the depscan is failing because it is running out of memory. Running the docker container with over 2GB of RAM (4GB is recommended) should help. Windows seems to be some kind of file write issue with WSL-Windows cross-over somewhere. Let me investigate.

What java version are you using to compile your project and can you do a find of all class files?

find . -name *.class

java -version

[jacace]

thanks @prabhu just switched from a t2.micro to a t2.medium instance and the BOM and depscan scans worked fine. One last question: does the grafeas json file contain vulnerability occurrences? If so, what would be the anchor note?

[prabhu]

Glad to hear this. For the note there are two models:

Model 1:

• You can periodically create notes using vulnerability-db and the grafeas createNote api
• While storing the data from depscan json you can refer to these notes via the name

In model 2:

• Just create occurrence without an associated note.

This approach would require the VulnerabilityDetails to be present in each occurrence. This is currently not possible with depscan since it only passes the search result in occurrence format and keeps the details only in the database.

Hope this helps.

[jacace]

thanks for your help @prabhu ! I will mark this question as closed now. I will review the bundled tools and reports and will come back if any question. regards,