search

oss-review-toolkit / ort

A suite of tools to automate software compliance checks.
https://oss-review-toolkit.org
Apache License 2.0
1.57k stars 308 forks source link

ORT not able to analyze any Python based project (setuptool breaking) #5159

Closed meeraa247 closed 2 years ago

meeraa247 commented 2 years ago

Hi @ all,

first of all: We appreciate this community a all and would like to thank you for all the support. Our team heavily relies on scanning & analyzing projects using ORT.

Recently we came across a small project based on Python (v.3.8.x). During the ANALYZE Step, we identified the following ERRORS (from the logs):

[DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Resolving PIP dependencies for path 'Python_v3.8.8/Lib/site-packages/numpy/ma/setup.py' failed with: IOException: Running '/tmp/ort-Pip-ma-virtualenv14989541890154123245/bin/pip --trusted-host pypi.org --trusted-host pypi.python.org install --no-warn-conflicts --prefer-binary .' in '/project/Python_v3.8.8/Lib/site-packages/numpy/ma' failed with exit code 1:
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-t9xwYP/
You are using pip version 18.1, however version 20.3.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

At first, it seemed to be an issue with the so called "virtualenv". @heliocastro also checked and confirmed this behaviour from side.

We also tried to specify a dedicated python version for ORT (using the docker build process), but that did not help.

Any help would be highly appreciated as this blocks our processes for this project at the moment.

Thank you.

heliocastro commented 2 years ago

I can confirm the bug. There's an open source package to easy reproduce it and can be used as reference: https://github.com/fastavro/fastavro

Look like some discrepancy on the manipulating the setuptools new versions, as our dependency analyzer tool is pydep is really old. To be investigated.

Here's the output of analyzer error

09:23:47.125 [DefaultDispatcher-worker-3] DEBUG org.eclipse.jgit.util.SystemReader - loading config FileBasedConfig[/workspace/.gitconfig]
09:23:47.181 [DefaultDispatcher-worker-3] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Resolving PIP dependencies for path 'developer_requirements.txt' failed with: IOException: Running '/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py info .' in '/worspace/code' failed with exit code 1:
/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py:17: UserWarning: Distutils was imported before Setuptools, but importing Setuptoolsalso replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
warnings.warn(
/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
Traceback (most recent call last):
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 161, in <module>
main()
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 43, in main
args.func(args)
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 68, in info
setup_dict, err = pydep.setup_py.setup_info_dir(args.dir)
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/pydep/setup_py.py", line 41, in setup_info_dir
return setup_info(setupfile), None
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/pydep/setup_py.py", line 53, in setup_info
setuptools_mod = __import__('setuptools')
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/setuptools/__init__.py", line 8, in <module>
import _distutils_hack.override  # noqa: F401
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/override.py", line 1, in <module>
__import__('_distutils_hack').do_override()
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 72, in do_override
ensure_local_distutils()
File "/tmp/ort-Pip-code-virtualenv2432521012921555598/lib/python3.8/site-packages/_distutils_hack/__init__.py", line 59, in ensure_local_distutils
assert '_distutils' in core.__file__, core.__file__
AssertionError: /opt/python/versions/3.8.13/lib/python3.8/distutils/core.py
sschuberth commented 2 years ago

To be investigated.

@heliocastro are you planning to work on this?

sschuberth commented 2 years ago

@meeraa247, while your original post does not detail the actual error message, I assume it's really the same as in @heliocastro's post. So, can you guys give https://github.com/oss-review-toolkit/ort/pull/5247 a try to confirm it fixes this specific issue?

sschuberth commented 2 years ago

I can confirm the bug. There's an open source package to easy reproduce it and can be used as reference: https://github.com/fastavro/fastavro

Interestingly, that project exclusively uses a developer_requirements.txt file, which does even trigger the code path to run pydep in ORT. So I'm wondering how you could be seeing

File "/tmp/ort-Pip-code-virtualenv2432521012921555598/bin/pydep-run.py", line 161, in <module>
sschuberth commented 2 years ago

So, can you guys give #5247 a try to confirm it fixes this specific issue?

Feel free to reopen if this exact same issue still appears.

HardikRa commented 1 year ago

I ran into the same error while trying to scan the repository for Django ort/cli/build/install/ort/bin/ort -P ort.analyzer.allowDynamicVersions=true analyze -i /home/complab304pc30/Documents/django -o /home/djangoORTAnalyzerOutput -f JSON

This is the output for the same

/        \_______   \__    ___/ The OSS Review Toolkit, version 9e1d31e224.
|    |   | |       _/ |    |
|    |   | |    |   \ |    |    Running 'analyze' as 'complab304pc30' under Java 17.0.5 on Linux
\________/ |____|___/ |____|    with 20 CPUs and a maximum of 7968 MiB of memory.

Environment variables:
ORT_CONFIG_DIR = /home/complab304pc30/.ort/config
ORT_DATA_DIR = /home/complab304pc30/.ort
SHELL = /bin/bash
TERM = xterm-256color
GOPATH = /home/complab304pc30/go

Looking for ORT configuration in the following file:
        /home/complab304pc30/.ort/config/config.yml (does not exist)

Looking for analyzer-specific configuration in the following files and directories:
        /home/complab304pc30/Documents/django/.ort.yml (does not exist)
        /home/complab304pc30/.ort/config/resolutions.yml (does not exist)
The following package managers are enabled:
        Bower, Bundler, Cargo, Carthage, CocoaPods, Composer, Conan, DotNet, GoDep, GoMod, Gradle, Maven, NPM, NuGet, PIP, Pipenv, PNPM, Poetry, Pub, SBT, SpdxDocumentFile, Stack, Unmanaged, Yarn, Yarn2
The following package curation providers are enabled:
        DefaultDir, DefaultFile
Analyzing project path:
        /home/complab304pc30/Documents/django
Found 1 NPM definition file(s) at:
        package.json
Found 3 PIP definition file(s) at:
        setup.py
        docs/requirements.txt
        tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt
Found 4 definition file(s) from 2 package manager(s) in total.
15:57:24.423 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/setup.py': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.648 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.utils.common.CommandLineTool - The command is required in version >=6.0.0 and <9.0.0, but you are using version 9.3.1. This could lead to problems.
15:57:24.658 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'setup.py' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.662 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/docs/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.677 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'docs/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.679 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:24.688 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
15:57:25.312 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - '/home/complab304pc30/Documents/django/package.json' does not define a version.
15:57:32.353 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
15:57:32.353 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
Writing analyzer result to '/home/complab304pc30/Documents/djangoORTAnalyzerOutput/analyzer-result.json'.
The analysis took 8.122189590s.
Found 4 project(s) and 241 package(s) in total (not counting excluded ones).
Applied 0 curation(s) from 2 provider(s).
Resolved issues: 0 errors, 0 warnings, 0 hints.
Unresolved issues: 3 errors, 0 warnings, 0 hints.
There are 3 unresolved issues with a severity equal to or greater than the WARNING threshold.

The following is the output when run with --stacktrace 

 ______________________________
/        \_______   \__    ___/ The OSS Review Toolkit, version 9e1d31e224.
|    |   | |       _/ |    |
|    |   | |    |   \ |    |    Running 'analyze' as 'complab304pc30' under Java 17.0.5 on Linux
\________/ |____|___/ |____|    with 20 CPUs and a maximum of 7968 MiB of memory.

Environment variables:
ORT_CONFIG_DIR = /home/complab304pc30/.ort/config
ORT_DATA_DIR = /home/complab304pc30/.ort
SHELL = /bin/bash
TERM = xterm-256color
GOPATH = /home/complab304pc30/go

Looking for ORT configuration in the following file:
        /home/complab304pc30/.ort/config/config.yml (does not exist)

Looking for analyzer-specific configuration in the following files and directories:
        /home/complab304pc30/Documents/django/.ort.yml (does not exist)
        /home/complab304pc30/.ort/config/resolutions.yml (does not exist)
The following package managers are enabled:
        Bower, Bundler, Cargo, Carthage, CocoaPods, Composer, Conan, DotNet, GoDep, GoMod, Gradle, Maven, NPM, NuGet, PIP, Pipenv, PNPM, Poetry, Pub, SBT, SpdxDocumentFile, Stack, Unmanaged, Yarn, Yarn2
The following package curation providers are enabled:
        DefaultDir, DefaultFile
Analyzing project path:
        /home/complab304pc30/Documents/django
Found 1 NPM definition file(s) at:
        package.json
Found 3 PIP definition file(s) at:
        setup.py
        docs/requirements.txt
        tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt
Found 4 definition file(s) from 2 package manager(s) in total.
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:07.932 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/setup.py': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.129 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.utils.common.CommandLineTool - The command is required in version >=6.0.0 and <9.0.0, but you are using version 9.3.1. This could lead to problems.
16:08:08.161 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'setup.py' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.166 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/docs/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.178 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'docs/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/docs"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.180 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.managers.Pip - Unable to determine dependencies for definition file '/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt': IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
java.io.IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1143)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:95)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:33)
        at org.ossreviewtoolkit.utils.common.ProcessCapture.<init>(ProcessCapture.kt:41)
        at org.ossreviewtoolkit.utils.common.CommandLineTool$DefaultImpls.run(CommandLineTool.kt:82)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:57)
        at org.ossreviewtoolkit.analyzer.managers.utils.PythonInspector.run(PythonInspector.kt:105)
        at org.ossreviewtoolkit.analyzer.managers.Pip.runPythonInspector(Pip.kt:119)
        at org.ossreviewtoolkit.analyzer.managers.Pip.resolveDependencies(Pip.kt:101)
        at org.ossreviewtoolkit.analyzer.PackageManager.resolveDependencies(PackageManager.kt:307)
        at org.ossreviewtoolkit.analyzer.PackageManagerRunner$run$3.invokeSuspend(Analyzer.kt:339)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
        at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Caused by: java.io.IOException: error=2, No such file or directory
        at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
        at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:314)
        at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:244)
        at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1110)
        ... 19 more
16:08:08.186 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'tests/admin_scripts/custom_templates/project_template/additional_dir/requirements.txt' failed with: IOException: Cannot run program "python-inspector" (in directory "/home/complab304pc30/Documents/django/tests/admin_scripts/custom_templates/project_template/additional_dir"): error=2, No such file or directory
Caused by: IOException: error=2, No such file or directory
16:08:08.784 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - '/home/complab304pc30/Documents/django/package.json' does not define a version.
16:08:15.788 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
16:08:15.788 [DefaultDispatcher-worker-1] WARN  org.ossreviewtoolkit.analyzer.managers.Npm - The '/home/complab304pc30/Documents/django/package.json' does not set a version, which is only allowed for unpublished packages.
Writing analyzer result to '/home/complab304pc30/Documents/djangoORTAnalyzerOutput/analyzer-result.json'.
The analysis took 8.031969722s.
Found 4 project(s) and 241 package(s) in total (not counting excluded ones).
Applied 0 curation(s) from 2 provider(s).
Resolved issues: 0 errors, 0 warnings, 0 hints.
Unresolved issues: 3 errors, 0 warnings, 0 hints.
There are 3 unresolved issues with a severity equal to or greater than the WARNING threshold.
sschuberth commented 1 year ago

@HardikRa, the ORT distribution (in contrast to the Docker image) does not come with third-party tools as required by the analyzer. In particular, you need to ensure that the python-inspector Python package is installed and available in the PATH environment of the user running ort.

HardikRa commented 1 year ago

I have installed the python-inspector package and added it to the PATH, it resolved most errors, however there is still one error that popped up

11:38:27.784 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Resolving PIP dependencies for path 'setup.py' failed with: IllegalArgumentException: Unable to determine a project name for '/home/complab304pc30/Documents/django/setup.py'.
sschuberth commented 1 year ago

@HardikRa that's a new issue and I've tracked it as such.