thoth-station / adviser

The recommendation engine for Python software stacks and Dependency Monkey in project Thoth.
https://thoth-station.github.io
GNU General Public License v3.0
34 stars 13 forks source link

Provide stack info (if any) even when the resolution fails #1478

Closed fridex closed 3 years ago

fridex commented 4 years ago

Is your feature request related to a problem? Please describe.

As discussed with @pacospace, we can provide stack information even when the resolution fails. This stack information can have valuable data to be presented to the user (e.g. CVEs found in the stack that could lead to resolution failures when security recommendation type was requested).

Describe the solution you'd like

The resulting JSON captures stack information even when the resolution fails. Thamos shows this stack info to the user even if the resolution fails.

Describe alternatives you've considered

Refer users to the resolution logs as done now, but it requires human reading of the logs. Moreover, we could machine-analyze the failed resolutions (stack info in the resolution output) even if the resolution fails.

pacospace commented 3 years ago

As discussed in the chat, using adviser 0.19.0 still does not show report but only:

2020-10-27 14:14:51,408 [220900] INFO     thamos.lib: Performing static analysis of sources to gather library usage
2020-10-27 14:14:52,329 [220900] INFO     thamos.lib: Successfully submitted advise analysis 'adviser-2ea1ebc3' to 'https://khemenu.thoth-station.ninja/api/v1'
Resolution was terminated based on errors encountered; see logs for more info - https://thoth-station.ninja/j/error_logs

but if we run thamos logs:

* Fixes an undefined behavior causing a segfault in `tf.raw_ops.Switch` (CVE-2020-15190)
* Fixes three vulnerabilities in conversion to DLPack format (CVE-2020-15191, CVE-2020-15192, CVE-2020-15193)
* Fixes two vulnerabilities in `SparseFillEmptyRowsGrad` (CVE-2020-15194, CVE-2020-15195)
* Fixes several vulnerabilities in `RaggedCountSparseOutput` and `SparseCountSparseOutput` operations (CVE-2020-15196, CVE-2020-15197, CVE-2020-15198, CVE-2020-15199, CVE-2020-15200, CVE-2020-15201) 
* Fixes an integer truncation vulnerability in code using the work sharder API (CVE-2020-15202)
* Fixes a format string vulnerability in `tf.strings.as_string` (CVE-2020-15203)
* Fixes segfault raised by calling session-only ops in eager mode (CVE-2020-15204)
* Fixes data leak and potential ASLR violation from `tf.raw_ops.StringNGrams` (CVE-2020-15205)
* Fixes segfaults caused by incomplete `SavedModel` validation (CVE-2020-15206)
* Fixes a data corruption due to a bug in negative indexing support in TFLite (CVE-2020-15207)
* Fixes a data corruption due to dimension mismatch in TFLite (CVE-2020-15208)
* Fixes several vulnerabilities in TFLite saved model format (CVE-2020-15209, CVE-2020-15210, CVE-2020-15211)
* Fixes several vulnerabilities in TFLite implementation of segment sum (CVE-2020-15212, CVE-2020-15213, CVE-2020-15214)
* Updates `sqlite3` to `3.33.00` to handle (CVE-2020-15358)
2020-10-27 12:30:39,817 WARNING: No more possible paths found for resolution, terminating resolver in iteration 1, see - https://thoth-station.ninja/j/no_paths
2020-10-27 12:30:39,817 INFO: Resolver took 7.55568 seconds in total, pipeline speed 0 stacks/second
2020-10-27 12:30:39,817 INFO: Pipeline strides discarded 0 and accepted 0 final states in total
2020-10-27 12:30:39,818 ERROR: Resolver run failed: Resolver did not find any stack that would satisfy requirements and stack characteristics given the time allocated- see https://thoth-station.ninja/j/no_stack
2020-10-27 12:30:39,870 ERROR: Child exited with exit code 256
2020-10-27 12:30:39,875 ERROR: Resolution was terminated based on errors encountered; see logs for more info - https://thoth-station.ninja/j/error_logs
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
tensorflow = "==2.3.0"

[dev-packages]

[requires]
python_version = "3.6"
host: khemenu.thoth-station.ninja
tls_verify: true
requirements_format: pipenv

runtime_environments:
  - name: ubi8
    operating_system:
      name: ubi
      version: "8"
    python_version: "3.6"
    recommendation_type: security
pacospace commented 3 years ago

I repeated with a different stack and recommendation and actually when there are errors in this case it works (maybe is a security recommendation issue??) but links are broken (we have another issue for that I think already):

                                                                               Application stack guidance                                                                              
                                  ╷                                                                                                                                                  ╷ 
  Link                            │ Message                                                                                                                                          │ 
 ═════════════════════════════════╪══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╪═
  https://thoth-station.ninja/j/… │ Using observations for RHEL instead of UBI, RHEL is ABI compatible with UBI                                                                      │ 
  https://thoth-station.ninja/j/… │ No platform provided in the configuration, setting to 'linux-x86_64' implicitly                                                                  │ 
  https://thoth-station.ninja/j/… │ No versions were found for direct dependency 'pandas'; operating system 'rhel' in OS version '8' for Python in version '3.7' using platform 'li… │ 
  https://thoth-station.ninja/j/… │ Resolver failed as it was unable to resolve direct dependencies      
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pandas = "*"

[requires]
python_version = "3.6"
host: khemenu.thoth-station.ninja
tls_verify: true
requirements_format: pipenv

runtime_environments:
  - name: ubi8
    operating_system:
      name: ubi
      version: "8"
    python_version: "3.7"
    recommendation_type: latest
fridex commented 3 years ago

I repeated with a different stack and recommendation and actually when there are errors in this case it works (maybe is a security recommendation issue??)

hm, that's strange... will try to look into this, thanks :+1:

but links are broken (we have another issue for that I think already):

+1, related https://github.com/thoth-station/thamos/issues/541