search

thoth-station / user-api

Web Service exposing Thoth functions to the RESTful worlds
https://thoth-station.github.io/
GNU General Public License v3.0
8 stars 18 forks source link

Container images endpoint results in 500 #1645

Closed fridex closed 2 years ago

fridex commented 2 years ago

Describe the bug

Try to access /countainer-images endpoint. It looks like user-api has issues with thoth-storages version (might be that the library was not updated).

To Reproduce Steps to reproduce the behavior:

  1. Access /container-images endpoint in any deployment
  2. See the inrnal error

Expected behavior

A listing of available container images should be available.

fridex commented 2 years ago

/priority critical-urgent /assign @mayaCostantini

harshad16 commented 2 years ago

This is still happening with new release: This is User API v0.34.1+messaging.0.16.0.storages.0.68.2.common.0.34.0.python.0.16.6

log copy: user-api-181-6xgp5-user-api.log

tested on test cluster: https://test.thoth-station.ninja/api/v1/ui/#/Container%20Images/list_thoth_container_images https://console-openshift-console.apps.ocp4.prod.psi.redhat.com/k8s/ns/thoth-test-core/pods/user-api-181-6xgp5/logs

harshad16 commented 2 years ago

The above can be due to data not being available Deploy on stage and testing this there.

harshad16 commented 2 years ago

The changes are deployed on the stage cluster: https://stage.thoth-station.ninja/api/v1/ui/#/Container%20Images/list_thoth_container_images

https://console-openshift-console.apps.ocp4.prod.psi.redhat.com/k8s/ns/thoth-frontend-stage/pods/user-api-22-rscdv/logs it runs successfully, we remove all default place holders.

with the set place holders it errors out with 500

Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/opt/app-root/lib64/python3.8/site-packages/flask_cors/extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/app-root/lib64/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/app-root/lib64/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/decorator.py", line 68, in wrapper
    response = function(request)
  File "/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/uri_parsing.py", line 149, in wrapper
    response = function(request)
  File "/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/validation.py", line 396, in wrapper
    return function(request)
  File "/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/response.py", line 109, in wrapper
    response = function(request)
  File "/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/parameter.py", line 116, in wrapper
    return function(**kwargs)
  File "/opt/app-root/src/thoth/user_api/api_v1.py", line 241, in list_thoth_container_images
    entries_count = GRAPH.get_software_environments_count_all(
  File "/opt/app-root/lib64/python3.8/site-packages/thoth/storages/graph/postgres.py", line 2869, in get_software_environments_count_all
    query = self._construct_software_environments_query(
  File "/opt/app-root/lib64/python3.8/site-packages/thoth/storages/graph/postgres.py", line 3041, in _construct_software_environments_query
    query = query.join(PythonPackageVersion).filter(PythonPackageVersion.package_name == package_name)
  File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/query.py", line 2391, in join
    return self._join(
  File "<string>", line 2, in _join
  File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/base.py", line 227, in generate
    fn(self, *args[1:], **kw)
  File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/query.py", line 2587, in _join
    self._join_left_to_right(
  File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/query.py", line 2611, in _join_left_to_right
    ) = self._join_determine_implicit_left_side(left, right, onclause)
  File "/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/query.py", line 2710, in _join_determine_implicit_left_side
    raise sa_exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Don't know how to join to <class 'thoth.storages.graph.models.PythonPackageVersion'>. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
harshad16 commented 2 years ago

failing statement: https://github.com/thoth-station/storages/blob/9d9f35575f3811c7dd2be6601e5d6d5d21558d43/thoth/storages/graph/postgres.py#L3041

we are trying to filter package_name between python_package_version and software_environment https://github.com/thoth-station/storages/blob/9d9f35575f3811c7dd2be6601e5d6d5d21558d43/thoth/storages/graph/models.py#L62 however, we don't have package_name in software_environment https://github.com/thoth-station/storages/blob/master/thoth/storages/graph/models.py#L804

harshad16 commented 2 years ago

We might have to get the package_name based on the match on package_extract run and filtering it out of a join of different tables. package_extract_run +identified table + python_package_version_entity

harshad16 commented 2 years ago

filter statements seems to be having issues:

 "Traceback (most recent call last):\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/flask/app.py\", line 2447, in wsgi_app\n    response = self.full_dispatch_request()\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/flask/app.py\", line 1952, in full_dispatch_request\n    rv = self.handle_user_exception(e)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/flask_cors/extension.py\", line 165, in wrapped_function\n    return cors_after_request(app.make_response(f(*args, **kwargs)))\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/flask/app.py\", line 1821, in handle_user_exception\n    reraise(exc_type, exc_value, tb)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/flask/_compat.py\", line 39, in reraise\n    raise value\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/flask/app.py\", line 1950, in full_dispatch_request\n    rv = self.dispatch_request()\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/flask/app.py\", line 1936, in dispatch_request\n    return self.view_functions[rule.endpoint](**req.view_args)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/decorator.py\", line 68, in wrapper\n    response = function(request)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/uri_parsing.py\", line 149, in wrapper\n    response = function(request)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/validation.py\", line 396, in wrapper\n    return function(request)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/response.py\", line 109, in wrapper\n    response = function(request)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/connexion/decorators/parameter.py\", line 116, in wrapper\n    return function(**kwargs)\n",
        "  File \"/opt/app-root/src/thoth/user_api/api_v1.py\", line 241, in list_thoth_container_images\n    entries_count = GRAPH.get_software_environments_count_all(\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/thoth/storages/graph/postgres.py\", line 2869, in get_software_environments_count_all\n    query = self._construct_software_environments_query(\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/thoth/storages/graph/postgres.py\", line 3050, in _construct_software_environments_query\n    query.join(PackageExtractRun)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/query.py\", line 2391, in join\n    return self._join(\n",
        "  File \"<string>\", line 2, in _join\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/base.py\", line 227, in generate\n    fn(self, *args[1:], **kw)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/query.py\", line 2587, in _join\n    self._join_left_to_right(\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/query.py\", line 2643, in _join_left_to_right\n    orm_join(\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/util.py\", line 1158, in join\n    return _ORMJoin(left, right, onclause, isouter, full)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/orm/util.py\", line 1057, in __init__\n    expression.Join.__init__(self, left, right, onclause, isouter, full)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/sql/selectable.py\", line 765, in __init__\n    self.onclause = self._match_primaries(self.left, self.right)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/sql/selectable.py\", line 914, in _match_primaries\n    return self._join_condition(left, right, a_subset=left_right)\n",
        "  File \"<string>\", line 2, in _join_condition\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/util/deprecations.py\", line 139, in warned\n    return fn(*args, **kwargs)\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/sql/selectable.py\", line 964, in _join_condition\n    cls._joincond_trim_constraints(\n",
        "  File \"/opt/app-root/lib64/python3.8/site-packages/sqlalchemy/sql/selectable.py\", line 1080, in _joincond_trim_constraints\n    raise exc.AmbiguousForeignKeysError(\n",
        "sqlalchemy.exc.AmbiguousForeignKeysError: Can't determine join between 'Join object on Join object on Join object on Join object on Join object on Join object on Join object on software_environment(140034398813056) and has_symbol(140034398130432)(140034077513232) and versioned_symbol(140034398070720)(140034077513328) and has_symbol(140034398130432)(140034077513808) and versioned_symbol(140034398070720)(140034077513856) and package_extract_run(140034399829200)(140034077514336) and identified(140034398967216)(140034430521648) and python_package_version_entity(140034400174912)' and 'package_extract_run'; tables have more than one foreign key constraint relationship between them. Please specify the 'onclause' of this join explicitly.\n"
      ]
harshad16 commented 2 years ago

library_name, symbol and rpm_package_name queries are also failing https://github.com/thoth-station/storages/blob/054a7bac0c0aa1923c11d1366467aefe60b7a4b7/thoth/storages/graph/postgres.py#L3034 https://github.com/thoth-station/storages/blob/054a7bac0c0aa1923c11d1366467aefe60b7a4b7/thoth/storages/graph/postgres.py#L3037 https://github.com/thoth-station/storages/blob/054a7bac0c0aa1923c11d1366467aefe60b7a4b7/thoth/storages/graph/postgres.py#L3047

mayaCostantini commented 2 years ago

library_name, symbol and rpm_package_name queries are also failing

This should be solved by https://github.com/thoth-station/storages/pull/2575/commits/d1b74abdd614e253c4ec800bc2b58881bf138131

harshad16 commented 2 years ago

The spin-up of user-api v0.34.3 is failing with the following error:

[2022-02-07 16:12:10 +0000] [1] [INFO] Handling signal: term
[2022-02-07 16:12:10 +0000] [23] [INFO] Worker exiting (pid: 23)
[2022-02-07 16:12:10 +0000] [22] [INFO] Worker exiting (pid: 22)
[2022-02-07 16:12:12 +0000] [1] [INFO] Shutting down: Master

log: user-api-28-pk4kc-user-api.log

harshad16 commented 2 years ago

we would have to check on this.

fridex commented 2 years ago

we would have to check on this.

https://github.com/thoth-station/user-api/pull/1663 might fix this:

Semantic error at components.schemas.PythonPackageVersionMetadataResponse.properties.metadata.properties.importlib_metadata.properties.files
Schemas with 'type: array', require a sibling 'items: ' field
Jump to line 3281
fridex commented 2 years ago

The issue seems to be port:

(app-root) sh-4.4$ curl localhost:8080/readiness
curl: (7) Failed to connect to localhost port 8080: Connection refused
(app-root) sh-4.4$ curl localhost:8000/readiness
{"status":"ready","version":"0.34.4+messaging.0.16.0.storages.0.68.3.common.0.34.0.python.0.16.6"}

It looks like we used 8080 before but now it is 8000. Not sure what has changed as we do not touch this configuration explicitly. It might be the base image started exposing HTTP on a different port. The docs say 8080 is the default though.

harshad16 commented 2 years ago

Tested this on stage 

curl -X 'GET' \
  'https://stage.thoth-station.ninja/api/v1/container-images?page=0&per_page=25&os_name=ubi&os_version=8&python_version=3.8' \
  -H 'accept: application/json'

Works as desired. Thanks for all the hard work :100: