weekly validation tests failing #532

Closed majamassarini closed 9 months ago

What happened? What is the problem?

Weekly validation tests are all failing because of the following exception. Probably there is something wrong with the gitlab.gnome instance but we should be able to skip it and go further with testing other instances. And the failure on the gitlab.gnome instance should be notified in some way.

INFO:root:GITLAB_TOKEN not set, skipping the validation for GitLab.
INFO:root:Running validation for GitLab (gitlab.gnome.org instance).
INFO:root:Run testcases where the build is triggered by a '/packit build' comment
Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/gitlab/exceptions.py", line 311, in wrapped_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/gitlab/mixins.py", line 145, in get
    server_data = self.gitlab.http_get(self.path, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/gitlab/client.py", line 765, in http_get
    result = self.http_request(
             ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/gitlab/client.py", line 731, in http_request
    raise gitlab.exceptions.GitlabHttpError(
gitlab.exceptions.GitlabHttpError: 503: <html>
  <head>
    <meta name="viewport" content="width=device-width, initial-scale=1">

    <style type="text/css">
      body {
        font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
        line-height: 1.66666667;
        font-size: 16px;
        color: #333;
        background-color: #fff;
        margin: 2em 1em;
      }
      h1 {
        font-size: 28px;
        font-weight: 400;
      }
      p {
        margin: 0 0 10px;
      }
      .alert.alert-info {
        background-color: #F0F0F0;
        margin-top: 30px;
        padding: 30px;
      }
      .alert p {
        padding-left: 35px;
      }
      ul {
        padding-left: 51px;
        position: relative;
      }
      li {
        font-size: 14px;
        margin-bottom: 1em;
      }
      p.info {
        position: relative;
        font-size: 20px;
      }
      p.info:before, p.info:after {
        content: "";
        left: 0;
        position: absolute;
        top: 0;
      }
      p.info:before {
        background: #0066CC;
        border-radius: 16px;
        color: #fff;
        content: "i";
        font: bold 16px/24px serif;
        height: 24px;
        left: 0px;
        text-align: center;
        top: 4px;
        width: 24px;
      }

      @media (min-width: 768px) {
        body {
          margin: 6em;
        }
      }
    </style>
  </head>
  <body>
    <div>
      <h1>Application is not available</h1>
      <p>The application is currently not serving requests at this endpoint. It may not have been started or is still starting.</p>

      <div class="alert alert-info">
        <p class="info">
          Possible reasons you are seeing this page:
        </p>
        <ul>
          <li>
            <strong>The host doesn't exist.</strong>
            Make sure the hostname was typed correctly and that a route matching this hostname exists.
          </li>
          <li>
            <strong>The host exists, but doesn't have a matching path.</strong>
            Check if the URL path was typed correctly and that the route was created using the desired path.
          </li>
          <li>
            <strong>Route and path matches, but all pods are down.</strong>
            Make sure that the resources exposed by this route (pods, services, deployment configs, etc) have at least one pod running.
          </li>
        </ul>
      </div>
    </div>
  </body>
</html>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 108, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/services/gitlab/pull_request.py", line 242, in get_list
    mrs = project.gitlab_repo.mergerequests.list(
          ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/services/gitlab/project.py", line 57, in gitlab_repo
    self._gitlab_repo = self.service.gitlab_instance.projects.get(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/services/gitlab/service.py", line 55, in gitlab_instance
    self._gitlab_instance.auth()
  File "/usr/lib/python3.11/site-packages/gitlab/client.py", line 347, in auth
    self.user = self._objects.CurrentUserManager(self).get()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/gitlab/v4/objects/users.py", line 146, in get
    return cast(Optional[CurrentUser], super().get(id=id, **kwargs))
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/gitlab/exceptions.py", line 313, in wrapped_f
    raise error(e.error_message, e.response_code, e.response_body) from e
gitlab.exceptions.GitlabGetError: 503: <html>
  <head>
    <meta name="viewport" content="width=device-width, initial-scale=1">

    <style type="text/css">
      body {
        font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
        line-height: 1.66666667;
        font-size: 16px;
        color: #333;
        background-color: #fff;
        margin: 2em 1em;
      }
      h1 {
        font-size: 28px;
        font-weight: 400;
      }
      p {
        margin: 0 0 10px;
      }
      .alert.alert-info {
        background-color: #F0F0F0;
        margin-top: 30px;
        padding: 30px;
      }
      .alert p {
        padding-left: 35px;
      }
      ul {
        padding-left: 51px;
        position: relative;
      }
      li {
        font-size: 14px;
        margin-bottom: 1em;
      }
      p.info {
        position: relative;
        font-size: 20px;
      }
      p.info:before, p.info:after {
        content: "";
        left: 0;
        position: absolute;
        top: 0;
      }
      p.info:before {
        background: #0066CC;
        border-radius: 16px;
        color: #fff;
        content: "i";
        font: bold 16px/24px serif;
        height: 24px;
        left: 0px;
        text-align: center;
        top: 4px;
        width: 24px;
      }

      @media (min-width: 768px) {
        body {
          margin: 6em;
        }
      }
    </style>
  </head>
  <body>
    <div>
      <h1>Application is not available</h1>
      <p>The application is currently not serving requests at this endpoint. It may not have been started or is still starting.</p>

      <div class="alert alert-info">
        <p class="info">
          Possible reasons you are seeing this page:
        </p>
        <ul>
          <li>
            <strong>The host doesn't exist.</strong>
            Make sure the hostname was typed correctly and that a route matching this hostname exists.
          </li>
          <li>
            <strong>The host exists, but doesn't have a matching path.</strong>
            Check if the URL path was typed correctly and that the route was created using the desired path.
          </li>
          <li>
            <strong>Route and path matches, but all pods are down.</strong>
            Make sure that the resources exposed by this route (pods, services, deployment configs, etc) have at least one pod running.
          </li>
        </ul>
      </div>
    </div>
  </body>
</html>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/packit-service-validation.py", line 714, in <module>
    ).run()
      ^^^^^
  File "/usr/bin/packit-service-validation.py", line 632, in run
    for pr in self.project.get_pr_list()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 108, in wrapper
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/utils.py", line 199, in indirectly_called
    return specialized_function(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 120, in wrapper
    __check_for_internal_failure(__wrap_exception(ex))
  File "/usr/lib/python3.11/site-packages/ogr/abstract.py", line 59, in __check_for_internal_failure
    raise GitForgeInternalError from ex.__cause__
ogr.exceptions.GitForgeInternalError
Sentry is attempting to send 2 pending events
Waiting up to 2 seconds
Press Ctrl-C to quit

What did you expect to happen?

No response

Example URL(s)

No response

Steps to reproduce

No response

Workaround

[ ] There is an existing workaround that can be used until this issue is fixed.

Participation

[ ] I am willing to submit a pull request for this issue. (Packit team is happy to help!)

It looks like the instance is down, not sure what we can do about that :/

It looks like the instance is down, not sure what we can do about that :/

I think we should try/except the exception and go further with the validation for other instances. And maybe send some feedback that the instance is down.

Should be fixed with packit/validation#8

packit / deployment