search

syndesisio / syndesis-rest

The API for Syndesis - a flexible, customizable, cloud-hosted platform that provides core integration capabilities as a service. It leverages Red Hat's existing product architecture using OpenShift Online/Dedicated and Fuse Integration Services.
https://syndesis-staging.b6ff.rh-idev.openshiftapps.com/api/v1/
Apache License 2.0
6 stars 17 forks source link

Getting occasional 503 #222

Closed gashcrumb closed 7 years ago

gashcrumb commented 7 years ago

Seems to be two cases, one is when looking at connection details:

{
  "timestamp": 1490206753908,
  "status": 500,
  "error": "Internal Server Error",
  "exception": "org.jboss.resteasy.spi.UnhandledException",
  "message": "java.lang.NullPointerException: position",
  "path": "\/api\/v1\/connections\/7"
}

But I also get a 503 occasionally when trying to fetch an individual integration, for example:

URL -> https://ipaas-staging.b6ff.rh-idev.openshiftapps.com/api/v1/integrations/1

<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>

This was actually on the pre-flight OPTIONS methond, not even the subsequent GET.

kahboom commented 7 years ago

@jludvice and I are also getting this for most of the day. I can't even work locally as I'm using the staging API. @jimmidyson would say this is why we should be running the API locally lol but shhh...

iocanel commented 7 years ago

I am also hitting that issue all the time.

KurtStam commented 7 years ago

Is it because the token is expiring too soon? Do we have the full stack trace? Just wondering if it's auth related.

gashcrumb commented 7 years ago

I don't think it has anything to do with the token, as it can fail right away after refreshing the browser etc. Unfortunately no info really comes back to the client beyond what I've reported.

On Fri, Mar 24, 2017 at 10:36 AM, Kurt T Stam notifications@github.com wrote:

Is it because the token is expiring too soon? Do we have the full stack trace? Just wondering if it's auth related.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/redhat-ipaas/ipaas-rest/issues/222#issuecomment-289040059, or mute the thread https://github.com/notifications/unsubscribe-auth/AAVdrMpmONuC4z-OGF-x31cqRbgQs9QXks5ro9T4gaJpZM4MlocL .

jludvice commented 7 years ago

Hello @gashcrumb

It might be unrelated, but @dsimansk mentioned some bug in openshift router. "drop create" of applications/routes might cause "router panic" and weird issues in routing IIRC. Is it possible that we're hitting this bug?

gashcrumb commented 7 years ago

@jludvice ooh, maybe, @iocanel @KurtStam WDYT?

jludvice commented 7 years ago

@dsimansk is this the bug you were talking about? https://bugzilla.redhat.com/show_bug.cgi?id=1371826

iocanel commented 7 years ago

Well, in my setup I don't edit nor delete routes, so I am not sure if this is related.

gashcrumb commented 7 years ago

K, things seem worse, it seems for /api/v1/integrations we now always get back:

{
  "timestamp": 1490380713212,
  "status": 500,
  "error": "Internal Server Error",
  "exception": "org.jboss.resteasy.spi.UnhandledException",
  "message": "com.redhat.ipaas.core.IPaasServerException: An error has occurred.",
  "path": "\/api\/v1\/integrations"
}
dsimansk commented 7 years ago

@jludvice there's newer issue against 3.4.1. It's probably not related to this issues, the effect of router panic consequently makes all routes->app unreachable permanently until router is restarted.

@jimmidyson do have enough permissions to access router pod logs or support ticket is needed?

jimmidyson commented 7 years ago

Any of us (as dedicated admins) should have access to router pod logs afaik.

jimmidyson commented 7 years ago

No looks like we can't view anything in default namespace... need to raise a ticket with openshift-provisioning@...

jimmidyson commented 7 years ago

This should now be fixed on staging by scaling down/up router pods (thanks ops!). Bug hit is apparently https://bugzilla.redhat.com/show_bug.cgi?id=1429823