search

xmidt-org / svalinn

The shield to protect our users from incoming events. Part of codex.
Apache License 2.0
1 stars 7 forks source link

Error parsing Bearer Token #158

Closed pedrojreis closed 2 years ago

pedrojreis commented 2 years ago

Hey hey! :)

So i'm diving into Installing the Codex and I've some doubts that might explain an error i'm getting.

If I understand correctly the registrationURL should be tr1d1um/api/v3/hook, correct? I believe it could also be Caduceus/hook but for some strange reason I keep getting 404 and tr1d1um is currently working ok for a side app I have.

Now I'm trying to use JWT all over the WebPA stack and it seems to work ok for the most part expect for svalling that I keep getting

{"error":"error parsing bearer token from http response body: unable to parse bearer token: invalid character 'e' looking for beginning of value","level":"error","msg":"Failed to register webhook","ts":"2022-07-12T18:17:11.527149403Z"}

In my svalinn 0.14.3 configuration I have this setup:

########################################
#   Webhook Registration Related Configuration
########################################

# webhook provides the information needed to register to a webhook.  If the 
# urls and event regex aren't empty and the interval is greater than 0, 
# registration occurs.
# (Optional)
webhook:
  # registrationInterval provides the time to wait between each registration to 
  # the webhook.  If this is set to 0, no registration is done.
  # (Optional)
  registrationInterval: "4m"

  # timeout provides the length of time the request should wait before timing 
  # out.
  timeout: "1m"

  # registrationURL provides the place to register the webhook.
  registrationURL: "http://tr1d1um.namespace:443/api/v3/hook"

  # request provides the information passed in the webhook registration request.
  request:
    # config provides configuration values for the requests to the webhook 
    # receiver.
    config:
      # url provides the server the webhook should send the events to.  If this 
      # value isn't set, no registration happens.
      url: "http://svalinn.namespace:7100/api/v1/device-status"

      # contentType provides the content type Svalinn expects from the requests 
      # it receives.
      # (Optional) defaults to "wrp"
      contentType: "wrp"

      # maxRetryCount is the number of times to retry on a failure.
      # (Optional)
      maxRetryCount: 3

    # events provides a list of regular expressions that tells the webhook 
    # which endpoints to send to Svalinn.  If the destination of an event 
    # matches a regular expression in this list, it is sent to Svalinn
    events: [".*"]

  # jwt provides a way to use Bearer Authorization when registering to a 
  # webhook.  If the below values are all provided, a request is made to the 
  # URL to get the token to be used in the registration request.  The 
  # header would look like: 
  # 
  # Authorization Bearer {token}
  # 
  # (Optional)
  jwt:
    # requestHeaders are added to the request for the token.
    # (Optional)
    # requestHeaders:
    #   "": ""

    # authURL is the URL to access for the token.
    authURL: "http://themis.namespace:6501/issue"

    # timeout is how long the request to get the token will take before 
    # timing out.
    timeout: "1m"

    # buffer is the length of time before a token expires to get a new token.
    # (Optional)
    buffer: "2m"

Any help on this matter would be highly appreciated :D

schmidtw commented 2 years ago

The bearer token you've passed in isn't correct. This is the most relevant line of code: https://github.com/xmidt-org/bascule/blob/d64a1d71a1e3702e1959ce81f885ec61b5425e69/acquire/parsers.go#L46

It looks like when the JWT token is sent into the JSON unmarshal function, it's not valid JSON.

pedrojreis commented 2 years ago

That looks logic, i've altered the authURL: o the themis's /key/{key}/key.json and now i'm getting a 401 from tr1d1um side :P

"auth":"Bearer","error":"unexpected authorization header value"

I Imagine that i'm missing something from Tr1d1um configuration, but some variables are not clear on what they do :P

I'll attempt to figure it out :)

Thanks for the help @schmidtw

schmidtw commented 2 years ago

Yeah, the thing I'd do is add some debug statements around the failure & use something like pp to output what you think is the JWT. I agree the fields aren't documented well enough. We've been discussing how to do that better given the distributed nature, but it needs to improve.

On Fri, Jul 15, 2022 at 8:39 AM Pedro Reis @.***> wrote:

That looks logic, i've altered the authURL: o the themis's /key/{key}/key.json and now i'm getting a 401 from tr1d1um side :P

"auth":"Bearer","error":"unexpected authorization header value"

I Imagine that i'm missing something from Tr1d1um configuration, but some variables are not clear on what they do :P

I'll attempt to figure it out :)

Thanks for the help @schmidtw

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

pedrojreis commented 2 years ago

I'll have to boot up a YugaByteDB locally and i'll try to debug it locally. Can you clear something for me please, is svalinn supposed to register Tr1d1um webhook or Caduceus? I'm really confused on this and when testing both (With Basic Auth as I'm yet to successfully configure Jwt :P ) :

schmidtw commented 2 years ago

it should register with either tr1d1um or scytale endpoints. Ideally scytale is more correct.

You need to provide a duration. In the webhook registration I'd register a duration of 5m. You can customize the value, but this is what we run with.

Internally, scytale/argus use the until value to denote when a record expires.

Caduceus providing webhook registration was a bug of some releases

On Mon, Jul 18, 2022 at 10:42 AM Pedro Reis @.***> wrote:

I'll have to boot up a YugaByteDB locally and i'll try to debug it locally. Can you clear something for me please, is svalinn supposed to register Tr1d1um webhook or Caduceus? I'm really confused on this and when testing both (With Basic Auth as I'm yet to successfully configure Jwt :P ) :

Caduceus /hook and /hooks always returns 404, /api/v4/notify seems to work tho. I've also attempted to add /api/v* to hook(s) and still returns 404. Even when I booted Caduceus locally no breakpoint is triggered when calling /hook Tr1d1um wouldn't allow to register with until and duration are both absent , any clue how can I configure this? I would assume it would be under webhook.request but I've found no configuration for it.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

pedrojreis commented 2 years ago

It seems that as of now scytale does not have the /hook endpoint, right? At least as far as i saw in the repo.

I'm linking svalinn to tr1d1um and attempted to add duration: "5m" under webhook.request in the config map but the output is

{"error":"received non-200 response: 400, body: {\"message\":\"failed webhook validation: duration value of webhook is out of bounds: 1347804h7m0.647174144s not between 0 and 0s\"}\n","level":"error","msg":"Failed to register webhook","ts":"2022-07-19T09:55:04.092790321Z"}

Any clue on what i'm doing wrong? 🙂

schmidtw commented 2 years ago

You're right. When I searched for scytale usage of ancla, it wasn't there. So we'll need to add that at some point. Sorry to waste time on this.

This is what our configuration looks like:

webhook: registrationInterval: "3m" timeout: "1m" registrationURL: "https://tr1d1um.example.com:443/api/v3/hooks" request: config: url: "https://svalinn.example.com" secret: "--removed--" maxRetryCount: 3 events: ["device-status.*"] duration: "5m"

I think the only part you're missing is the duration in the request config.

On Tue, Jul 19, 2022 at 3:34 AM Pedro Reis @.***> wrote:

It seems that as of now scytale does not have the /hook endpoint, right? At least as far as i saw in the repo.

I'm linking svalinn to tr1d1um and attempted to add duration: "5m" under webhook.request in the config map but the output is

{"error":"received non-200 response: 400, body: {\"message\":\"failed webhook validation: duration value of webhook is out of bounds: 1347804h7m0.647174144s not between 0 and 0s\"}\n","level":"error","msg":"Failed to register webhook","ts":"2022-07-19T09:55:04.092790321Z"}

Any clue on what i'm doing wrong? 🙂

— Reply to this email directly, view it on GitHub https://github.com/xmidt-org/svalinn/issues/158#issuecomment-1188888725, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFKTSK265ZEQGHGRHS4UGLVU2ABRANCNFSM53L6E3JA . You are receiving this because you were mentioned.Message ID: @.***>

pedrojreis commented 2 years ago

No worries 😄

This is my Svalinn Config, i've added the duration under the request as you can see and i believe is correct, however I get the message saying that the duration value of webhook is out of bounds that I quoted in the previous message, ence my confusion on all this.

########################################
#   Webhook Registration Related Configuration
########################################

# webhook provides the information needed to register to a webhook.  If the 
# urls and event regex aren't empty and the interval is greater than 0, 
# registration occurs.
# (Optional)
webhook:
  # registrationInterval provides the time to wait between each registration to 
  # the webhook.  If this is set to 0, no registration is done.
  # (Optional)
  registrationInterval: "4m"

  # timeout provides the length of time the request should wait before timing 
  # out.
  timeout: "1m"

  # registrationURL provides the place to register the webhook.
  registrationURL: "http://tr1d1um.namespace:443/api/v3/hook"

  # request provides the information passed in the webhook registration request.
  request:
    # config provides configuration values for the requests to the webhook 
    # receiver.
    config:
      # url provides the server the webhook should send the events to.  If this 
      # value isn't set, no registration happens.
      url: "http://svalinn.namespace:7100/api/v1/device-status"

      # contentType provides the content type Svalinn expects from the requests 
      # it receives.
      # (Optional) defaults to "wrp"
      contentType: "wrp"

      # secret provides the key for hashing the messages the webhook sends. 
      # If this is empty, no authentication is done later for incoming requests 
      # to Svalinn.
      # (Optional)
      # secret: "super secret"

      # maxRetryCount is the number of times to retry on a failure.
      # (Optional)
      maxRetryCount: 3

      # alternativeURLs provides endpoints to round robin through on a failure 
      # to send an event.  This is only used if the maxRetryCount is greater 
      # than 0.
      # (Optional)
      #AlternativeURLs: []

    # events provides a list of regular expressions that tells the webhook 
    # which endpoints to send to Svalinn.  If the destination of an event 
    # matches a regular expression in this list, it is sent to Svalinn
    events: [".*"]

    # matcher provides regular expressions to match against the event source.
    # (Optional) default is [".*"]
    # matcher:
    #   deviceID: [".*"]

    duration: "5m"
schmidtw commented 2 years ago

Ok, that sounds like a new issue, so the addition of duration made a difference. Change the duration to something like: "1m" or 60 to see if one of those works. I think the v3 endpoint accepts go's time.Duration value, but it may accept seconds as an integer instead.

On Wed, Jul 20, 2022 at 1:51 AM Pedro Reis @.***> wrote:

No worries 😄

This is my Svalinn Config, i've added the duration under the request as you can see and i believe is correct, however I get the message saying that the duration value of webhook is out of bounds that I quoted in the previous message, ence my confusion on all this.

########################################

Webhook Registration Related Configuration

########################################

webhook provides the information needed to register to a webhook. If the

urls and event regex aren't empty and the interval is greater than 0,

registration occurs.

(Optional)

webhook:

registrationInterval provides the time to wait between each registration to

the webhook. If this is set to 0, no registration is done.

(Optional)

registrationInterval: "4m"

timeout provides the length of time the request should wait before timing

out.

timeout: "1m"

registrationURL provides the place to register the webhook.

registrationURL: "http://tr1d1um.namespace:443/api/v3/hook"

request provides the information passed in the webhook registration request.

request:

# config provides configuration values for the requests to the webhook

# receiver.

config:

  # url provides the server the webhook should send the events to.  If this

  # value isn't set, no registration happens.

  url: "http://svalinn.namespace:7100/api/v1/device-status"

  # contentType provides the content type Svalinn expects from the requests

  # it receives.

  # (Optional) defaults to "wrp"

  contentType: "wrp"

  # secret provides the key for hashing the messages the webhook sends.

  # If this is empty, no authentication is done later for incoming requests

  # to Svalinn.

  # (Optional)

  # secret: "super secret"

  # maxRetryCount is the number of times to retry on a failure.

  # (Optional)

  maxRetryCount: 3

  # alternativeURLs provides endpoints to round robin through on a failure

  # to send an event.  This is only used if the maxRetryCount is greater

  # than 0.

  # (Optional)

  #AlternativeURLs: []

# events provides a list of regular expressions that tells the webhook

# which endpoints to send to Svalinn.  If the destination of an event

# matches a regular expression in this list, it is sent to Svalinn

events: [".*"]

# matcher provides regular expressions to match against the event source.

# (Optional) default is [".*"]

# matcher:

#   deviceID: [".*"]

duration: "5m"

You're right. When I searched for scytale usage of ancla, it wasn't there. So we'll need to add that at some point. Sorry to waste time on this. This is what our configuration looks like: webhook: registrationInterval: "3m" timeout: "1m" registrationURL: " https://tr1d1um.example.com:443/api/v3/hooks" request: config: url: " https://svalinn.example.com" secret: "--removed--" maxRetryCount: 3 events: ["device-status.

*"] duration: "5m" I think the only part you're missing is the duration in the request config. … <#m-6322026640324564538> On Tue, Jul 19, 2022 at 3:34 AM Pedro Reis @.> wrote: It seems that as of now scytale does not have the /hook endpoint, right? At least as far as i saw in the repo. I'm linking svalinn to tr1d1um and attempted to add duration: "5m" under webhook.request in the config map but the output is {"error":"received non-200 response: 400, body: {"message":"failed webhook validation: duration value of webhook is out of bounds: 1347804h7m0.647174144s not between 0 and 0s"}\n","level":"error","msg":"Failed to register webhook","ts":"2022-07-19T09:55:04.092790321Z"} Any clue on what i'm doing wrong? 🙂 — Reply to this email directly, view it on GitHub <#158 (comment) https://github.com/xmidt-org/svalinn/issues/158#issuecomment-1188888725>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFKTSK265ZEQGHGRHS4UGLVU2ABRANCNFSM53L6E3JA https://github.com/notifications/unsubscribe-auth/AAFKTSK265ZEQGHGRHS4UGLVU2ABRANCNFSM53L6E3JA . You are receiving this because you were mentioned.Message ID: @.>

— Reply to this email directly, view it on GitHub https://github.com/xmidt-org/svalinn/issues/158#issuecomment-1190006430, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFKTSIABG7MJAKKLT46N2LVU642VANCNFSM53L6E3JA . You are receiving this because you were mentioned.Message ID: @.***>

pedrojreis commented 2 years ago

"1m" value returns 1294379h56m18.871345152s. 60 value returns 1m0s

The weirdest part if that tr1d1um is apparently forcing the value to be between 0 and 0s. I have tr1d1um 0.7.4 - I'll do another look onto tr1d1um.yaml but I don't recall that I have any configuration commented for times, durations and whatnot.

I have traced the error and it lead me to ancla, traced back to tr1d1um and I believe the setting that should be responsible for the max time of the webhook duration should be tr1d1um's webhook.WebhookValidationConfig.max, however upon closer inspection it seems that WebhookValidationConfig has been removed and now ttl is under webhook.

Added: Ok, it seems that i'm now past that time issue. However if tr1d1um's ttl.max is set for 600 a max os 600ns is returned, i've set it to "5m" and it seems to accept the value now :)

Thank you @schmidtw for all the help 😃