Use paws to authenticate a httr2 request?

[hadley]

Is it possible to use paws to authenticate a request that I'm making with httr2? I want to perform a request to the bedrock runtime ConverseStream operation using the (very new) httr2::req_perform_connection().

(Related to #839)

[DyfanJones]

Is it possible to use paws to authenticate a request that I'm making with httr2?

In it's current state I don't think so 🤔. I believe the request to httr::VERB would have to be expose to translate it over to a httr2 request instead.

https://github.com/paws-r/paws/blob/5a37466b9ef25cc312310069fba89a9b9441fb1b/paws.common/R/net.R#L128-L136

I have been trying to think in how to handle streaming data and was pondering if a new class would be the best approach 🤔

Currently paws is using httr. I guess to fully utilise streaming functionality paws should update to httr2 and take advantage of all the benefits it offers. I am going on holiday for the next 3 weeks but when I get back I will try an experimental branch to migrate paws to httr2.

Sorry this isn't a full answer to your question.

[hadley]

For the project I need it for, I might just bite the bullet and implement the AWS SigV4 signing protocol myself (I'm talking to a bunch of other LLMs with pure httr2 calls). I don't know how it would be to extract that logic out of paws into an exported function, but that would certainly make life easier for me.

[DyfanJones]

When I am back from holiday I am happy to expose paws's AWS SigV4 (it was on my todo list). That should make it simpler for you.

From my knowledge you might need to convert some of the raw response into int8, int16, int32, int64 and uint8, uint16, uint32, uint64. To extract some key information before parse the message back

https://github.com/boto/botocore/blob/8e2e8fd7ab59f8c1337902acc32d2ee10cb184ad/botocore/eventstream.py

[DyfanJones]

I have been playing around with some ideas in how to do this in R. The pkd package looks like a useful however it isn't on cran.

I have managed to implement a method in R:

big_endian <- function(vec, dtype) {
  switch(
    dtype,
    "int64" = c(
      vec[8:1], vec[16:9], vec[24:17], vec[32:25], vec[40:33], vec[48:41], vec[56:49], vec[64:57]
    ),
    "int32" = c(vec[8:1], vec[16:9], vec[24:17], vec[32:25]),
    "int16" = c(vec[8:1], vec[16:9]),
    "int8" = vec[8:1]
  )
}

int_to_uint <- function (x, adjustment=2^32) {
  if (sign(x) < 0) {
    return(x + adjustment)
  }
  return(x)
}

# Convert raw vector into integers with big-endian
int64 <- function(x) {
  bits <- as.integer(big_endian(rawToBits(x), "int64"))
  sum(bits[-1] * 2^(62:0)) - bits[[1]] * 2^63
}

int32 <- function(x) {
  bits <- as.integer(big_endian(rawToBits(x), "int32"))
  sum(bits[-1] * 2^(30:0)) - bits[[1]] * 2^31
}

int16 <- function(x) {
  bits <- as.integer(big_endian(rawToBits(x), "int16"))
  sum(bits[-1] * 2^(14:0)) - bits[[1]] * 2^15
}

int8 <- function(x) {
  bits <- as.integer(big_endian(rawToBits(x), "int8"))
  sum(bits[-1] * 2^(6:0)) - bits[[1]] * 2^7
}

# Converts raw vector into unsigned integers with big-endian
uint64 <- function(x) {
  int_to_uint(int64(x), 2^64)
}

uint32 <- function(x) {
  int_to_uint(readBin(x, "integer", n=length(x), size = 4, endian = "big"))
}

uint16 <- function(x) {
  readBin(x, "integer", n=length(x), size=2, signed = F, endian = "big")
}

uint8 <- function(x) {
  readBin(x, "integer", n=length(x), size=1, signed = F, endian = "big")
}

obj <- openssl::rand_bytes(8)

uint8(obj[1])
#> [1] 228
uint16(obj[1:2])
#> [1] 58508
uint32(obj[1:4])
#> [1] 3834393554
uint64(obj)
#> [1] 1.646859e+19

int8(obj[1])
#> [1] -28
int16(obj[1:2])
#> [1] -7028
int32(obj[1:4])
#> [1] -460573742
int64(obj)
#> [1] -1.978149e+18

pkd::uint8(obj[1])
#> <pkd_uint8[1]>
#> [1] 228
pkd::uint16(obj[1:2], endian = 0)
#> <pkd_uint16[1]>
#> [1] 58508
pkd::uint32(obj[1:4], endian = 0)
#> <pkd_uint32[1]>
#> [1] 3834393554
pkd::uint64(obj, endian = 0)
#> <pkd_uint64[1]>
#> [1] 1.646859e+19

pkd::int8(obj[1])
#> <pkd_int8[1]>
#> [1] -28
pkd::int16(obj[1:2], endian = 0)
#> <pkd_int16[1]>
#> [1] -7028
pkd::int32(obj[1:4], endian = 0)
#> <pkd_int32[1]>
#> [1] -460573742
pkd::int64(obj, endian = 0)
#> <pkd_int64[1]>
#> [1] -1.978149e+18

^{Created on 2024-10-09 with reprex v2.1.1}

[hadley]

I was hoping it would use server-sent events like every other API 😭

[DyfanJones]

AWS can be a bit of a pain at times :) If you managed to get a working prototype I would be really interested as it should help with the implementation in paws. :)

[hadley]

@jcheng5 discovered that curl actually has a native implementation: https://curl.se/libcurl/c/CURLOPT_AWS_SIGV4.html. So auth, at least, will be easier than expected.

[hadley]

Some docs for the protocol at https://docs.aws.amazon.com/transcribe/latest/dg/streaming-setting-up.html#streaming-event-stream