block / goose

Goose is a developer agent that operates from your command line to help you do the boring stuff.
https://block.github.io/goose/
Apache License 2.0
109 stars 17 forks source link

test: convert Google Gemini tests to VCR #118

Closed codefromthecrypt closed 2 weeks ago

codefromthecrypt commented 2 weeks ago

part of https://github.com/square/exchange/issues/67 closes https://github.com/square/exchange/pull/71

codefromthecrypt commented 2 weeks ago

@baxen I'll move each pending PR from exchange to here one-by-one, here's the first!

lamchau commented 2 weeks ago

:nit: could we format the json so it's easier to read?

        string: |-
          {
            "candidates": [
              {
                "content": { ... }
               }
codefromthecrypt commented 2 weeks ago

@lamchau

:nit: could we format the json so it's easier to read?

technically yes, it will require some engineering to write a custom serializer as vcr has no options built-in. I'm not opposed to this, but it also will affect the openai data already here, and could be done as a separate task. Is it ok if we decouple this? You can assign that issue to me which I can try on a weekend or a flight.

lamchau commented 2 weeks ago

@codefromthecrypt sure! just a nit - i just like to know what's being passed in

just for learning: is the custom serializer required to dump the request/response to yaml? never used it before so i was naively thinking using a yq transform for post-processing - the serializer sounds much more robust (and involved)

edit: made issue #127, can't seem to assign it - thanks for taking a look!

codefromthecrypt commented 1 week ago

@lamchau

just for learning: is the custom serializer required to dump the request/response to yaml? never used it before so i was naively thinking using a yq transform for post-processing - the serializer sounds much more robust (and involved)

So, I didn't consider yq (like something we could do in just format), it is possible that doing that wouldn't taint the input expectations vcrpy and look like request drift. That could be the easier way out vs a custom serializer, which is the current suggestion from the project. https://github.com/kevin1024/vcrpy/issues/580