adobe / helix-run-query

service that executes queries on BigQuery datasets generated by Helix-Logging

Apache License 2.0

6 stars 11 forks source link

feat(sendquery): Enable cost logging for all jobs #1150

Closed lydiapuric closed 3 months ago

lydiapuric commented 3 months ago

Please ensure your pull request adheres to the following guidelines:

[x] make sure to link the related issues in this description
[x] when merging / squashing, make sure the fixed issue references are visible in the commits, for easy compilation of release notes

Related Issues

1143 [Feature] Log costs of a query to Coralogix

Refactoring done:

Unified creation of job and stream
Code that consumes the stream
Always redact authorization headers in Polly recordings.

Thanks for reviewing!

lydiapuric commented 3 months ago

@trieloff the 2 index tests are failing because Polly never recorded the HTTP request to fetch the metadata. I haven't committed the recordings for those requests because Polly records, in addition to the request parameters, also the token in the Authorization header.

I added the recording, but I also added some additional code to configure Polly to always redact the authorization header, so we won't have this problem in the future.

github-actions[bot] commented 3 months ago

This PR will trigger a minor release when merged.

trieloff commented 3 months ago

:tada: This PR is included in version 3.31.0 :tada:

The release is available on:

v3.31.0
GitHub release
v3.31.0

Your semantic-release bot :package::rocket: