googleapis / google-cloud-go

Google Cloud Client Libraries for Go.
https://cloud.google.com/go/docs/reference
Apache License 2.0
3.67k stars 1.25k forks source link

logging/logadmin - no way to set PageSize or PageToken when getting entries #10083

Closed pkpfr closed 2 months ago

pkpfr commented 3 months ago

It is currently not possible to set the PageSize or PageToken when using the admin client to get log entries.

Methods should be added to set EntryOptions for these two values, or if they can be set in the filter, there should be clear documentation on how to do so. The Iterator also has no method to set pagination.

Currently, the package is not fit for purpose. The timestamp filters only allow second precision. Logs could be missed or duplicated if using a timestamp for pagination purposes, and excessive may be transmitted due to not being able to restrict the page size.

gkevinzheng commented 2 months ago

@pkpfr What method requiring EntriesOptions are you using? It looks like there is a way to set PageSizehttps://github.com/googleapis/google-cloud-go/blob/0183dd57dbc12b307ca185da33158dedebe1acf2/logging/logadmin/logadmin.go#L225-L230

gkevinzheng commented 2 months ago

@pkpfr Also to address your concern about the timestamp filters only allowing second precision, the Cloud Logging query language does allow you to enter timestamps with nanosecond accuracy: https://cloud.google.com/logging/docs/view/logging-query-language

In Go, you can format a Time object with RFC3339Nano to include nanoseconds into the passed in timestamp instead of RFC3339.

pkpfr commented 2 months ago

Thanks - my issue seems to be that this code hasn't been released in the current publicly available versions. cloud.google.com/go/logging v1.9.0.

gkevinzheng commented 2 months ago

@pkpfr I've cut a release for logging: https://github.com/googleapis/google-cloud-go/releases/tag/logging/v1.10.0 that has the pageSize changes in them. Hope this resolves your issue.

pkpfr commented 2 months ago

Unfortunately, setting the pageSize seems to have no effect. No matter what the value is set to, it is ignored.