fetching volume timeout

chuan137 commented 1 month ago

A note for the community

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
If you are interested in working on this issue or have submitted a pull request, please let us know in a comment

Problem

I see timeout errors with feching volumes sometimes.

error="failed to fetch data: error making request StatusCode: 200, Error: context deadline exceeded (Client.Timeout or context cancellation while reading body), API: /api/private/cli/volume?fields=snaplock_type%2Csize_available_for_snapshots%2Csnapshot_reserve_size%2Ccompression_space_saved%2Cexpected_available%2Cdedupe_space_saved%2Csnapshot_count%2Cnodes%2Clogical_used%2Coverwrite_reserve_used%2Csis_space_saved%2Csnapshot_reserve_available%2Cused%2Cfilesystem_size%2Cphysical_used%2Cavailable%2Cstate%2Clogical_used_by_afs%2Coverwrite_reserve%2Csize%2Csize_used_by_snapshots%2Cis-space-enforcement-logical%2Cdedupe_space_saved_percent%2Cfiles_used%2Ctype%2Csis_space_saved_percent%2Cvserver%2Cclone_parent_vserver%2Cclone_volume%2Cis_encrypted%2Cfiles%2Clogical_used_percent%2Cpercent_snapshot_space%2Csnapshot_space_used%2Ctotal%2Cis_sis_volume%2Cautosize_grow_threshold_percent%2Cpercent_used%2Cvolume%2Cclone_parent_name%2Csnapshot_policy%2Ccomment%2Clogical_used_by_snapshots%2Caggr_list%2Ccompression_space_saved_percent%2Cmax_autosize%2Cis_space_reporting_logical%2Cuuid%2Cvsroot%2Cphysical_used_percent%2Cjunction_path%2Cvolume_style_extended&is_constituent=%2A&return_records=true" Poller=stnpca1-st090 collector=Rest:Volume task=data

what is the default timeout time? is it configurable? Can I configure it to fetch in pages?

Configuration

No response

Poller

Volume

Version

harvest version 23.11.0-1 (commit 166b0263) (build date 2023-11-13T14:57:09+0000) linux/amd64

Poller logs

No response

OS and platform

Debian GNU/Linux 11 (bullseye)

ONTAP or StorageGRID version

NetApp Release 9.14.1P2: Tue Mar 19 23:28:08 UTC 2024

Additional Context

No response

References

No response

rahulguptajss commented 1 month ago

@chuan137 This is a client side timeout controlled by the client_timeout parameter in the template for the relevant collector. The default value is 30 seconds. You can adjust this timeout within Rest volume template. For more details, please refer to the documentation.

https://netapp.github.io/harvest/latest/help/troubleshooting/#workarounds https://netapp.github.io/harvest/latest/configure-rest/#collector-configuration-file https://netapp.github.io/harvest/latest/configure-templates/#how-to-extend-a-restrestperfstoragegridems-collectors-existing-object-template

chuan137 commented 1 month ago

Thank you, I will try it out

rahulguptajss commented 2 weeks ago

@chuan137 Any updates on this?

cgrinds commented 2 weeks ago

Closing. Feel free to reopen if you still see this issue

NetApp / harvest