search

openvstorage / alba

Open vStorage ALBA (alternate backend) creates a replicated or flexible network raid’ed object storage backend out of Seagate Kinetic drives and local disk supporting compression, encryption.
Other
28 stars 10 forks source link

Proxy startup fails to become fully operational #824

Closed toolslive closed 5 years ago

toolslive commented 7 years ago

Launching a proxy:

...
2017-10-09 07:06:47 021021 -0400 - NY1SRV0003 - 29552/0000 - alba/proxy - 456 - info - alba_base_client: refresh namespace info: Client_helper.MasterLookupResult.Error(0)
2017-10-09 07:06:47 021077 -0400 - NY1SRV0003 - 29552/0000 - alba/proxy - 457 - info - 902fa18c-04b9-4d69-85b6-3d73b1b6b157 "Client_helper.MasterLookupResult.Error(0)": should_invalidate:false should_retry:true
2017-10-09 07:06:47 021093 -0400 - NY1SRV0003 - 29552/0000 - alba/proxy - 458 - info - osd_access: osd:0 "902fa18c-04b9-4d69-85b6-3d73b1b6b157" retry (max_retries:2) delay:5.000000
2017-10-09 07:06:47 021127 -0400 - NY1SRV0003 - 29552/0000 - alba/proxy - 459 - info - Exception in alba_id_t: Client_helper.MasterLookupResult.Error(0)
2017-10-09 07:06:48 021183 -0400 - NY1SRV0003 - 29552/0000 - alba/proxy - 460 - info - Exception in alba_id_t: (Failure "attempting to use a resource from a pool which is being finalized")

It never fully recovers from this, and this results in partial failure (failing partial reads)

wimpers commented 6 years ago

@toolslive does restarting the proxy fix the issue? Otherwise as work around we (BE or India OPS) could monitor the log message ´Failure "attempting to use a resource from a pool which is being finalized"´ and restart the proxy accordingly?

JeffreyDevloo commented 5 years ago

No longer seen since 1.5.25-20