department-of-veterans-affairs / connect_vbms

VBMS SOAP API client
Other
17 stars 9 forks source link

Ensure that an error is raised if any VBMS request times out. #155

Closed amprokop closed 7 years ago

amprokop commented 7 years ago

We have a hypothesis from https://github.com/department-of-veterans-affairs/appeals-deployment/issues/768 that VBMS timeouts are never handled properly.

AC

joofsh commented 7 years ago

Scenario 1: Very low open_timeout

connect_vbms gem:

      request.open_timeout               = 0.1 # seconds
      request.read_timeout               = 300 # seconds

Running VBMS request from Caseflow, error is raised:

irb(main):001:0> Appeal.first.documents
  Appeal Load (0.9ms)  SELECT  "appeals".* FROM "appeals"  ORDER BY "appeals"."id" ASC LIMIT 1
[2017-08-18 10:04:49 -0400]   Appeal Load (0.9ms)  SELECT  "appeals".* FROM "appeals"  ORDER BY "appeals"."id" ASC LIMIT 1
STARTED sent VBMS request VBMS::Requests::ListDocuments for 522722137
[2017-08-18 10:04:49 -0400] STARTED sent VBMS request VBMS::Requests::ListDocuments for 522722137
HTTPClient::ConnectTimeoutError: execution expired

Scenario 2: Very low read_timeout

connect_vbms gem:

      request.open_timeout               = 300 # seconds
      request.read_timeout               = 0.1 # seconds

Running VBMS request from Caseflow, error is raised:

irb(main):001:0> Appeal.first.documents
  Appeal Load (0.9ms)  SELECT  "appeals".* FROM "appeals"  ORDER BY "appeals"."id" ASC LIMIT 1
[2017-08-18 10:06:24 -0400]   Appeal Load (0.9ms)  SELECT  "appeals".* FROM "appeals"  ORDER BY "appeals"."id" ASC LIMIT 1
STARTED sent VBMS request VBMS::Requests::ListDocuments for 522722137
[2017-08-18 10:06:24 -0400] STARTED sent VBMS request VBMS::Requests::ListDocuments for 522722137
HTTPClient::ReceiveTimeoutError: execution expired
joofsh commented 7 years ago

So an HTTPClient::ConnectTimeoutError or HTTPClient::ReceiveTimeoutError will definitely be raised and properly bubbled up on the Caseflow side.

amprokop commented 7 years ago

I grepped the Caseflow Sidekiq worker logs for "Timeout" over a 3 day period yesterday and got 0 results. So potentially either VBMS never/very rarely times out or those errors are not thrown when it does.

amprokop commented 7 years ago

Or I made some mistake while searching (best case)

ghost commented 7 years ago

"Oh UAT is just slow and everything is lightning fast and timeouts never happen in production because it's perfect" - Shane, 2017

joofsh commented 7 years ago

Research is complete, looks like we do have timeouts that are working. Closing the issue