backoff to snowflake connection establishment

pivnicek commented 7 years ago

rel: https://github.com/keboola/connection/issues/576

Halama commented 7 years ago

@pivnicek there were proble in tests in master. I've fixed it and rebased this PR.

pivnicek commented 7 years ago

@Halama I saw that, thank you!

Halama commented 7 years ago

@pivnicek can you please check these issues?

pivnicek commented 7 years ago

yes, sorry, mmt.

Halama commented 7 years ago

there are still CS issue in build...

pivnicek commented 7 years ago

gah, I really should check the build outputs!

Halama commented 7 years ago

This is that message with translation:

Nov 14 08:46:40 kbc-vpc-sapi-WORKER-i-908dbb09 Zend_Log:  {
message: Queue process error
priority: WARN
app: keboola-connection
pid: 8520
cliCommand: ./scripts/cli.php --ansi storage:queue:receive main_fast 2
module: storage
messageId: 523805a3-f0f6-4dc3-aafe-d7c9bd298015
newMessageId: 666ccc91-d9d4-4896-9da7-c61de52fc165
retryCount: 1
delaySecs: 120
jobId: 18995608
queueMessage: {
id: 523805a3-f0f6-4dc3-aafe-d7c9bd298015
body: {
jobId: 18995608
timestamp: 2016-11-14T08:46:39+01:00
retryCount: 1
}
}
exception: {
message: Initializing Snowflake connection failed: odbc_connect(): SQL error: [unixODBC][Snowflake][Snowflake] (4) 
      REST request for URL https://keboola.snowflakecomputing.com:443/session/v1/login-request?databaseName="sapi_258"&schemaName=&warehouse="KEBOOLA_PROD"&requestId=e6823fd9-5e9b-4c74-a8a5-513b999bc6c3 failed: Unknown error (http error).
    , SQL state S1000 in SQLConnect
code: 0
attachment: https://connection.keboola.com/admin/utils/logs?file=2016/11/14/08/2016-11-14-08-46-39-58296bdfe48cb-exception
}
component: JobExecutor
}

pivnicek commented 7 years ago

should probably retry that too eh

Halama commented 7 years ago

I think we can check that S1000 it is present in both versions.

Halama commented 7 years ago

These logs without translated message are from older worker with ODBC drivers ODBC 2.12.86 54 the second one newer where the messages are translated has ODBC 2.12.87.

I'll shut down the older one.

pivnicek commented 7 years ago

ok. even if that S1000 is a generic error, worst thing that happens is it tries to reconnect 5 times.

Halama commented 7 years ago

yes. it is ok. I have tried some common errors like bad login and it is returning other codes so it should be ok.

Halama commented 7 years ago

:)

ODBC 2.12.89    
SNOW-22102: Fixed a potential deadlock when the main application thread waiting for a result chunk, being downloaded by an asynchronous thread, times out.
SNOW-21795, SNOW-24366, SNOW-24589, SNOW-24519: Improved handling of connection failures and re-establishing a connection.

keboola / php-db-import

backoff to snowflake connection establishment #19