Closed s26 closed 7 years ago
Is your MySQL server running?
Yes. My MySql server is running. I am able to start the app successfully, but not all the time. I get this error randomly. 8 out of 10 times I get above error while restarting the app. It is annoying, Sometimes I have to try for 10-15 times to start the app. Everytime I am getting above error.
@s26 That is the worse case because it should be 100% of the time. It'll make it much harder to reproduce if it's intermittent like that. Can you try scaffolding a new basic LoopBack application to see if it occurs there? I'm thinking it's something else.
I have this to lately.
Mostly I have it when I'm starting my application in the morning the first time (when a lot of other processes are starting too), or when I try to start it with the debugger (which is kind a heavy) .
I think it occurs when my vagrant is under heavy load. Is there any way to set a timeout option in my loopback datasource config to see if increasing the timeout helps?
@ernie58 I'm not sure that it solves your problem, but a connectTimeout
connection option value can be passed to the underlying mysqljs/mysql module through your Loopback \server\datasources.json configuration file.
connectTimeout: The milliseconds before a timeout occurs during the initial connection to the MySQL server. (Default: 10000)
https://github.com/mysqljs/mysql#connection-options
You would add the option to datasources.json something like this for a 30 second timeout:
{
"my_mysql_db": {
"port": 3306,
"url": "localhost",
"database": "my_mysql_db",
"password": "myPassword",
"name": "my_mysql_db",
"user": "my_node_mysql_user",
"connector": "mysql",
"connectTimeout": 30000
}
}
Would you like to submit a PR? You can see the options we pass in at https://github.com/strongloop/loopback-workspace/blob/master/available-connectors.json#L280
@superkhau If the above comment is addressed to me, I'm confused...
connectTimeout
value will pass through to mysqljs/mysqlConnection options beyond those listed in the JSON you linked to _will_ pass through to mysql. I have been using dateStrings
for some time and just tested connectTimeout
.
@bbito I thought we wanted to set a default timeout permanently to make it more explicit (ie. the underlying driver's 10000 you mentioned). Anyways, it's all good and thanks for helping out. You are right though, any options you set in datasources.json will be passed down to the driver -- that should solve @s26's issue.
@s26 Can you confirm whether @bbito's solution works for you? Can I close this?
I have the same issue. It happens much often if I try to debug app with WebStorm. Without debugging it happens in ~5%, with debugging it happends in ~70% for me.
@probil does extending the timeout reduce the appearance of the errors?
@bbito Yeah, it seems like this works. But sometimes I got another error:
Connection fails: Error: Handshake inactivity timeout
It will be retried for the next request.
events.js:160
throw er; // Unhandled 'error' event
^
Error: Handshake inactivity timeout
at Handshake.<anonymous> (/media/multimedia/Code/node-js/test/node_modules/mysql/lib/protocol/Protocol.js:160:17)
at emitNone (events.js:86:13)
at Handshake.emit (events.js:185:7)
at Handshake._onTimeout (/media/multimedia/Code/node-js/test/node_modules/mysql/lib/protocol/sequences/Sequence.js:126:8)
at ontimeout (timers.js:365:14)
at tryOnTimeout (timers.js:237:5)
at Timer.listOnTimeout (timers.js:207:5)
--------------------
at Protocol._enqueue (/media/multimedia/Code/node-js/test/node_modules/mysql/lib/protocol/Protocol.js:141:48)
at Protocol.handshake (/media/multimedia/Code/node-js/test/node_modules/mysql/lib/protocol/Protocol.js:52:41)
at PoolConnection.connect (/media/multimedia/Code/node-js/test/node_modules/mysql/lib/Connection.js:136:18)
at Pool.getConnection (/media/multimedia/Code/node-js/test/node_modules/mysql/lib/Pool.js:48:16)
at MySQL.connect (/media/multimedia/Code/node-js/test/node_modules/loopback-connector-mysql/lib/mysql.js:89:17)
at Object.initializeDataSource [as initialize] (/media/multimedia/Code/node-js/test/node_modules/loopback-connector-mysql/lib/mysql.js:43:28)
at DataSource.setup (/media/multimedia/Code/node-js/test/node_modules/loopback-datasource-juggler/lib/datasource.js:339:19)
at new DataSource (/media/multimedia/Code/node-js/test/node_modules/loopback-datasource-juggler/lib/datasource.js:117:8)
at Registry.createDataSource (/media/multimedia/Code/node-js/test/node_modules/loopback/lib/registry.js:355:12)
at dataSourcesFromConfig (/media/multimedia/Code/node-js/test/node_modules/loopback/lib/application.js:440:19)
at EventEmitter.app.dataSource (/media/multimedia/Code/node-js/test/node_modules/loopback/lib/application.js:235:14)
at /media/multimedia/Code/node-js/test/node_modules/loopback-boot/lib/executor.js:190:9
at /media/multimedia/Code/node-js/test/node_modules/loopback-boot/lib/executor.js:281:5
at Array.forEach (native)
at forEachKeyedObject (/media/multimedia/Code/node-js/testt/node_modules/loopback-boot/lib/executor.js:280:20)
at setupDataSources (/media/multimedia/Code/node-js/test/node_modules/loopback-boot/lib/executor.js:180:3)
Hi @probil - thanks, that is useful information. I assume that your CPU is stressed since enabling debugging is having a significant effect. Is this a dev or production environment? Perhaps @superkhau is familiar with the Handshake inactivity error you reach with the increased connectTimeout
.
It's dev
environment. Sometimes I got the error even without debugging enabled
I thinks it should be possible to pass acquireTimeout
param
@bbito It seems like no issue anymore with:
{
"mySQL":{
//...
"connectTimeout": 20000,
"acquireTimeout": 20000
}
in datasources.json
@superkhau I have uninstasll strongloop and reinstall it. And cloned the project again and it is working fine. Thanks
FWIW I work with webstorm debugger to. I Did have the connect timeout and handshake activity timeout. Setting the 2 variables in the config helped me out
Had the same issue running Loopback + PM2 + MariaDB on Docker. Setting larger timeouts has fixed - thanks :)
Just to note if anyone else has problems - I had to put this timeout directly on datasources.json
. Putting it on one of the env
namespaced ones doesn't seem to work i.e datasources.production.js
Hello,
I am getting following error randomly when I run try to restart my web. Not able to figure out how can I fix it. Can someone please assist me to resolve this error?
C:\Users\User\Desktop\20-7-2017\webservice_cron\webservice_cron\webservice\app.js:140 if (err) throw err; ^
at Protocol._enqueue (C:\Users\User\Desktop\20-7-2017\webservice_cron\webservice_cron\webservice\node_modules\mysql\lib\protocol\Protocol.js:141:48)
at Protocol.handshake (C:\Users\User\Desktop\20-7-2017\webservice_cron\webservice_cron\webservice\node_modules\mysql\lib\protocol\Protocol.js:52:41)
at Connection.connect (C:\Users\User\Desktop\20-7-2017\webservice_cron\webservice_cron\webservice\node_modules\mysql\lib\Connection.js:136:18)
at getOpenTarget (C:\Users\User\Desktop\20-7-2017\webservice_cron\webservice_cron\webservice\app.js:112:5)
at Query._callback (C:\Users\User\Desktop\20-7-2017\webservice_cron\webservice_cron\webservice\app.js:89:13)
at Query.Sequence.end (C:\Users\User\Desktop\20-7-2017\webservice_cron\webservice_cron\webservice\node_modules\mysql\lib\protocol\sequences\Sequence.js:85:24)
at Query._handleFinalResultPacket (C:\Users\User\Desktop\20-7-2017\webservice_cron\webservice_cron\webservice\node_modules\mysql\lib\protocol\sequences\Query.js:144:8)
at Query.EofPacket (C:\Users\User\Desktop\20-7-2017\webservice_cron\webservice_cron\webservice\node_modules\mysql\lib\protocol\sequences\Query.js:128:8)
at Protocol._parsePacket (C:\Users\User\Desktop\20-7-2017\webservice_cron\webservice_cron\webservice\node_modules\mysql\lib\protocol\Protocol.js:280:23)
at Parser.write (C:\Users\User\Desktop\20-7-2017\webservice_cron\webservice_cron\webservice\node_modules\mysql\lib\protocol\Parser.js:74:12)
Still have this issue after YEARS.
I have literally tried everything. It just won't connect randomly where all the other app have no problems.
Thank you so much @probil, I got this problem today as well.
Loopback just started failing when trying to connect to the database. It was completely random. Increasing the timeouts, as suggested, fixed the problem.
I am getting the same issue. The solution doesn't work for me.
Our solution was to remove the initial database connection from the loopback boot by enabling lazyConnect in the datasource config, and then to separately call the datasource connection after the boot is finished. We came to a similar conclusion that CPU stress during the loopback boot causes the db connection to fail.
We used the connectTimeout
and acquireTimeout
workaround for a while but started getting this error again. Increasing them further didn't work.
Since we're using Kubernetes, we were able to increase the CPU limit (spec.containers[].resources.limits.cpu
) and that fixed the issue.
But I'm very interested in the solution proposed by @strideynet . Do you have any code snippets you can share?
Thanks!
Interesting, it looks like lazyConnect
is an undocumented feature of the MySQL connector: https://github.com/strongloop/loopback-connector-mysql/blob/7ce66b5b819669114fc828c2dbef5b16c4ff8fd8/lib/mysql.js#L39
I imagine it works similarly as described in the MongoDB documentation:
When set to true, the database instance will not be attached to the datasource and the connection is deferred. It will try to establish the connection automatically once users hit the endpoint. If the MongoDB server is offline, the app will start, however, the endpoints will not work.
(https://loopback.io/doc/en/lb3/MongoDB-connector.html#additional-properties)
This issue seems to have some steps on how to use it:
- Specify "lazyConnect": true in datasources.json for the given datasource.
- Programmatically connect to this datasource after boot using:
app.datasources.MyDatasource.connect(...);
(https://github.com/strongloop/loopback-datasource-juggler/issues/1592)
I am getting this error while trying to connect with the MySQL database, It was already hosted on the server.
Hello,
I am getting following error randomly when I run try to restart my app. Not able to figure out how can I fix it. Can someone please assist me to resolve this error?