ChuckBell / MySQL_Connector_Arduino

Database connector library for using MySQL with your Arduino projects.
332 stars 133 forks source link

Stuck on conn.connect() #78

Closed Gabe-Soares closed 5 years ago

Gabe-Soares commented 5 years ago

Hey! I'm working on a arduino UNO project with a Ethernet shield w5100. My project should be able to connect to a online database and return some coordinates under some where clauses (the day and time the code is running, solving the bug I'm dealing with right now, I'm probably going for TimeStamp or something like that). So, I'm using strcat() to concatenate the strings and generate the Select query. It's also good to tell that both datas I'm selecting are doubles, and the fields I'm comparing in the where clauses are date and time type (I don't have idea if the bugs there). The weird thing is that I've made a LOT of tests with slightly different codes to try to catch the bug, but i couldn't, and the results on the tests somehow did vary. (Most of my code runs on the setup function) Some of the tests simply stopped on the following line: if (conn.connect(server_ip, 3306, user, password)) Other tests I don't know why run the code until this line, and then begin a looping, restarting the setup till this if, and doing it again, and again.... Some codes I did write on the begining of the project (with less code after the connection, but the same code before it) are still working, so I'm suspecting of memory overload maybe [?]. That's it... I don't have any idea what's causing this, but it can connect to the internet, but not to the database server. Thanks for the atention! I hope you can help me solve this, I'm stuck on this for too long!

I'll post some of my code soon.

Gabe-Soares commented 5 years ago

include

include

include

include

include

include "calcDist.h"

include

include

byte mac_address[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED }; byte ip_address[] = { 192, 168, 0, 20 }; byte dns_address[] = { 192, 168, 0, 1 }; byte gateway_address[] = { 192, 168, 0, 1 }; byte netmask[] = { 255, 255, 255, 0 };

IPAddress server_ip(, , , ); //Did hide the actual database server IP. EthernetClient client; MySQL_Connection conn((Client *)&client);

char user[] = "", password[] = "", use_bd[] = "USE bd_ic"; //Same to the user and password. bool aux_conn = false;

void setup() { coord c_atual; c_atual.lat = -23.515884; c_atual.lon = -47.756303; Serial.begin(57600); while(!Serial); Serial.print("Serial initialized.\n"); Ethernet.begin(mac_address, ip_address, dns_address, gateway_address, netmask); delay(3000); Serial.print("Connecting to internet.\n"); Serial.println(Ethernet.localIP());

//This is where the code stops. Sometimes it runs on a looping from the begining of setup() till this line, sometimes it just stops.

if (conn.connect(server_ip, 3306, user, password)){

  //This code isn't executed. 
  Serial.print("Connection stablished.\n");

  //This is the query I'm concatenating.
 char select[251] = "SELECT coord_lat_cli,coord_long_cli FROM cliente_tb, atividade_tb WHERE coord_lat_cli!=\"\" AND coord_long_cli!=\"\" AND fk_id_cli=id_cli AND data_ativ='";      //String base do SELECT.
 char data[31], hora_ini[31], hora_fin[10];     //Strings de apoio para a montagem do SELECT.

 Serial.print("Definindo variáveis de data e hora.\n");
 strcpy(data, "20180220' AND hora_ini_ativ<='");
 strcpy(hora_ini, "11:30:00' AND hora_fin_ativ>='");
 strcpy(hora_fin, "12:30:00'");

 Serial.print("Montando SELECT.\n");
 strcat(select, data);
 strcat(select, hora_ini);
 strcat(select, hora_fin);

 //Exibindo select montado.
 Serial.println(select);

  MySQL_Cursor *cursor_mem = new MySQL_Cursor(&conn);
  cursor_mem->execute(use_bd);      //Executa o comando de "USE" do sql.
  delete cursor_mem;                //Deletar o cursor criado, recomendação do desenvolvedor da biblioteca, pelo uso de memória.
  aux_conn = true;

} else{ Serial.println("!- Conexão falhou."); conn.close(); Serial.println("!- Tentativa de conexão finalizada."); }

if(aux_conn){ Serial.println("Executando a consulta."); MySQL_Cursor *cursor_mem = new MySQL_Cursor(&conn); cursor_mem->execute(select); //Executa o comando de "SELECT" do sql.

//There's more code after this, but i don't think it's necessary , so if it's needed, I could copy it later.

} }

I did try to think where could the bug be, i tought about:

Once again, anticipated thanks for the atention!

ChuckBell commented 5 years ago

What version of MySQL are you using? Is the server on the same network and is the connection reliable?

How much memory is the compiler reporting? If you’re seeing < about 800k of data memory, your sketch will be unstable.

ChuckBell commented 5 years ago

Sorry, I meant 800 bytes. :P

Gabe-Soares commented 5 years ago

The version is 5.6.26. I'm not sure if I got your question about being on the same network, but yes, my database is in a server online (in WAN I mean), and yes the connection is reliable considering that some codes does work and i test with my notebook connected as well, so if the connection were unstable or off, i would know. The compiler is reporting this: The sketch's using 20450 bytes (63%) of the program's storage space. The max is 32256 bytes. Global variables uses 1486 bytes (72%) of the dinamic memory, leaving 562 bytes for the local variables. Max is 2048 bytes. (The original report was in portuguese as I'm from Brazil, so I did translate it, sorry if I made any mistakes on it.) The memory left it's 11806 bytes, am I right? So the memory problem probably isn't the cause?

ChuckBell commented 5 years ago

It's the local variables memory that is the concern. In this case, you have 562 bytes available, which is below the threshold for a stable execution.

So, what can you do? First, reduce the number of strings in your code (or use them in program memory - see the connector documentation), reduce the number of variables, and generally trim your code as much as possible.

As for why it hangs on connection, I have a theory, which I have seen a number of times. If you connect to the server a number of times, at some point, the connection stalls and the connector code times out. You can treat this by adjusting the timings in the connector code (but it is not recommended).

However, reducing the size of your local variables is more likely to give you the stability to run the sketch reliably.

One thing I did notice is the use of the "use" command. You do not need that. Simply include the database in your SELECT query instead. The way you've implemented could result in a memory leak.

For example, say you want to query a table named t1 in the testdb database. Instead of "use testdb; select from t1;" sequence, you can simply do "select from testdb.t1;", which will use less memory.

Keep me posted on your progress. Dr. Bell

On 12/20/18 6:15 PM, Gabe-Soares wrote:

The version is 5.6.26. I'm not sure if I got your question about being on the same network, but yes, my database is in a server online (in WAN I mean), and yes the connection is reliable considering that some codes does work and i test with my notebook connected as well, so if the connection were unstable or off, i would know. The compiler is reporting this: The sketch's using 20450 bytes (63%) of the program's storage space. The max is 32256 bytes. Global variables uses 1486 bytes (72%) of the dinamic memory, leaving 562 bytes for the local variables. Max is 2048 bytes.

(The original report was in portuguese as I'm from Brazil, so I did translate it, sorry if I made any mistakes on it.) The memory left it's 11806 bytes, am I right? So the memory problem probably isn't the cause?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ChuckBell/MySQL_Connector_Arduino/issues/78#issuecomment-449168614, or mute the thread https://github.com/notifications/unsubscribe-auth/AH0j4I52-VbqVUsbZ_n9nfK_Ajpqry8eks5u7BongaJpZM4Zc0iN.

Gabe-Soares commented 5 years ago

Thanks for the tips! I will put them on pratice as soon as possible! But I'm not sure if it's the memory, I made some tests and came to this: The EXACTLY same code executing the EXACTLY same way, this select: char select[] = "SELECT coord_lat_cli, coord_long_cli FROM cliente_tb WHERE coord_lat_cli!=\"\""; Works as expected, but with this select: char select[] = "SELECT coord_lat_cli,coord_long_cli FROM cliente_tb, atividade_tb WHERE coord_lat_cli!=\"\" AND coord_long_cli!=\"\" AND fk_id_cli=id_cli AND data_ativ='20180220' AND hora_ini_ativ<='11:30:00' AND hora_fin_ativ>='12:30:00'"; It keeps looping on setup() to conn.connect() as i said. So could it be the quantity of conditions the problem? Or the size of the select string? Does this results gives you any idea of where you would bet the problem is?

Once again, thanks for the GREAT help, Dr. Bell! This is helping me a lot with my reserch for the university.

ChuckBell commented 5 years ago

Your second query is where your code is running out of memory. When an Arduino runs out of memory, it typically reboots, which can manifest as observing the setup() code run repeatedly.

I think you need to reduce the size of the string you are using. There are many ways to do this; using progmem is one, but you can also consider using views. This can permit you to use a shorter string in your code. Regardless, minimizing your code data (strings, variables) is the right strategy.

On Dec 20, 2018, at 7:30 PM, Gabe-Soares notifications@github.com wrote:

Thanks for the tips! I will put them on pratice as soon as possible! But I'm not sure if it's the memory, I made some tests and came to this: The EXACTLY same code executing the EXACTLY same way, this select: char select[] = "SELECT coord_lat_cli, coord_long_cli FROM cliente_tb WHERE coord_lat_cli!="""; Works as expected, but with this select: char select[] = "SELECT coord_lat_cli,coord_long_cli FROM cliente_tb, atividade_tb WHERE coord_lat_cli!="" AND coord_long_cli!="" AND fk_id_cli=id_cli AND data_ativ='20180220' AND hora_ini_ativ<='11:30:00' AND hora_fin_ativ>='12:30:00'";

It keeps looping on setup() to conn.connect() as i said. So could it be the quantity of conditions the problem? Or the size of the select string? Does this results gives you any idea of where you would bet the problem is?

Once again, thanks for the GREAT help, Dr. Bell! This is helping me a lot with my reserch for the university.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ChuckBell/MySQL_Connector_Arduino/issues/78#issuecomment-449187111, or mute the thread https://github.com/notifications/unsubscribe-auth/AH0j4N_Or7cjJ2dWU9mifvkaWTplAV6lks5u7CubgaJpZM4Zc0iN.

Gabe-Soares commented 5 years ago

Thanks!! I'll try this things, and some others too as soon as I can (probably next week), and then I'll post here the results and conclusion!!

Gabe-Soares commented 5 years ago

Hey! Just showing I'm alive hahaha Last 2 weeks I travelled and couldn't use any computer, so I'll run the tests this weekend and post the conclusion after doing it.

chakthy commented 5 years ago

I have a similar issue. I have been trying to use some of the examples to establish a connection with MySQL server. So far the WiFi connection is established, but once it goes to the "conn.connect(server_addr, 3306, user, password)" it seems to hang there. Memory doesn't seem to be an issue: Sketch uses 16612 bytes (33%) of program storage space. Maximum is 49152 bytes. Global variables use 683 bytes (11%) of dynamic memory, leaving 5461 bytes for local variables. Maximum is 6144 bytes.

MySQL server is running on a virtual machine inside my Mac. I have tested the mysql connection between the Mac and the virtual machine (both using different IP addresses). I added code on the Arduino to ping the virtual machine and it gets detected. I have tried another code to connect directly into the server that has the databases and it works, but of course this doesn't connect to MySQL so I tried to use the conn.connect() afterwards but it didn't work either. The connection doesn't fail, and the Arduino remains connected to the network. But the code doesn't go to the next line after the conn.connect(). Not sure if this is related to slight differences on the Arduino WiFi rev.2 Any ideas on how to debug the problem?

Gabe-Soares commented 5 years ago

@chakthy Hey! One question, did you use a Select query? If so, does it have one or more conditions? Could you show us your queries? I did ran a lot of tests, and I don't know why yet, but I'm pretty sure it has something to do with the query.

ChuckBell commented 5 years ago

What version of MySQL are you using?

chakthy commented 5 years ago

I'm able to connect to mysql without issues now after Chuck's recommendations to use only dynamic cursors commenting lines 306-309 in MySQL_Packet.cpp. I tested different mysql versions and the connector works, so at least from my experience that is not the issue. The current problem for me right now is that I can only do 8 Inserts into the table and then I'm unable to connect again until I reset the board.

Gabe-Soares commented 5 years ago

Hey @ChuckBell, sorry for not showing up around here. The thing is that my project had some changes, and now I do have other issues not related to this one, should I close this issue and open another one? Now I'm using ESP32, and it's conflicting with Ethernet.h, but I'm investigating it yet.

Gabe-Soares commented 5 years ago

Also, if you could answer a simple question, please, is there a way to get from MySQL_Cursor *cursor_mem the quantity of rows returned? I want to persist my data returned by the select inside an array, but I'm not able to get the number of rows. Once again, thanks for your attention and help! 😄

ChuckBell commented 5 years ago

Close and open another, please.

On 3/31/19 9:17 AM, Gabe-Soares wrote:

Hey @ChuckBell https://github.com/ChuckBell, sorry for not showing up around here. The thing is that my project had some changes, and now I do have other issues not related to this one, should I close this issue and open another one? Now I'm using ESP32, and it's conflicting with Ethernet, but I'm investigating it yet.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChuckBell/MySQL_Connector_Arduino/issues/78#issuecomment-478341109, or mute the thread https://github.com/notifications/unsubscribe-auth/AH0j4PrytKeXsL5WnixrLQbelATnUOmUks5vcLVvgaJpZM4Zc0iN.

ChuckBell commented 5 years ago

Yes, it is possible. The code currently throws this away, but if you'd like, I can try to find a solution that doesn't alter the existing code. Please open a new issue for this to I can track it.

On 3/31/19 10:58 AM, Gabe-Soares wrote:

Also, if you could answer a simple question, please, is there a way to get from MySQL_Cursor *cursor_mem the quantity of rows returned? I want to persist my data returned by the select inside an array, but I'm not able to get the number of rows. Once again, thanks for your attention and help! 😄

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ChuckBell/MySQL_Connector_Arduino/issues/78#issuecomment-478349132, or mute the thread https://github.com/notifications/unsubscribe-auth/AH0j4KKIa8ttPqO_7ard6dncuFol7xq8ks5vcM0NgaJpZM4Zc0iN.