OBD data not uploading

[zkvvoob]

Hello,

Today I found your Torque server and absolutely loved the idea. So, I installed in on my webserver (Ubuntu 14.04, Apache/2), configured it according to the instructions. The web interface had some issues with a missing Google Maps API key, but I got one for myself and added it to the session.php file, so that was that.

However, even though Torque on my Android phone reported "No problems" when testing the server, I couldn't get it to upload any of the 19 trips it said it had. The app kept trying to upload (when I clicked the back button to exit), I waited patiently, but in the end no information appeared in the web interface.

To test whether it was a phone issue, I switched back to the default Torque URL (in.torque-bhp.com) and was able to see my trips in their viewer online app.

The cred.php file did initially contain a torque ID, but then I removed it. There are web credentials set up, but I hope they are not an issue.

Would you be able to help me figure out where the problem is? I would really like to keep using it, instead of the official server.

If you need any additional information, let me know.

[surfrock66]

Ok, a bunch of possibilities:

1) App isn't correctly sending to the right place 2) App is sending it to the web server, web server is rejecting it 3) Web server is accepting it, bug is preventing writes to DB 4) Web server is trying to write to DB, DB write is failing 5) DB write is working, displaying stored data is failing.

1 and 5 are gonna be easiest to start with.

1) Look in your apache access logs, look for the web requests. Default path is /var/log/apache2/access.log You should see the COMPLETE upload string, and the result (hopefully 200). You can actually copy that string to your browser and try to load it, you should get an "OK!" if the log is successful. 5) Do you know any mysql? If so, can you do a select * in the logs table?

[zkvvoob]

Hi surfrock66, thanks for taking the time to answer.

I'd like to test all these, but I haven't been able to figure out how to force Torque to manually upload the logs to the webserver. So far, my experience has been - go to the car, start, connect to OBD, try to close the app, agree to upload the logs, wait. Which is a bit inconvenient, not to mention slow. Is there any way to do it on demand?

[surfrock66]

Not really, and I've asked the author of the torque app to work on that. Basically, the logs upload live, and the app will keep trying until it gets the "OK!" signal, or when the app closes. There's no way to re-upload old tracks, even though they are stored on the phone...I had planned to do a csv import in my fork, but I haven't gotten to it yet.

If you have data in the access logs, you can kind of recreate the upload through a web browser. The data is a "GET" request, so your log looks like this:

172.58.33.226 - - [19/Sep/2016:07:35:38 -0700] "GET /torquetest/upload_data.php?eml=surfrock66@surfrock66.com&v=8&session=1474293354033&id=8a10dcdc03a8b509d9ff1dc42b855a4e&time=1474295737909&kff1005=-121.47819648&kff1006=38.56753245&kff1001=19.333206&kff1007=109.6&kff1264=408.706&kff1265=239.01&k47=12.156863&kff1223=0.0037858798&kff1220=-0.84155273&kff1221=0.45996094&kff1222=0.195166&k49=8.627451&k4a=4.3137255&kff124d=13.79&k46=22.0&kff1263=41.0148&kff1272=37.758884&kff1270=1012.1994&k33=101.0&k3c=724.0&k3d=724.0&k44=1.0&kff1258=535.8124&kff1257=459.82465&kff126a=-10798.156&k31=35261.0&k21=0.0&k5=97.0&k4=28.62745&k43=16.470589&kc=653.0&k32=-429.0&kff125c=2.7479768&kff125d=2.955407&kff125a=49.256783&k2f=61.960785&kff126b=-1680.1243&k7=-3.125&k14=-2.34375&k6=0.0&k9=-3.90625&k18=-2.34375&k8=-3.125&kff1271=4.5227065&kff1239=12.0&kff1010=-12.0&kff123b=109.6&kff1006=38.56753245&kff1005=-121.47819648&kff123a=9.0&kff1237=9.333206&kf=38.0&kb=35.0&kff1203=5.752069&kff5202=8.489015&kff1207=17.384687&kff5203=11.779685&kff1201=16.248783&kff5201=23.980268&kff1214=0.82&kff1215=0.645&kff1218=0.84&kff1219=0.72499996&k45=1.9607843&k1f=2393.0&kff1001=19.333206&kd=10.0&k11=14.509804&ke=26.5&kff1206=6.784217&kff1208=14.739789&kff1205=19.164454&kff1204=24.381277&kff120c=16333.649&kff1266=2361.0&kff1268=2161.782&kff1267=199.61&kff1202=-9.572491&k42=14.221&kff1238=13.0&kff126d=0.11198516&kff126e=0.18142615 HTTP/1.0" 200 248 "-" "-"

You can pull the URL from that and plug it in the web browser.

One thing that burned me...there's a maximum URL size, and php bombed out. I had to up that in my php config to accept a string that long. If you're logging a ton of PID's, that could be it.

I should also say, I'm using a fork of this repo that has diverged greatly including a totally different DB schema, so while my advice is general, I probably am not the best to help on specifics.

[zkvvoob]

Hi surfrock66,

Just went through the Apache access.log and found the request to upload_data.php, put it in the browser and got an OK! response. However, there are still no sessions in Open Torque Viewer and yesterday, when I finished driving and tried closing the Torque app, it kept trying to upload the data, but to no avail.

The GET request actually ends with HTTP/1.0" 500 618 "-" "-", don't know if that's relevant.

What would you suggest I try now?

Thanks!

EDIT: OK, so I was digging through the Apache error.log to find out what's throwing the Error 500 up there, and came across a bunch of these:

[Wed Sep 21 11:30:54.438829 2016] [rewrite:trace3] [pid 26194] mod_rewrite.c(475): [client 79.*.*.*:15563] 79.*.*.* - - [domain.com/sid#b5c4e058][rid#b2308058/initial] [perdir /var/www/domain.com/web/] strip per-dir prefix: /var/www/domain.com/web/t/t/upload_data.php -> t/t/upload_data.php

which is weird. The Open Torque viewer is located at https://www.domain.com/t, but why is an additional /t/ being added? As far as I can see, the parent director's .htaccess doesn't have anything wrong: http://pastebin.com/MjtZcLQc

Any ideas?

[surfrock66]

In the torque app, what's the url string you're using for the server, and do you get test success when asking it to test the server settings? I suck at rewrite rules, but everything there looks ok to me. Are there any other nested .htaccess files in directories?

[zkvvoob]

The URL in Torque is set as https://www.domain.com/t/upload_data.php. There are .htaccess files in some of the other subdirectories, but I don't see how they should apply to the /t directory where the Torque Viewer is.

[surfrock66]

Weird. It definitely looks like a rewrite rule, but I don't know enough about .htaccess to really deep dive...if you temporarily remove the .htaccess do you get different results?

[zkvvoob]

So, I'm still struggling with the rewrite issues, but in the meantime I've moved the torque viewer to a different server. The access.log now reports GET 200, there are entries in the raw_logs MySQL table

but no sessions appear in the Viewer:

What could I be doing wrong now?

EDIT: Truth be told, I just checked the MySQL table on the original server, and it too contains data (probably as a result of my manual input in the browser). However, the Viewer shows no sessions. Am I missing something?

[surfrock66]

No...though at this point we're exiting the area I can be helpful. I forked this project and redid the way sessions present, and the DB is no longer compatible...it's been too long since working on this version of the code to remember how to fix it.

The issue is, the only table here is raw_logs, so to populate the list of sessions, it does a query on unique session entries. If you're like me and have a ton of logs, it took FOREVER. I split out a sessions table where you could index the sessions with 1 row per session. Also, all of the metadata about the fields was stored as a comment in the raw table...I made a keys table. Because of this, there's now 3 tables and the session.php calls all 3 when populating.

One of the BIGGEST causes of lag was the fact that it didn't just pull the most current session, it tried to pull the next one as well in getting ready to populate the Merge feature. Because of this, I think it fails if you don't have at least 2 sessions in your DB.

What I would do is open up session.php and echo out a whole bunch of variables at various points in the code to see what is populated where...from this, it seems like trouble in the get_sessions.php file, but that feeds info from session.php so it could be before it even gets there.

[zkvvoob]

I appreciate all your help, surfrock66, and your time. It looks like I'll have to take it from here. Thank you!