No matching points visible in barefoot monitor

[florianpinzel]

Hi Sebastian,

first of all thanks a lot for your continuous commitment to the project. I was very pleased to read you're looking for other contributors to help you maintaining and improving the project beyond the current state. I got the matcher server running, tested it with own input data and the results are really promising so far. But I still have a couple of issues and questions regarding the project though.

Issue: Monitor Not Showing States Whenever I start the tracker server unfortunately I can't see any output on the barefoot monitor besides the map. This happens with an "all default"-setup and the provided sample-data as well as with own map- and input-data. I'm receiving "SUCCESS" responses using the provided stream.py-utility, so I guess it's not the TrackerServer itself (which also doesn't give any errors or warnings) but rather the monitor backend or its interface. So I find the situation, that the monitor interface displays the map correctly, but does not show any "states" (green/red points).

Isolating the source of failure Until now I'm not able to properly build with Maven without skiping the tests with -DskipTests. Both mvn package (-e -X -U) and mvn (clean) install (-e -X -U) used to have led to following error message:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project barefoot: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?

described in Issue #38. The solution provided there and similarly in Issue #42 unfortunately seem to be working only partly.

Docker status:

pinzel@VINENT077-VB:~/Cartox/barefoot$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                     NAMES
2e469f85f45f        barefoot-map        "/bin/sh -c 'servi..."   20 hours ago        Up 20 hours         0.0.0.0:32768->5432/tcp   barefoot

According to your solution I have set database.port=32768 in oberbayern.properties. Do I have to set the port somewhere else as well, e.g in tracker.properties or something?

Because now I receive the following error message:

Tests run: 76, Failures: 2, Errors: 5, Skipped: 2
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:43 min
[INFO] Finished at: 2018-01-24T16:32:36+01:00
[INFO] Final Memory: 16M/60M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project barefoot: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/pinzel/Cartox/barefoot/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project barefoot: There are test failures.

So I assume Maven is not able to build the project successfully because some surefire test failed which subsequently may lead to the original issue outlined above. My best guess so far is that the root of all evil here is the particular surefire test error message:

2018-01-24 16:32:09,746 ERROR [Thread-12] AbstractServer: opening server socket failed - Die Adresse wird bereits verwendet (Bind failed)
Exception in thread "Thread-12" java.lang.RuntimeException
    at com.bmwcarit.barefoot.util.AbstractServer.runServer(AbstractServer.java:270)
    at com.bmwcarit.barefoot.tracker.TrackerControl.runServer(TrackerControl.java:101)
    at com.bmwcarit.barefoot.tracker.TrackerServerTest$Server.run(TrackerServerTest.java:50)
    at java.lang.Thread.run(Thread.java:748)

This error message is occurring - among some other errors, e.g. Executing PostgreSQL query failed: ERROR: parse error - invalid geometry - three times during the process in slight variations but always while testing "AbstractServer".

So I'm curious what the function of AbstractServer is and of course if you could you think of any solution for fixing that issue? Do you reckon it's the unsuccessful build caused by that "binding error" message that leads to the core problem with the monitor not shwoing states or are these two issues indepent from each other? Alternative: Is there a way to get the monitor working using a successful build by using the -DskipTests-Option?

FYI: I work inside a ubuntu VM but the exact same error message occurs at a collegues desktop running a native ubuntu OS. I'm happy to provide further information like the surefire reports or whatever that lead to a proper solution. Your help is greatly appreciated =)

[smattheis]

Thanks for feedback and the detailed description. It seems there are mutliple issues of your current settings which we have to check out. Could you please post the output of this command on your VM java -XshowSettings:locale -version? Then let's see what to do next.

[florianpinzel]

Thanks for the quick reply and here you go:

user@VB:~/project/barefoot$ java -XshowSettings:locale -version
Locale settings:
    default locale = Deutsch
    default display locale = Deutsch (Deutschland)
    default format locale = Deutsch (Deutschland)
    available locales = , ar, ar_AE, ar_BH, ar_DZ, ar_EG, ar_IQ, ar_JO, 
        ar_KW, ar_LB, ar_LY, ar_MA, ar_OM, ar_QA, ar_SA, ar_SD, 
        ar_SY, ar_TN, ar_YE, be, be_BY, bg, bg_BG, ca, 
        ca_ES, cs, cs_CZ, da, da_DK, de, de_AT, de_CH, 
        de_DE, de_GR, de_LU, el, el_CY, el_GR, en, en_AU, 
        en_CA, en_GB, en_IE, en_IN, en_MT, en_NZ, en_PH, en_SG, 
        en_US, en_ZA, es, es_AR, es_BO, es_CL, es_CO, es_CR, 
        es_CU, es_DO, es_EC, es_ES, es_GT, es_HN, es_MX, es_NI, 
        es_PA, es_PE, es_PR, es_PY, es_SV, es_US, es_UY, es_VE, 
        et, et_EE, fi, fi_FI, fr, fr_BE, fr_CA, fr_CH, 
        fr_FR, fr_LU, ga, ga_IE, hi, hi_IN, hr, hr_HR, 
        hu, hu_HU, in, in_ID, is, is_IS, it, it_CH, 
        it_IT, iw, iw_IL, ja, ja_JP, ja_JP_JP_#u-ca-japanese, ko, ko_KR, 
        lt, lt_LT, lv, lv_LV, mk, mk_MK, ms, ms_MY, 
        mt, mt_MT, nl, nl_BE, nl_NL, no, no_NO, no_NO_NY, 
        pl, pl_PL, pt, pt_BR, pt_PT, ro, ro_RO, ru, 
        ru_RU, sk, sk_SK, sl, sl_SI, sq, sq_AL, sr, 
        sr_BA, sr_BA_#Latn, sr_CS, sr_ME, sr_ME_#Latn, sr_RS, sr_RS_#Latn, sr__#Latn, 
        sv, sv_SE, th, th_TH, th_TH_TH_#u-nu-thai, tr, tr_TR, uk, 
        uk_UA, vi, vi_VN, zh, zh_CN, zh_HK, zh_SG, zh_TW

openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

[smattheis]

Alright, try this patch locale.txt as follows:

cd barefoot
git apply path/to/locale.txt
mvn clean test

I think this fixes the issue for running the tests. If so, try running the tracker with

java -Duser.language=en -Duser.country=US -jar ...

instead of only java -jar .... If it's not working after that, please post full log outputs of maven or the tracker, respectively ... whatever causes the problem.

Acknowledgement: This issue is known thanks to @argltuc as reported with PR https://github.com/bmwcarit/barefoot/pull/51 and which we should merge soon. (I apologize for not having merged it yet!)

Edit: Had a little error in the patch, but it's updated now.

[florianpinzel]

Thanks, unfortunately that doesn't seem to help. The patch is applied but the error messages when running mvn clean test command are pretty much the same as before.

Full log here: mvn_test.txt

[smattheis]

It looks like you're using a different map than the oberbayern map because the tracker can't find any suitable state emissions. Have you set up the oberbayern map for the test? Is it up and running?

[florianpinzel]

My bad, sorry! So running the mvn clean test command with the correct map data 1.) takes for some reason very long for execution. Restarted VM and Host, test still takes 10-30(!) minutes and 2.) eventually outputs following error log: mvn_test2.txt

Error messages seem to be somewhat different, but I wouldn't even have a clue what's wrong here. This error message might be of interest:

2018-01-26 15:12:16,271 ERROR [Thread-24] AbstractServer: 127.0.0.1:44736 response handler timeout
2018-01-26 15:12:16,271 ERROR [pool-3-thread-4] Group: group com.bmwcarit.barefoot.scheduler.Group@461ac1a4 sync interrupted, gets cancelled

FYI: I'm back to the default option using database port 5432, but that seems to be ok.

pinzel@VINENT077-VB:~/Cartox/barefoot$ docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
71a4dfd1fc68        barefoot-map        "/bin/sh -c 'servi..."   2 hours ago         Up 25 minutes       0.0.0.0:5432->5432/tcp   barefoot-oberbayern

[smattheis]

Try setting server.timeout.response=10000 in config/tracker.properties.

[florianpinzel]

Ok, will do that in a minute. I just realized that I can't even apply the patch anymore:

User@VB:~/Project/barefoot$ git apply locale.txt
error: Anwendung des Patches fehlgeschlagen: pom.xml:14
error: pom.xml: Patch konnte nicht angewendet werden

with pom.xml

...
9               <groupId>org.apache.maven.plugins</groupId>
10              <artifactId>maven-compiler-plugin</artifactId>
11              <version>3.1</version>
12              <configuration>
13                  <source>1.7</source>
14                  <target>1.7</target>
15              </configuration>
...

I try to fix that beforehand...

[smattheis]

It would be anyway interesting to see if the patch changes something for you, so it's not the worst idea to first try without the patch. Therefore, you could first reset the pom.xml:

git checkout HEAD -- pom.xml

give it a try with mvn clean test and then apply the patch:

git apply path/to/locale.txt

[florianpinzel]

I increased all timeout levels: Matcher, Tracker, request, response.

1.) After executing git checkout HEAD -- pom.xml and mvn clean test the error log mvn_test3.txt is the same as at the very beginning, which at least shows that the patch does something.

2.) After executing git apply locale.txt and mvn clean test the log differs as mentioned above and now looks like this: mvn_test4.txt

As I mentioned above executing mvn clean test in 2.) now takes unusually long at some unit tests so I just provided the beginning of the log. Will answer you asap if the command terminates eventually succesfully or if it gets stuck.

Thanks for your patience =)

[smattheis]

Okay, that means the issue about locale settings is actually verified. That's good to know. The execution time for matching single samples is not only high but has also a large variance in your output. (The first part of the test trace is in the city centre of Munich and, hence, is computationally intense.) That may be to some degree due to the VM ... Anyways, the initial issue should be resolved, i.e., "monitor not showing states". Is that right?

[florianpinzel]

To be honest that would be a bit surprising for me. Can't reach my laptop in the moment so just guessing, but I somehow doubt that a build will be successful without skipping the tests.

I will give it a shot though to build with -DskipTests option (and also without before) and the java -Duser.language=en -Duser.country=US -jar ... command and check if that actually helped to fix the initial problem.

Come back to you soon. Have a nice weekend =)

[florianpinzel]

Hooray! 🎉 I actually works and I can display the position and the trace.

Concluding: You find the solution that worked for me here: https://github.com/bmwcarit/barefoot/issues/86#issuecomment-360559726 In my case I had to use mvn package -DskipTests for a successful build.

Thanks a million for your help Sebastian! Me and the entire community really appreciate it 😄