NoSuchElementException when calling MatcherKState.toGeoJSON()

[adng]

Sometimes I get the following exception when exporting the k-state to GeoJSON: java.util.NoSuchElementException at java.util.HashMap$HashIterator.nextNode(HashMap.java:1439) at java.util.HashMap$KeyIterator.next(HashMap.java:1461) at com.bmwcarit.barefoot.markov.KState.sequence(KState.java:300) at com.bmwcarit.barefoot.matcher.MatcherKState.toGeoJSON(MatcherKState.java:75)

It seems to me that sometimes, states could contain 0 candidate. And there is no exception handling for this case.

How could this happens? And how can we prevent or process this exception when it arrives?

[smattheis]

That seems like a bug because empty vectors in the k-state are prohibited. I can't identify the bug at a first glance. How exactly do you use it? - To make debugging easier for me: Do you use parameters k or t with non-zero values? Do you export it to JSON format and import again? Can you provide somehow an example to reproduce the error?

[adng]

Here is our matcher's workflow: MatcherKState state = new MatcherKState();

for each sample in a trace

  Set<MatcherCandidate> vector = execute(state.vector(), state.sample(), sample);

  state.update(vector, sample);

JSONObject stateGeoJSON = state.toGeoJSON();

So we set the parameters k and t to their default values. And we just export the state to GeoJSON format once when the processing is done.

Here is the json of the state exactly when the geojson export fails. {"sequence":[{"vector":[{"candid":"806f1ac6-9118-4e24-8168-a76260e6f81e","predid":""}],"sample":{"time":1513344663000,"id":"612","point":"POINT (2.32283935882367 48.8712888071769)"}}, .... // VECTOR IS EMPTY {"vector":[],"sample":{"time":1513365638000,"id":"1909","point":"POINT (2.7624680008765 49.6559869498477)"}}, {"vector":[],"sample":{"time":1513365639000,"id":"1910","point":"POINT (2.76280093007047 49.6566617349625)"}}, .... {"vector":[],"sample":{"time":1513365778000,"id":"2047","point":"POINT (2.79032592661939 49.7525311355011)"}}, // VECTOR IS NO LONGER EMPTY {"vector":[{"candid":"5d89da2e-adc2-4066-be55-4cfc392828c2","predid":""}],"sample":{"time":1513365778000,"id":"2048","point":"POINT (2.79061669484041 49.753236388834)"}}, ...
{"candidate":{"seqprob":-3943.406958873673,"filtprob":1,"id":"817f5951-d015-47f1-aade-3774067bb767","transition":{"route":{"roads":[{"road":158277421,"heading":"forward"}],"source":{"road":158277421,"heading":"forward","frac":0.12037988199350792},"target":{"road":158277421,"heading":"forward","frac":0.15334785376597618}}},"point":{"road":158277421,"heading":"forward","frac":0.15334785376597618}},"count":1}],"t":-1,"k":-1}

We can see that sometimes, the vector in the state sequence could be empty. By looking at our logs, I can see the following pattern: MATCHING OK MATCHING OK ... MATCHING OK Filter: HMM break - no state transitions MATCHING OK ... MATCHING OK Filter: HMM break - no state transitions MATCHING OK ...

The HMM breaks correspond exactly to the samples with which the empty vectors start and end.

So I doubt that when the HMM breaks happened, somehow in the state update phase, all the existing candidates in the vectors were removed.

Hope that this will helps you to debug. Otherwise, let me know the informations you need. If there is still no clue, I'll try to fabricate a trace with which you can reproduce the error.

[smattheis]

Thanks, I think I could identify the bug from your description. I will check out a possible patch which you could test before I push it to the master branch. That means I will be back soon with a little patch. In the meantime, thank you so much for finding, reporting and describing this issue!

[smattheis]

Could you please try this patch: kstate.txt? To apply the patch do the following:

cd barefoot
git apply path/to/kstate.txt
mvn package -DskipTests

This should fix it for you. Later I will push it together with a unit test specific to your bug on to the master branch. Thanks again!

[adng]

The bug is gone! Thank you very much! Looking forward to the new version of barefoot.

[smattheis]

This is now pushed to the master branch and included in release 0.1.2.