Method A - Non continuous frames - files are not saved

[anna-braun]

The simulation with JPScore breaks because of a time limit. The analysis with JPSreport seems to run without any problems (it does not crash). The output dictionary has been created, but there are no files saved in it. The log file shows, that there is a problem with a non continuous ID.

I guess for Method A there has to be a warning in the log file, but the results should be saved anyway, because it does not effect the results too much, if there is missing one single person in one frame.

The files can be found here: https://fz-juelich.sciebo.de/s/9LOZgOQxbGrZmaR

[anna-braun]

@JetteSchumann Hi, are there any updates here?

[JetteSchumann]

@anna-braun
I'm working on it. You will get an update tomorrow ;)

[JetteSchumann]

The actual problem is that that the combined traj-file contains position information that is not sorted by time. E.g. data for frame 5656 is read in before data for frame 3717. This leads to indexing issues which I am trying to solve at the moment.

[gjaeger]

@JetteSchumann and @anna-braun You can use python with pandas and sort your trajectory file.

Like:

import pandas as pd

df = pd.read_csv('12-combine.txt',
                 comment='#', 
                 sep='\t',
                 names=['ID','FR','X','Y','Z','A','B','ANGLE','COLOR'],
                 index_col = False)

# sort dataframe by frame and time
df = df.sort_values(by=['FR','ID'])

f = open('traj_12-combine.txt', 'w')
f.write('# description: 12-combine\n# framerate: 8\n\n# ID\t\FR\tX\tY\tZ\tA\tB\tANGLE\tCOLOR\n')

df.to_csv(f, sep='\t', index=False, header=False, float_format='%.4f', columns=('ID','FR','X','Y','Z','A','B','ANGLE','COLOR') )

f.close()

[gjaeger]

my new files:

• ini_13-report-ini-RE.xml
• traj_12-combine.txt

but with

----
JuPedSim - JPSreport

Current date   : Thu Apr 02 13:29:37 2020
Version        : 0.8.4
Compiler       : clang++ (11.0.0)
Commit hash    : v0.8.4-928-g49c2c1d4-dirty
Commit date    : Thu Feb 27 14:28:35 2020
Branch         : master
INFO:   Parsing the ini file <ini_13-report-ini-RE.xml>
...

... I get a segmentation fault

• log_new.txt

... strange

[anna-braun]

The actual problem is that that the combined traj-file contains position information that is not sorted by time. E.g. data for frame 5656 is read in before data for frame 3717. This leads to indexing issues which I am trying to solve at the moment.

Okay, this is crazy, because for most of my simulations this works fine, it is only one case, which seems to mix the data and with it the frames

[gjaeger]

Okay, this is crazy, because for most of my simulations this works fine, it is only one case, which seems to mix the data.

Is this a problem of jpsreport?

[anna-braun]

@JetteSchumann and @anna-braun You can use python with pandas and sort your trajectory file.

Like:

import pandas as pd

df = pd.read_csv('12-combine.txt',
                 comment='#', 
                 sep='\t',
                 names=['ID','FR','X','Y','Z','A','B','ANGLE','COLOR'],
                 index_col = False)

# sort dataframe by frame and time
df = df.sort_values(by=['FR','ID'])

f = open('traj_12-combine.txt', 'w')
f.write('# description: 12-combine\n# framerate: 8\n\n# ID\tFR\tX\tY\tZ\tA\tB\tANGLE\tCOLOR\n')

df.to_csv(f, sep='\t', index=False, header=False, float_format='%.4f', columns=('ID','FR','X','Y','Z','A','B','ANGLE','COLOR') )

f.close()

Okay, I will check if it works for me

[JetteSchumann]

@anna-braun could you upload more trajectory files? If the data for the last frame (maximum) frame are at the end of those files, it should work correctly.

[JetteSchumann]

@gjaeger I think it is a jpsreport problem. I mean it depends on how we define a valid trajectory file. I think jpsreport should be able to deal with unsorted frame information

[gjaeger]

@JetteSchumann I admit that my question could be understood as a rhetorical question.

@anna-braun Is the trajectory file 12-combine.txt created directly by jpscore and not edited further?

[anna-braun]

@anna-braun could you upload more trajectory files? If the data for the last frame (maximum) frame are at the end of those files, it should work correctly.

Here are 4 more examples. If you need more, please tell me. https://fz-juelich.sciebo.de/s/EXglchlGSzpRKt7

[gjaeger]

I've no problem with the latest commit 4226836b:

% jpsreport_debug ini_13-report-ini-RE.xml
----
JuPedSim - JPSreport

Current date   : Thu Apr 02 13:55:19 2020
Version        : 0.8.4
Compiler       : clang++ (11.0.3)
Commit hash    : v0.8.4-940-g4226836b-dirty
Commit date    : Tue Mar 31 15:56:55 2020
Branch         : master
INFO:   Parsing the ini file <ini_13-report-ini-RE.xml>
INFO:   logfile </Users/gjaeger/Documents/Simulationen/JPSreport_issue_719/log_new.txt>
lineNr 100000
...
lineNr 6200000
INFO:   Success with Method A using measurement area id 3
INFO:   Success with Method A using measurement area id 103
INFO:   Success with Method A using measurement area id 2301
INFO:   Success with Method A using measurement area id 2302
INFO:   Success with Method A using measurement area id 2303
INFO:   Success with Method A using measurement area id 2304
INFO:   Success with Method A using measurement area id 2305
INFO:   Success with Method A using measurement area id 2306
INFO:   Success with Method A using measurement area id 2307
INFO:   Success with Method A using measurement area id 2308
INFO:   Success with Method A using measurement area id 2309
INFO:   Success with Method A using measurement area id 2310
INFO:   Success with Method A using measurement area id 2311
INFO:   Success with Method A using measurement area id 2312
INFO:   Success with Method A using measurement area id 2313
INFO:   Success with Method A using measurement area id 2314
INFO:   Success with Method A using measurement area id 2315
INFO:   Success with Method A using measurement area id 2316
INFO:   Success with Method A using measurement area id 4
INFO:   Success with Method A using measurement area id 104
INFO:   Success with Method A using measurement area id 2401
INFO:   Success with Method A using measurement area id 2402
INFO:   Success with Method A using measurement area id 2403
INFO:   Success with Method A using measurement area id 2404
INFO:   Success with Method A using measurement area id 2405
INFO:   Success with Method A using measurement area id 2406
INFO:   Success with Method A using measurement area id 2407
INFO:   Success with Method A using measurement area id 2408
INFO:   Success with Method A using measurement area id 2409
INFO:   Success with Method A using measurement area id 2410
INFO:   Success with Method A using measurement area id 2411
INFO:   Success with Method A using measurement area id 2412
INFO:   Success with Method A using measurement area id 2413
INFO:   Success with Method A using measurement area id 2414
INFO:   Success with Method A using measurement area id 2415
INFO:   Success with Method A using measurement area id 2416
INFO:   End Analysis for the file: traj_12-combine.txt
Time elapsed:    791.755 [s]

[anna-braun]

It works after sorting the trajectory file!

[gjaeger]

@anna-braun Can we close this issue?

[anna-braun]

I'm not sure. What is with the point of missing frames?

[gjaeger]

@anna-braun @JetteSchumann When I remove the lines for frame 4, I get the following message in the logfile:

INFO: Leave CreateGlobalVariables()
INFO: Create Global Variables done
Error:  The trajectory of ped with ID <1> is not continuous. Please modify the trajectory file!
Error:   actual_totalfame = <4804>, expected_totalframe = <4805> 
ERROR:  Could not parse the file traj_13-report-ini-ICE_m.txt

So, jpsreport requires continuous trajectories. We should add a note in the documentation.

[zeroset]

@JetteSchumann Can you please add this to the Documentation and close the issue afterwards? Thanks.

[gjaeger]

One small addition: As @JetteSchumann already reported, this message is generated when the trajectories are read in. So all methods are affected.

[JetteSchumann]

Issue hase been fixed with PR #735 and #733