Segmentation fault - Githubissues

gottfriedmrs commented 3 years ago

Dear Can,

I meet some issue since I pull the last version of the master branch. Before that some traces were non-matched but the output was only empty. Like this:

Since the last update, these traces make some Segmentation fault. Here is one example, where there is a gap between consecutive points (132 and 131)

When I take all the points, there is one segmentation fault:

When I take one part of this trace (point 0 to 131) the map-matching is working well

FMM parameters: CRS: WGS84 lat/lon network + gps points delta(ubodt) = 0.01 deg = 1km k=8 r=0.002 deg = 200m gps error std = 0.0001 deg = 10m

I am using Windows10 and Cygwin.

cyang-kth commented 3 years ago

Can you share the GPS trace and the network data (by sending to my email) so that I can figure the issue?

gottfriedmrs commented 3 years ago

For sure ! Thanks

cyang-kth commented 3 years ago

It would be better if you could also provide which detailed command fmm or stmatchyou run on the data.

gottfriedmrs commented 3 years ago

I delete some parts for readability

from fmm import Network,NetworkGraph,FastMapMatch,FastMapMatchConfig,UBODT
from fmm import UBODTGenAlgorithm
from fmm import GPSConfig,ResultConfig

network = Network(network_file,"fid","u","v")
print("Nodes {} edges {}".format(network.get_node_count(),network.get_edge_count()))
graph = NetworkGraph(network)

#status = ubodt_gen.generate_ubodt("graph_grenoble_drive_utm31/ubodt_2000.txt", 4, binary=False, use_omp=True)

if os.path.isfile(ubodt_file):
    print("File " + ubodt_file + " exists")
else:
    print("File " + ubodt_file + " creation:")
    ubodt_gen = UBODTGenAlgorithm(network,graph)
    status = ubodt_gen.generate_ubodt(ubodt_file, ubodt_value, binary=False, use_omp=True)
    print(status)

ubodt = UBODT.read_ubodt_csv(ubodt_file)

model = FastMapMatch(network,graph,ubodt)

k = 8
radius = 200 * meter2deg
gps_error = 10 * meter2deg
reverse_tolerance = 0

str_radius = str(int(radius/meter2deg))
str_gps_error = str(int(gps_error/meter2deg))
str_reverse = str(reverse_tolerance)

fmm_config_name = "fmm_" + ubodt_name + "_k" + str(k) + "_r" + str_radius + "_gpserror" + str_gps_error + "_reverse" + str_reverse
outfmm_path = data_path + 'map_matching\\mm_outputs\\fmm\\tmr_bike\\' + fmm_config_name + '\\'

inputs = os.listdir(input_path)
print('List of inputs file to map-match with FMM:')
print(inputs)

fmm_config = FastMapMatchConfig(k,radius,gps_error)#, reverse_tolerance)

for trace in inputs:
    output_file =  outfmm_path + trace
    if not os.path.isfile(output_file):

        # trips_file = seed_path + fgps_name + "trips_noised" + param_noise + ".shp"
        input_file = input_path + trace
        input_config = GPSConfig()
        # input_config.file = trips_file
        input_config.file = input_file

        input_config.id = "id" #id est différent  au sein d'un même csv si plusieurs trajectoires dans le csv
        input_config.x = 'longitude'
        input_config.y = 'latitude'
        input_config.gps_point = True # pour des fichiers csv: True

        # input_config.gps_geom = "geom"
        # input_config.log_level = 3
        print(input_config.to_string())
        result_config = ResultConfig()
        result_config.file = output_file
        # result_config.output_config.write_ogeom = True #original trajectory geometry
        result_config.output_config.write_pgeom = True #a line connecting the matched points
        result_config.output_config.write_error = True #distance from each point to its matched point
        result_config.output_config.write_opath = True #edge matched to each point in trajectory
        result_config.output_config.write_offset = True # distance from the matched point to the start of the matched edge
        result_config.output_config.write_spdist = True #shortest path distances traversed between consecutive points
        result_config.output_config.write_length = True # length of the matched edge for each point

        print(result_config.to_string())

        status = model.match_gps_file(input_config, result_config, fmm_config)
        print(status)
    else:
        print("File " + output_file + " already exists, skipping FMM")

cyang-kth commented 3 years ago

When I run it on the network file you send me, there is no segment fault error.

Status: success
Time takes 0.023 seconds
Total points 553 matched 553
Total trajectories 1 matched 1
Map match percentage 1
Map match speed 24043.5 points/s

gottfriedmrs commented 3 years ago

I try with the command line but I get just this unique line:

gottfriedmrs@GOTTF /cygdrive/c/2_Map_matching
$ fmm --ubodt ubodt.txt --network edges.shp --gps 20160727_175600.csv --gps-point --output out.csv
[info][fmm_app_config.cpp:49 ] Start reading FMM configuration from arguments

Did you map-match with the command line from cygwin or with a python script or ubuntu or other ?

I send you another trajectory that creates the same Segmentation fault. Maybe it comes from my current setup (cygwin+w10) ?

Thanks again for your answers !

cyang-kth commented 3 years ago

I run the previous result from Jupyter-notebook on Mac.

The configuration you type is not correct. On Mac it says

fmm --ubodt ubodt.txt --network edges.shp --gps 20160727_175600.csv --gps-point --output out.csv
[info][fmm_app_config.cpp:49 ] Start reading FMM configuration from arguments
libc++abi.dylib: terminating with uncaught exception of type cxxopts::option_not_exists_exception: Option ‘gps-point’ does not exist
Abort trap: 6

You need to refer to the wiki to type the correct configuration name gps_point.

cyang-kth commented 3 years ago

fmm --ubodt ubodt_2000.txt --network edges.shp --gps 20160727_175600.csv --gps_point --output out.csv --network_id fid --source u --target v --gps_x longitude --gps_y latitude

[info][fmm_app.cpp:80 ] Time takes excluding input 2.835
[info][fmm_app.cpp:82 ] Finish map match total points 395 matched 395
[info][fmm_app.cpp:83 ] Matched percentage: 1
[info][fmm_app.cpp:84 ] Point match speed: 139.036
[info][fmm_app.cpp:86 ] Point match speed (excluding input): 139.33
[info][fmm_app.cpp:87 ] Time takes 2.841

gottfriedmrs commented 3 years ago

Seg fault with r=0.002 (200m)

$ fmm --ubodt ubodt_agglo_grenoble_osm_bike_1km.txt --network edges.shp --gps 20160322_173852.csv --gps_point --output out.csv --network_id fid --source u --target v --gps_x longitude --gps_y latitude -k 8 -r 0.002 -e 0.0001
[info][fmm_app_config.cpp:49 ] Start reading FMM configuration from arguments
[info][fmm_app_config.cpp:78 ] Finish with reading FMM arg configuration
[info][fmm_app_config.cpp:98 ] ----   Print configuration    ----
[info][network_config.cpp:6  ] NetworkConfig
[info][network_config.cpp:7  ] File name: edges.shp
[info][network_config.cpp:8  ] ID name: fid
[info][network_config.cpp:9  ] Source name: u
[info][network_config.cpp:10 ] Target name: v
[info][network_config.cpp:11 ] Mode name: drive
[info][gps_config.cpp:19 ] GPS format: CSV point
[info][gps_config.cpp:20 ] File name: 20160322_173852.csv
[info][gps_config.cpp:21 ] ID name: id
[info][gps_config.cpp:22 ] x name: longitude
[info][gps_config.cpp:23 ] y name: latitude
[info][gps_config.cpp:24 ] Timestamp name: timestamp
[info][result_config.cpp:34 ] ResultConfig
[info][result_config.cpp:35 ] File: out.csv
[info][result_config.cpp:36 ] Fields: cpath mgeom
[info][fmm_algorithm.cpp:25 ] FMMAlgorithmConfig
[info][fmm_algorithm.cpp:26 ] k 8 radius 0.002 gps_error 0.0001
[info][fmm_app_config.cpp:103] Log level 2-info
[info][fmm_app_config.cpp:104] Step 100
[info][fmm_app_config.cpp:105] Use omp false
[info][fmm_app_config.cpp:106] ---- Print configuration done ----
[warning][result_config.cpp:221] Overwrite existing result file out.csv
[info][network.cpp:88 ] Read network from file edges.shp
ERROR 1: PROJ: proj_identify: Cannot find proj.db
[info][network.cpp:179] Number of edges 48291 nodes 20576
[info][network.cpp:180] Field index: id 17 source 14 target 15
[info][network.cpp:183] Read network done.
[info][network_graph.cpp:17 ] Construct graph from network edges start
[info][network_graph.cpp:30 ] Graph nodes 20576 edges 48291
[info][network_graph.cpp:31 ] Construct graph from network edges end
[info][ubodt.cpp:187] Reading UBODT file (CSV format) from ubodt_agglo_grenoble_osm_bike_1km.txt
[info][ubodt.cpp:215] Read rows 1000000
[info][ubodt.cpp:215] Read rows 2000000
[info][ubodt.cpp:215] Read rows 3000000
[info][ubodt.cpp:215] Read rows 4000000
[info][ubodt.cpp:215] Read rows 5000000
[info][ubodt.cpp:222] Finish reading UBODT with rows 5138092
[info][gps_reader.cpp:334] GPS data in point CSV format
[warning][gps_reader.cpp:239] Time stamp timestamp not found, will be estimated
[info][gps_reader.cpp:241] Id index 10 x index 3 y index 2 time index -1
[info][fmm_app.cpp:27 ] Progress report step 100
[info][fmm_app.cpp:29 ] Start to match trajectories
[info][fmm_app.cpp:58 ] Run map matching in single thread
[info][fmm_app.cpp:61 ] Progress 0
Segmentation fault (core dumped)

With log level = 1, the end is:


[debug][ubodt.cpp:84 ] Check a 26832 b 26832
[trace][ubodt.cpp:109] Insert index 22
[debug][ubodt.cpp:84 ] Check a 26832 b 26832
[trace][ubodt.cpp:109] Insert index 22
[debug][ubodt.cpp:84 ] Check a 26832 b 26832
[trace][ubodt.cpp:109] Insert index 22
[debug][ubodt.cpp:84 ] Check a 26832 b 26832
[trace][ubodt.cpp:109] Insert index 22
[debug][ubodt.cpp:84 ] Check a 26832 b 26832
[trace][ubodt.cpp:109] Insert index 22
[debug][ubodt.cpp:84 ] Check a 26832 b 26832
[trace][ubodt.cpp:109] Insert index 22
[debug][ubodt.cpp:84 ] Check a 26832 b 26832
[trace][ubodt.cpp:109] Insert index 22
[debug][ubodt.cpp:84 ] Check a 26832 b 26832
[trace][ubodt.cpp:109] Insert index 22
[debug][ubodt.cpp:84 ] Check a 26832 b 26832
[trace][ubodt.cpp:109] Insert index 22
[debug][ubodt.cpp:84 ] Check a 26832 b 26832
[trace][ubodt.cpp:109] Insert index 22
[debug][ubodt.cpp:84 ] Check a 26832 b 26832
[trace][ubodt.cpp:109] Insert index 22
[debug][ubodt.cpp:84 ] Check a 26832 b 14185
[debug][ubodt.cpp:95 ] Edges ab are adjacent
[trace][ubodt.cpp:106] Insert index 23
[debug][ubodt.cpp:84 ] Check a 14185 b 10597
Segmentation fault (core dumped)

With r=0.0002 (20m) but the map percentage is 0%

$ fmm --ubodt ubodt_agglo_grenoble_osm_bike_1km.txt --network edges.shp --gps 20160322_173852.csv --gps_point --output out.csv --network_id fid --source u --target v --gps_x longitude --gps_y latitude -k 8 -r 0.0002 -e 0.0001
[info][fmm_app_config.cpp:49 ] Start reading FMM configuration from arguments
[info][fmm_app_config.cpp:78 ] Finish with reading FMM arg configuration
[info][fmm_app_config.cpp:98 ] ----   Print configuration    ----
[info][network_config.cpp:6  ] NetworkConfig
[info][network_config.cpp:7  ] File name: edges.shp
[info][network_config.cpp:8  ] ID name: fid
[info][network_config.cpp:9  ] Source name: u
[info][network_config.cpp:10 ] Target name: v
[info][network_config.cpp:11 ] Mode name: drive
[info][gps_config.cpp:19 ] GPS format: CSV point
[info][gps_config.cpp:20 ] File name: 20160322_173852.csv
[info][gps_config.cpp:21 ] ID name: id
[info][gps_config.cpp:22 ] x name: longitude
[info][gps_config.cpp:23 ] y name: latitude
[info][gps_config.cpp:24 ] Timestamp name: timestamp
[info][result_config.cpp:34 ] ResultConfig
[info][result_config.cpp:35 ] File: out.csv
[info][result_config.cpp:36 ] Fields: cpath mgeom
[info][fmm_algorithm.cpp:25 ] FMMAlgorithmConfig
[info][fmm_algorithm.cpp:26 ] k 8 radius 0.0002 gps_error 0.0001
[info][fmm_app_config.cpp:103] Log level 2-info
[info][fmm_app_config.cpp:104] Step 100
[info][fmm_app_config.cpp:105] Use omp false
[info][fmm_app_config.cpp:106] ---- Print configuration done ----
[warning][result_config.cpp:221] Overwrite existing result file out.csv
[info][network.cpp:88 ] Read network from file edges.shp
ERROR 1: PROJ: proj_identify: Cannot find proj.db
[info][network.cpp:179] Number of edges 48291 nodes 20576
[info][network.cpp:180] Field index: id 17 source 14 target 15
[info][network.cpp:183] Read network done.
[info][network_graph.cpp:17 ] Construct graph from network edges start
[info][network_graph.cpp:30 ] Graph nodes 20576 edges 48291
[info][network_graph.cpp:31 ] Construct graph from network edges end
[info][ubodt.cpp:187] Reading UBODT file (CSV format) from ubodt_agglo_grenoble_osm_bike_1km.txt
[info][ubodt.cpp:215] Read rows 1000000
[info][ubodt.cpp:215] Read rows 2000000
[info][ubodt.cpp:215] Read rows 3000000
[info][ubodt.cpp:215] Read rows 4000000
[info][ubodt.cpp:215] Read rows 5000000
[info][ubodt.cpp:222] Finish reading UBODT with rows 5138092
[info][gps_reader.cpp:334] GPS data in point CSV format
[warning][gps_reader.cpp:239] Time stamp timestamp not found, will be estimated
[info][gps_reader.cpp:241] Id index 10 x index 3 y index 2 time index -1
[info][fmm_app.cpp:27 ] Progress report step 100
[info][fmm_app.cpp:29 ] Start to match trajectories
[info][fmm_app.cpp:58 ] Run map matching in single thread
[info][fmm_app.cpp:61 ] Progress 0
[info][fmm_app.cpp:75 ] MM process finished
[info][fmm_app.cpp:81 ] Time takes 0.006
[info][fmm_app.cpp:82 ] Time takes excluding input 0.004
[info][fmm_app.cpp:83 ] Finish map match total points 1321 matched 0
[info][fmm_app.cpp:85 ] Matched percentage: 0
[info][fmm_app.cpp:86 ] Point match speed: 0
[info][fmm_app.cpp:87 ] Point match speed (excluding input): 0
[info][fmm_app.cpp:89 ] Time takes 0.006

cyang-kth commented 3 years ago

@gottfriedmrs

From your log information, I think that you are not using the latest version of the program

Your version is at least before this one

https://github.com/cyang-kth/fmm/blob/6d6fa9c2a1dc73e6061a2e9f8c62971faf8bf7ed/src/mm/fmm/ubodt.cpp#L84-L110

This is the latest version of ubodt.cpp file

https://github.com/cyang-kth/fmm/blob/344fb8c6eb6b4ea3ffbb8607ea781f3cdd8d65a4/src/mm/fmm/ubodt.cpp#L84-L124

Please pull the latest version to check the result.

gottfriedmrs commented 3 years ago

Here after the update

https://pastebin.com/egE3eh21


[debug][ubodt.cpp:89 ] Check point 327 a 14645 b 14645
[debug][ubodt.cpp:110] Edges connecting ab are 43555,31105
[debug][ubodt.cpp:89 ] Check point 328 a 14645 b 14645
[debug][ubodt.cpp:110] Edges connecting ab are 43555,31105
[debug][ubodt.cpp:89 ] Check point 329 a 14645 b 14645
[debug][ubodt.cpp:110] Edges connecting ab are 43555,31105
[debug][ubodt.cpp:89 ] Check point 330 a 14645 b 14645
[debug][ubodt.cpp:110] Edges connecting ab are 43555,31105
[debug][ubodt.cpp:89 ] Check point 331 a 14645 b 10593
Segmentation fault (core dumped)

I run it multiple times and the Segfault appears always at [debug][ubodt.cpp:89 ] Check point 331 a 14645 b 10593

gottfriedmrs commented 3 years ago

I re-compute the UBODT table with delta=0.01degree and everything is working fine now

Indeed the old UBODT was incomplete, the end of the table looks like:


15259;623;18161;622;35067;0.00793212
15259;15272;18161;175;35067;0.00409116
15259;11238;18161;11328;35067;0.00996191
15259;18177;18161;18194;35067;0.00192453
15259;7418;18161;7420;35067;0.0029295
15259;6430;18161;6429;35067;0.00678415
15259;14911;18161;14909;35067;0.00572934
15259;19241;18161;14479;35067;0.0076995
15259;7419;18161;7418;35067;0.00325798
15259;6268;18161;9153;35067;0.0039604
15259;6269;

Feature suggestion: Warning for incomplete UBODT Table

Thanks again for your time Can.

gottfriedmrs commented 3 years ago

One quick question:

I am not very comfortable with the process of update/downgrade of FMM. How to know the actual version of FMM in a given python environment ? And how to know which fmm.exe is used when we type "fmm" in cygwin ?

Thanks !

cyang-kth commented 3 years ago

Currently there is no version information recorded in FMM. You can only run git log to check which commit the current version is on.

git log
commit 344fb8c6eb6b4ea3ffbb8607ea781f3cdd8d65a4 (HEAD -> master, tag: v0.1.1, origin/master, origin/HEAD)
Author: Can Yang 
Date:   Fri Dec 4 13:54:37 2020 +0100

    Update README of cpp-api

commit 12cf08e77366e6e197a272caf994520112f02f79
Author: Can Yang 
Date:   Fri Dec 4 10:57:53 2020 +0100

    Update cpp-api example

You can run

git pull origin master

to fetch the latest version of the code and rebuild the program.

You can also checkout an old version according to the commit id.

git checkout 12cf08e77366e6e197a272caf994520112f02f79

cyang-kth / fmm

Segmentation fault #156