search

cyang-kth / fmm

Fast map matching, an open source framework in C++
https://fmm-wiki.github.io/
Apache License 2.0
886 stars 213 forks source link

Edges shapefile csv example #48

Closed MGasztold closed 4 years ago

MGasztold commented 4 years ago

Describe the bug I cannot understand deep enough what is the exact format of the shapefile that needs to be provided for ubodt_gen program.

Expected behavior a csv file from which the edges.shp file can be generated with ogr2ogr binary is available.

cyang-kth commented 4 years ago

@MGasztold

Currently the program does not support reading network in CSV format. The network should be stored as a shapefile with fields of id, source and target.

You can also run ogrinfo -so -al edges.shp to see the example provided.

INFO: Open of `edges.shp'
      using driver `ESRI Shapefile' successful.

Layer name: edges
Geometry: Line String
Feature Count: 30
Extent: (0.000000, 0.000000) - (4.000000, 4.000000)
Layer SRS WKT:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0,
        AUTHORITY["EPSG","8901"]],
    UNIT["degree",0.0174532925199433,
        AUTHORITY["EPSG","9122"]],
    AUTHORITY["EPSG","4326"]]
_uid_: String (21.0)
id: String (21.0)
source: String (21.0)
target: String (21.0)
cost: Real (24.15)
x1: Real (24.15)
y1: Real (24.15)
x2: Real (24.15)
y2: Real (24.15)

The network file should contain these 3 fields. 

id: String (21.0)
source: String (21.0)
target: String (21.0)
MGasztold commented 4 years ago

The problem is that I cannot see it:

example git:(master) ✗ ogrinfo -so -al edges.shp
FAILURE:
Unable to open datasource `edges.shp' with the following drivers.
  -> PCIDSK
  -> netCDF
  -> JPEG2000
  -> PDF
  -> MBTiles
  -> EEDA
  -> ESRI Shapefile
  -> MapInfo File
  -> UK .NTF
  -> OGR_SDTS
  -> S57
  -> DGN
  -> OGR_VRT
  -> REC
  -> Memory
  -> BNA
  -> CSV
  -> NAS
  -> GML
  -> GPX
  -> KML
  -> GeoJSON
  -> GeoJSONSeq
  -> ESRIJSON
  -> TopoJSON
  -> Interlis 1
  -> Interlis 2
  -> OGR_GMT
  -> GPKG
  -> SQLite
  -> OGR_DODS
  -> ODBC
  -> WAsP
  -> PGeo
  -> MSSQLSpatial
  -> PostgreSQL
  -> OpenFileGDB
  -> XPlane
  -> DXF
  -> CAD
  -> Geoconcept
  -> GeoRSS
  -> GPSTrackMaker
  -> VFK
  -> PGDUMP
  -> OSM
  -> GPSBabel
  -> SUA
  -> OpenAir
  -> OGR_PDS
  -> WFS
  -> WFS3
  -> HTF
  -> AeronavFAA
  -> Geomedia
  -> EDIGEO
  -> GFT
  -> SVG
  -> CouchDB
  -> Cloudant
  -> Idrisi
  -> ARCGEN
  -> SEGUKOOA
  -> SEGY
  -> XLS
  -> ODS
  -> XLSX
  -> ElasticSearch
  -> Walk
  -> Carto
  -> AmigoCloud
  -> SXF
  -> Selafin
  -> JML
  -> PLSCENES
  -> CSW
  -> VDV
  -> GMLAS
  -> MVT
  -> TIGER
  -> AVCBin
  -> AVCE00
  -> NGW
  -> HTTP

Here is my version info:

example git:(master) ✗ ogrinfo --version
GDAL 2.4.4, released 2020/01/08
MGasztold commented 4 years ago

What I would like to be able to do is to have a .csv and .vrt files from which edges.shp can be generated with ogr2ogr like:

ogr2ogr -f "ESRI Shapefile" edges.shp edges.vrt

Is it possible that you provide that to example folder so a full picture of the necessary input is visible?

Without it I have bad time with generating proper input from my own system for which I consider using fmm.

Thank you

cyang-kth commented 4 years ago

@MGasztold

The edges.shp is available under the path example/data/edges.shp. I think you forget to add the data folder to your path when running ogrinfo.

MGasztold commented 4 years ago

ok you are right about the data folder.

data git:(master) ✗ ogrinfo -so -al edges.shp 
INFO: Open of `edges.shp'
      using driver `ESRI Shapefile' successful.

Layer name: edges
Geometry: Line String
Feature Count: 30
Extent: (0.000000, 0.000000) - (4.000000, 4.000000)
Layer SRS WKT:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0,
        AUTHORITY["EPSG","8901"]],
    UNIT["degree",0.0174532925199433,
        AUTHORITY["EPSG","9122"]],
    AUTHORITY["EPSG","4326"]]
_uid_: String (21.0)
id: String (21.0)
source: String (21.0)
target: String (21.0)
cost: Real (24.15)
x1: Real (24.15)
y1: Real (24.15)
x2: Real (24.15)
y2: Real (24.15)

That part works now.

Now I just want to create a edges.csv and edges.vrt manually so I can reproduce edges.shp with ogr2ogr2. I need my system to produce proper formats to match fmm.

Maybe I am asking for too basic thing, I am new to GDAL. I can probably derive the proper csv format using ogrinfo outputs.

cyang-kth commented 4 years ago

@MGasztold If you have a network as CSV file as you mentioned, you can also use QGIS to add it as a delimited csv layer then export as a shapefile. It may be a new feature added to fmm to directly support CSV network file as input.

MGasztold commented 4 years ago

I do not have a network as .csv file. I want to generate it properly in my system so it matches fmm. That's my issue which I am trying to solve. To solve it I need to learn how to generate your example edges.shp file from scratch using only text files and terminal so it runs successfuly with fmm. Only then I can code proper network .csv file generation on my side.

MGasztold commented 4 years ago

To clear the situation. My plan is that a .csv file of the network graph that I have in my system, that today is not using GDAL formats, is generated. Then a shapefile will be generated from this csv and then this shapefile is the input to fmm.

MGasztold commented 4 years ago

Ok so I managed to get the csv. I stripped it a bit to make it shorter. it looks like this:

_uid_,id,source,target,cost,x1,y1,x2,y2
"1","1","1","2",1.000000000000000,2.000000000000000,1.000000000000000,2.000000000000000,0.000000000000000
"2","2","2","1",1.000000000000000,2.000000000000000,0.000000000000000,2.000000000000000,1.000000000000000
"3","3","3","1",1.000000000000000,3.000000000000000,1.000000000000000,2.000000000000000,1.000000000000000
"4","4","4","3",1.000000000000000,4.000000000000000,1.000000000000000,3.000000000000000,1.000000000000000
"5","5","1","5",1.000000000000000,2.000000000000000,1.000000000000000,2.000000000000000,2.000000000000000

Now my task is to write proper .vrt XML file so this .csv file is properly converted to shapefile that fmm expects, like edges.shp.

All with command: ogr2ogr -f "ESRI Shapefile" edges.shp edges.vrt

If anybody can help me with that it would be great.

cyang-kth commented 4 years ago

@MGasztold

Your CSV file cannot be transformed into a shapefile as there is no geometry information.

You need to generate a CSV with geometry information then try the commands below.

1. Generate CSV from shapefile

ogr2ogr -f "CSV" edges.csv edges.shp -lco GEOMETRY=AS_WKT

The content of edge.csv is 

WKT,_uid_,id,source,target,cost,x1,y1,x2,y2
"LINESTRING (2 1,2 0)","1","1","1","2",1.000000000000000,2.000000000000000,1.000000000000000,2.000000000000000,0.000000000000000
"LINESTRING (2 0,2 1)","2","2","2","1",1.000000000000000,2.000000000000000,0.000000000000000,2.000000000000000,1.000000000000000
"LINESTRING (3 1,2 1)","3","3","3","1",1.000000000000000,3.000000000000000,1.000000000000000,2.000000000000000,1.000000000000000
"LINESTRING (4 1,3 1)","4","4","4","3",1.000000000000000,4.000000000000000,1.000000000000000,3.000000000000000,1.000000000000000

2. Generate shapefile from CSV file

Create a convert.vrt file with content of (the OGRVRTLayer name should be the same name as the csv file)

<OGRVRTDataSource>
    <OGRVRTLayer name="edges">
        <SrcDataSource>edges.csv</SrcDataSource>
        <GeometryType>wkbLineString</GeometryType>
        <Field name="id" src="id" />
        <Field name="source" src="source" />
        <Field name="target" src="target" />
        <GeometryField encoding="WKT" field='WKT' > </GeometryField >
    </OGRVRTLayer>
</OGRVRTDataSource>

Then execute 

ogr2ogr -f "ESRI Shapefile" edges2.shp convert.vrt

It will generate a new shapefile called: edges2.shp.

MGasztold commented 4 years ago

@cyang-kth This works! Thank you a lot, it is a great help!

I think you should add this to the https://github.com/cyang-kth/fmm/wiki/Input-and-output , maybe somebody will find it useful as I do one day.

Thank you !

cyang-kth commented 4 years ago

@MGasztold

Thanks for this suggestion. I have updated the wiki page of fmm with that instructions

https://fmm-wiki.github.io/docs/documentation/input/#convert-road-network-in-shapefile-fromto-csv-format