pelias-deprecated / quattroshapes

(DEPRECATED) Pelias import pipeline for Quattroshapes
https://github.com/pelias/whosonfirst
5 stars 1 forks source link

Remove child processes from centroid computations #17

Closed sevko closed 9 years ago

sevko commented 9 years ago

The importer currently instantiates a child process per polygon centroid computation, which is insanely expensive and slows it down significantly. Move that operation back into the parent process.

sevko commented 9 years ago

Needs to be tested in a dev import.

sevko commented 9 years ago

Everything looks fine in the latest dev import. A sample query returns the following results:

{
    "bbox": [
        -85.10653,
        33.472521,
        -0.275005,
        51.840105
    ],
    "date": 1425934658090,
    "features": [
        {
            "geometry": {
                "coordinates": [
                    -84.111525,
                    41.326702
                ],
                "type": "Point"
            },
            "properties": {
                "admin0": "United States",
                "admin1": "Ohio",
                "admin2": "Henry",
                "alpha3": "USA",
                "id": "313:adm2:us:usa:henry",
                "layer": "admin2",
                "name": "Henry",
                "text": "Henry, Ohio",
                "type": "admin2"
            },
            "type": "Feature"
        },
        {
            "geometry": {
                "coordinates": [
                    -0.275005,
                    51.840105
                ],
                "type": "Point"
            },
            "properties": {
                "admin0": "United Kingdom",
                "admin1": "Hertfordshire",
                "alpha3": "GBR",
                "id": "168:adm1:gb:gbr:hertfordshire",
                "layer": "admin1",
                "name": "Hertfordshire",
                "text": "Hertfordshire, United Kingdom",
                "type": "admin1"
            },
            "type": "Feature"
        },
        {
            "geometry": {
                "coordinates": [
                    -84.13845,
                    33.472521
                ],
                "type": "Point"
            },
            "properties": {
                "admin0": "United States",
                "admin1": "Georgia",
                "admin2": "Henry",
                "alpha3": "USA",
                "id": "1011:adm2:us:usa:henry",
                "layer": "admin2",
                "name": "Henry",
                "text": "Henry, Georgia",
                "type": "admin2"
            },
            "type": "Feature"
        },
        {
            "geometry": {
                "coordinates": [
                    -85.10653,
                    38.466212
                ],
                "type": "Point"
            },
            "properties": {
                "admin0": "United States",
                "admin1": "Kentucky",
                "admin2": "Henry",
                "alpha3": "USA",
                "id": "1589:adm2:us:usa:henry",
                "layer": "admin2",
                "name": "Henry",
                "text": "Henry, Kentucky",
                "type": "admin2"
            },
            "type": "Feature"
        },
        {
            "geometry": {
                "coordinates": [
                    -82.503221,
                    35.322422
                ],
                "type": "Point"
            },
            "properties": {
                "admin0": "United States",
                "admin1": "North Carolina",
                "admin2": "Henderson",
                "alpha3": "USA",
                "id": "159:adm2:us:usa:henderson",
                "layer": "admin2",
                "name": "Henderson",
                "text": "Henderson, North Carolina",
                "type": "admin2"
            },
            "type": "Feature"
        },
        {
            "geometry": {
                "coordinates": [
                    -84.116753,
                    39.754774
                ],
                "type": "Point"
            },
            "properties": {
                "admin0": "United States",
                "admin1": "Ohio",
                "admin2": "Montgomery",
                "id": "4888:_:_:usa:hearthstone",
                "layer": "neighborhood",
                "name": "Hearthstone",
                "text": "Hearthstone, Montgomery, Ohio",
                "type": "neighborhood"
            },
            "type": "Feature"
        },
        {
            "geometry": {
                "coordinates": [
                    -82.2052,
                    38.80975
                ],
                "type": "Point"
            },
            "properties": {
                "admin0": "United States",
                "admin1": "Ohio",
                "admin2": "Gallia",
                "id": "4901:_:_:usa:hedgewood",
                "layer": "neighborhood",
                "name": "Hedgewood",
                "text": "Hedgewood, Gallia, Ohio",
                "type": "neighborhood"
            },
            "type": "Feature"
        },
        {
            "geometry": {
                "coordinates": [
                    -81.590159,
                    40.035028
                ],
                "type": "Point"
            },
            "properties": {
                "admin0": "United States",
                "admin1": "Ohio",
                "admin2": "Guernsey",
                "id": "4917:_:_:usa:henderson_heights",
                "layer": "neighborhood",
                "name": "Henderson Heights",
                "text": "Henderson Heights, Guernsey, Ohio",
                "type": "neighborhood"
            },
            "type": "Feature"
        }
    ],
    "type": "FeatureCollection"
}
dianashk commented 9 years ago

+1

sevko commented 9 years ago

Waiting on @missinglink .