CityOfNewYork / geoclient

Geoclient geocoder
56 stars 14 forks source link

Inconsistencies between geoclient and geo support #32

Closed SPTKL closed 5 years ago

SPTKL commented 5 years ago

Hi there,

I'm currently using python-geosupport (https://github.com/ishiland/python-geosupport) with geosupport linux version-18d_18.4 and I noticed an issue.

For some of the addresses that have address number our of range, geosupport(with the help of geosupport python wrapper here) would give the results with return code 42 without a lat lon, but in geoclient, same address would give message2 "address number out of range" but with a corresponding lat and lon. Both methods are using function 1B, however they are giving different output.

Here are some examples: g.address(house_number='45-06', street_name='64 STREET', borough_code='QN') would return: { 'Geosupport Return Code 2 (GRC 2)': '42', 'Warning Code': '', 'Continuous Parity Indicator / Duplicate Address Indicator': '', 'Federal Classification Code': '', 'Return Code': '42', 'Feature Type Code': '', 'Cooperative ID Number': '', 'Bike Traffic Direction': '', 'LIST OF GEOGRAPHIC IDENTIFIERS': [], 'Reserved for Tax Map Page Number': '', 'From Actual Segment Node ID': '0066650', '2000 Census Block': '1003', 'FDNY ID': '', 'Number of Street Frontages of Lot': '', 'Filler - Tax Lot Version No. Billing BBL': '', 'From Preferred LGCs First Set of 5': '', 'No. of Cross Streets at High Address End': '1', 'Physical ID': '0082643', 'UNIT - SORT FORMAT': { 'UNIT - SORT FORMAT': '', 'Unit - Identifier': '', 'Unit - Type': '' }, 'Irregularly-Shaped Lot Flag': '', 'Reason Code 2': '', "High BBL of this Building's Condominium Units": '', 'High House Number of Block Face': '100046098AA', 'Low House Number - Display Format': '', 'Bike Lane': '', 'TPAD New BIN': '', 'PUMA Code': '04109', 'List of Street Codes': [], 'Radius of Circle': '', 'Fire Company Type': 'E', 'DCP Preferred LGC': '01', 'To LION Node ID': '0045277', 'Blockface ID': '0102263027', 'Community Development Eligibility Indicator': 'E', 'First Street Name Normalized': '64 STREET', 'Street Width': '30', 'Side of Street Indicator': 'R', 'BOROUGH BLOCK LOT (BBL)': { 'Tax Block': '', 'Tax Lot': '', 'BOROUGH BLOCK LOT (BBL)': '', 'Borough Code': '' }, 'Police Patrol Borough Command': '6', 'Bike Lane 2': '', 'Sanitation Regular Collection Schedule': 'WS', 'Number of Parking Lanes on the Street': '2', 'List of Geographic Identifiers Overflow Flag': '', 'LION Key for Vanity Address': '4572602010', 'Filler': '', 'Third Street Name Normalized': '', 'Filler - Tax Log Version No. of High BBL': '', 'Node Number': '', 'Street Width Irregular': '', 'Turn Restrictions': '', 'List of Cross Street Names at High Address End': 'LAUREL HILL BOULEVARD', 'Street Attribute Indicator': 'H', 'Building Identification Number (BIN)': '', 'BOE Preferred B7SC': '41349001', 'Speed Limit': '25', 'No Cross Street Calculation Flag': '', 'Segment Identifier': '0113246', 'Generic ID': '0071729', 'Atomic Polygon': '149', 'Fire Division': '14', '2010 Census Block Suffix': '', 'RPAD Building Classification Code': '', 'DSNY Bulk Pickup Schedule': 'EWS', 'Continuous Parity Indicator/Duplicate Address Indicator': '', 'Assembly District': '30', 'Angle to To Node - Alpha Value': '', 'TPAD BIN Status': '', 'Segment Type Code': 'U', 'Building Identification Number (BIN) of Input Address or NAP': '', 'Sanitation Recycling Collection Schedule': 'EW', 'B10SC - First Borough and Street Code': '41349001010', 'Self-Check Code (SCC) of Billing BBL': '', 'Secant Location Related to Curve': '', 'Internal Use': '', 'NYPD ID': '', 'Marble Hill/Rikers Island Alternate Borough Flag': '', 'Underlying Address Number for NAPs': '', 'Alley or Cross Street List Flag': '', 'Tax Map Number Section & Volume': '', 'Reason Code Qualifier': '', 'SPATIAL COORDINATES OF ACTUAL SEGMENT': { 'Z Coordinate, High Address End': '', 'Y Coordinate, Low Address End': '0209247', 'SPATIAL COORDINATES OF ACTUAL SEGMENT': '10114960209247 10114760208598', 'Y Coordinate, High Address End': '0208598', 'X Coordinate, High Address End': '1011476', 'X Coordinate, Low Address End': '1011496', 'Z Coordinate, Low Address End': '' }, 'Filler for Tax Lot Version Number': '', 'Fire Company Number': '292', "Low BBL of this Building's Condominium Units": '', 'Traffic Direction': 'T', 'List of 4 LGCs': '01', 'Hurricane Evacuation Zone (HEZ)': 'X', 'To Preferred LGCs First Set of 5': '01', 'Marble Hill/Rikers Island Alternative Borough Flag': '', 'COMMUNITY DISTRICT': { 'COMMUNITY DISTRICT': '402', 'Community District Number': '02', 'Community District Borough Code': '4' }, 'Underlying B7SC': '41349001', 'Right Of Way Type': '', 'Legacy Segment ID': '0113246', 'SBVP (SANBORN MAP IDENTIFIER)': { 'Volume Number': '', 'SBVP (SANBORN MAP IDENTIFIER)': '', 'Page Number Suffix': '', 'Sanborn Borough Code': '', 'Page Number': '', 'Volume Number Suffix': '' }, 'Unit - Display Format': '', 'Angle to From Node - Beta Value': '', 'Condominium Unit ID Number': '', 'Filler - Tax Lot Version No. of Low BBL': '', 'Sanitation District': '402', 'DOT Street Light Contractor Area': '4', 'Business Improvement District (BID)': '', 'Reason Code': '', 'Community School District': '24', 'List of Cross Street Names at Low Address End': 'BEND', 'SPATIAL COORDINATES OF CENTER OF CURVATURE': { 'SPATIAL COORDINATES OF CENTER OF CURVATURE': '00000000000000', 'X Coordinate': '0000000', 'Y Coordinate': '0000000', 'Z Coordinate': '' }, 'Police Patrol Borough': 'QN', 'Low House Number of Block Face': '100044000AA', 'NIN': '', 'Message': '', 'Number of Cross Streets at Low Address End': '1', 'Individual Segment Length': '', 'Number of Street Codes and Street Names in List': '', 'Number of Existing Structures on Lot': '', 'Congressional District': '14', 'Street Status': '2', '2010 Census Tract': '245', 'Number of Total Lanes on the Street': '3', 'Curve Flag': '', 'State Senatorial District': '12', 'House Number - Display Format': '45-06', 'Number of Travel Lanes on the Street': '1', 'Longitude': '', 'B10SC - Third Borough and Street Code': '', 'X-Y Coordinates of Lot Centroid': '', 'Condominium Flag': '', 'TPAD New BIN Status': '', 'RPAD Self-Check Code (SCC) for BBL': '', 'Number of Cross Streets at High Address End': '1', 'Civil Court District': '02', 'Low House Number - Sort Format': '', 'Street Width Maximum': '30', 'From Preferred LGCs Second Set of 5': '', 'Segment Orientation': 'S', 'NTA Name': 'WOODSIDE', 'Segment Azimuth': '268', 'ZIP Code': '11377', 'List of Cross Streets at High Address End': '45309001', 'Segment Length in Feet': '00650', 'Election District': '081', 'To Actual Segment Node ID': '0045277', 'Neighborhood Tabulation Area (NTA)': 'QN63', 'Number of Entries in List of Geographic Identifiers': '', 'From LION Node ID': '0066650', 'Reserved for Possible Z Coordinate': '', 'Special Address Generated Record Flag': '', 'Health Area': '0920', '2000 Census Block Suffix': '', 'Split Election District Flag': '', 'Low House Number of Defining Address Range': '', 'Sanitation Collection Scheduling Section and Subsection': '1E', '2010 Census Block': '3004', 'Second Street Name Normalized': '', 'Vacant Lot Flag': '', 'Warning Code 2': '', 'Reason Code Qualifier 2': '', 'Spatial X-Y Coordinates of Address': '10114830208932', 'DSNY Snow Priority Code': 'S', 'Roadway Type': '1', 'Coincidence Segment Count': '1', '1990 Census Tract': '245', 'SPATIAL COORDINATES OF SEGMENT': { 'Z Coordinate, High Address End': '', 'Z Coordinate, Low Address End': '', 'Y Coordinate, Low Address End': '0209247', 'Y Coordinate, High Address End': '0208598', 'X Coordinate, High Address End': '1011476', 'X Coordinate, Low Address End': '1011496', 'SPATIAL COORDINATES OF SEGMENT': '10114960209247 10114760208598' }, 'List of Cross Streets at Low Address End': '435780', 'Split Low House Number': '100045000AA', 'Interior Lot Flag': '', 'DSNY Organic Recycling Schedule': 'EW', 'Fraction for Curve Calculation': '', 'List of Street Names': [], 'Health Center District': '41', 'Fire Battalion': '46', 'TPAD Conflict Flag': '', '2000 Census Tract': '245', 'Street Attribute Indicators': 'H', 'Geosupport Return Code (GRC)': '00', 'BOE LGC Pointer': '1', 'B10SC - Second Borough and Street Code': '', 'Latitude': '', 'Police Precinct': '108', 'First Borough Name': 'QUEENS', 'DCP Zoning Map': '', 'Message 2': 'ADDRESS NUMBER OUT OF RANGE', 'Side of Street of Vanity Address': 'R', 'List of Second Set of 5 LGCs': '', 'DOF Condominium Identification Number': '', 'House Number - Sort Format': '100045006AA', 'USPS Preferred City Name': 'WOODSIDE', 'LION KEY': { 'LION KEY': '4572602010', 'Borough Code': '4', 'Sequence Number': '02010', 'Face Code': '5726' }, 'City Council District': '26', 'DCP Commercial Study Area': '', 'To Preferred LGCs Second Set of 5': '', 'Condominium Billing BBL': '', 'BOE Preferred Street Name': '64 STREET', 'Corner Code': '', 'STROLLING KEY': { 'Side of Street Indicator': '', '5-Digit Street Code of ON- Street': '', 'STROLLING KEY': '', 'High House Number': '', 'Borough': '' } }

and in geoclient { "address": { "assemblyDistrict": "30", "boardOfElectionsPreferredLgc": "1", "boePreferredStreetName": "64 STREET", "boePreferredstreetCode": "41349001", "boroughCode1In": "4", "censusBlock2000": "1003", "censusBlock2010": "3004", "censusTract1990": " 245 ", "censusTract2000": " 245 ", "censusTract2010": " 245 ", "cityCouncilDistrict": "26", "civilCourtDistrict": "02", "coincidentSegmentCount": "1", "communityDistrict": "402", "communityDistrictBoroughCode": "4", "communityDistrictNumber": "02", "communitySchoolDistrict": "24", "congressionalDistrict": "14", "crossStreetNamesFlagIn": "E", "dcpPreferredLgc": "01", "dotStreetLightContractorArea": "4", "dynamicBlock": "149", "electionDistrict": "081", "fireBattalion": "46", "fireCompanyNumber": "292", "fireCompanyType": "E", "fireDivision": "14", "firstBoroughName": "QUEENS", "firstStreetCode": "41349001010", "firstStreetNameNormalized": "64 STREET", "fromLionNodeId": "0066650", "genericId": "0071729", "geosupportFunctionCode": "1B", "geosupportReturnCode": "00", "geosupportReturnCode2": "42", "healthArea": "0920", "healthCenterDistrict": "41", "highCrossStreetB5SC1": "453090", "highCrossStreetCode1": "45309001", "highCrossStreetName1": "LAUREL HILL BOULEVARD", "highHouseNumberOfBlockfaceSortFormat": "100046098AA", "houseNumber": "45-06", "houseNumberIn": "45-06", "houseNumberSortFormat": "100045006AA", "hurricaneEvacuationZone": "X", "instructionalRegion": "QN", "interimAssistanceEligibilityIndicator": "E", "latitude": 40.74010473007666, "legacySegmentId": "0113246", "lionBoroughCode": "4", "lionBoroughCodeForVanityAddress": "4", "lionFaceCode": "5726", "lionFaceCodeForVanityAddress": "5726", "lionKey": "4572602010", "lionKeyForVanityAddress": "4572602010", "lionSequenceNumber": "02010", "lionSequenceNumberForVanityAddress": "02010", "listOf4Lgcs": "01", "longitude": -73.9017263539154, "lowCrossStreetB5SC1": "435780", "lowCrossStreetCode1": "435780", "lowCrossStreetName1": "BEND", "lowHouseNumberOfBlockfaceSortFormat": "100044000AA", "message2": "ADDRESS NUMBER OUT OF RANGE", "nta": "QN63", "ntaName": "Woodside", "numberOfCrossStreetB5SCsHighAddressEnd": "1", "numberOfCrossStreetB5SCsLowAddressEnd": "1", "numberOfCrossStreetsHighAddressEnd": "1", "numberOfCrossStreetsLowAddressEnd": "1", "numberOfStreetCodesAndNamesInList": "02", "physicalId": "0082643", "policePatrolBoroughCommand": "6", "policePrecinct": "108", "returnCode1a": "42", "returnCode1e": "00", "roadwayType": "1", "sanitationCollectionSchedulingSectionAndSubsection": "1E", "sanitationDistrict": "402", "sanitationOrganicsCollectionSchedule": "EW", "sanitationRecyclingCollectionSchedule": "EW", "sanitationRegularCollectionSchedule": "WS", "sanitationSnowPriorityCode": "S", "segmentAzimuth": "268", "segmentIdentifier": "0113246", "segmentLengthInFeet": "00650", "segmentOrientation": "S", "segmentTypeCode": "U", "sideOfStreetIndicator": "R", "sideOfStreetOfVanityAddress": "R", "splitLowHouseNumber": "100045000AA", "stateSenatorialDistrict": "12", "streetCode1": "43578001", "streetCode6": "45309001", "streetName1": "BEND", "streetName1In": "64 STREET", "streetName6": "LAUREL HILL BOULEVARD", "streetStatus": "2", "streetWidth": "30", "toLionNodeId": "0045277", "toPreferredLgcsFirstSetOf5": "01", "trafficDirection": "T", "underlyingStreetCode": "41349001", "uspsPreferredCityName": "WOODSIDE", "workAreaFormatIndicatorIn": "C", "xCoordinate": "1011483", "xCoordinateHighAddressEnd": "1011476", "xCoordinateLowAddressEnd": "1011496", "xCoordinateOfCenterofCurvature": "0000000", "yCoordinate": "0208932", "yCoordinateHighAddressEnd": "0208598", "yCoordinateLowAddressEnd": "0209247", "yCoordinateOfCenterofCurvature": "0000000", "zipCode": "11377" } }

Here are another sample address houseNumber=718&street=57+STREET&borough=Brooklyn

How are the lon lat generated in geoclient for those case?

mlipper commented 5 years ago

Hi,

As you mentioned, the /geoclient/v1/address endpoint is simply a pass-through to the Geosupport function 1B. The information returned by function 1B is composed of results from two separate address-related functions 1E and 1A. These functions can succeed or fail independently as was the case in the example you've given (1E succeeded, 1A falied). The confusion (I think) stems from the differing spatial fields each function returns:

Within the unique scope of each function, you''ll notice the same real-world point location is returned in two different coordinate reference systems:

Sooo...in the example you've given, we see that:

{ address: {
    ...
    "geosupportReturnCode": "00",  <-- function 1E succeeded
    "returnCode1e": "00",
    "geosupportReturnCode2": "42", <-- function 1A falied
    "returnCode1a": "42",
    ...
    // 1E location is available:
    "latitude": 40.74010473007666,  <-- EPSG:4326 ("lat/long")
    "longitude": -73.9017263539154,
    "xCoordinate": "1011483", <-- EPSG:2263 ("NY State Plane")
    "yCoordinate": "0208932",
    ...
    // 1A location info is NOT available
}}

In plain English, the above essentially means "45-06 is valid house number on the range assigned to 64th street in queens but there's no record of that actual address on an existing tax lot".

For comparision, try a call with address 106 East 64 street, Manhattan which succeeds for both functions. In addtion to the 1E fields, you'll notice the 1A info also:

{ address: {
    ...
    // 1A is available (yay!)
    "internalLabelXCoordinate": "0993316",
    "internalLabelYCoordinate": "0218250",
    ...
    "latitudeInternalLabel": 40.76571755841329,
    "longitudeInternalLabel": -73.9672716432342,
    ...
}}

I'm not sure what's going on in the case of python-geosupport, so I'd recommend asking for clarification there.

Please let me know if I've answered the right question.

Thanks! Matt

SPTKL commented 5 years ago

Thank you, that's super helpful!

mlipper commented 5 years ago

Anytime. Glad it helped.