Streets-Data-Collaborative / OpenStreetCam-GeoParsing-Tool

Create a tool that, given a city, give can pull each OpenStreetCam (OSM) Track file No. associated with that city.
Apache License 2.0
7 stars 2 forks source link

Define a getNearbytracks() function that takes a lat/lng pair as an input, and returns the `sequence_id`s of nearby OpenStreetCam tracks #3

Closed dmarulli closed 6 years ago

dmarulli commented 6 years ago

This function will essentially be mimicking the post requests sent when users click on the OpenStreetCam map. See the example chrome dev tools screenshots.

charlie-moffett commented 6 years ago

In mimicking the post requests, I'm getting an error for missing parameter "apiCode". I'm not finding that parameter in Requests Headers from the chrome dev tools screenshots.

parameters = {'Host': 'openstreetcam.org','Connection': 'keep-alive', 'Content-Length': '97', 'Accept': '*/*', 
            'Origin': 'http://openstreetcam.org', 'X-Requested-With': 'XMLHttpRequest', 'User-Agent': 
            'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36',
            'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'DNT': '1', 
            'Referer': 'http://openstreetcam.org/map/@37.809817858285484,-122.26915240287782,16z', 
            'Accept-Encoding': 'gzip, deflate', 'Accept-Language': 'en-GB,en-US;q=0.8,en;q=0.6',
            'Cookie': '_ga=GA1.2.2016380652.1514242679; PHPSESSID=r3c3t03t2h2t90r26lb3ka4nk5; _gid=GA1.2.1263359526.1514919380; currentLocation=%2Fmap%2F%4037.809817858285484%2C-122.26915240287782%2C16z'}

r = requests.post(url='http://openstreetcam.org/nearby-tracks',data=parameters)
r.text

Output: '{"status":{"apiCode":"This value should not be blank.","apiMessage":"([lat])","httpCode":400,"httpMessage":"Bad Request"}}'

I wasn't able to resolve by reading through Varun's convo with Telenav or Aleph's testOSCapi script. As I continue to troubleshoot, are you able to spot any obvious flaws in the way I'm structuring the post request?

charlie-moffett commented 6 years ago

Resolved the above - should have been passing Form Data instead of Request Headers. Here's what I'm using now to explore the post response:

# data to be sent to api
data = {'lat': '37.811784337568625', 'lng': '-122.26420640945436', 'distance': '3.125',
        'myTracks': 'false', 'filterUserNames': 'false'}

# sending post request and saving response as response object
r = requests.post(url='http://openstreetcam.org/nearby-tracks',data=data)

# extracting data in json format
jason = r.json()

# index sequence_id of nearby track
seq_id = jason['osv']['sequences'][0]['sequence_id'] # integer will cycle through nearby tracks
seq_id

Output: '11494'

Moving on now to actually defining the function that takes lat/lng as input and returns sequence_id's of all nearby tracks.

charlie-moffett commented 6 years ago

There's a parameter in the post request sent when users click on the OpenStreetCam map that we don't get from the previous function: 'distance'. Do you have any ideas as to how to deal with distance here?

Here's what I have so far:

url = 'http://openstreetcam.org/nearby-tracks'
sequence_ids = []

def getNearbytracks(lat, lng):
    '''
    Takes lat and lng as strings and
    returns nearby OSC tracks as a list
    of sequence_ids
    '''

    # form data to be sent to API
    data = {'lat': lat, 'lng': lng, 'distance': '3.125', ## what to do about distance?!
                'myTracks': 'false', 'filterUserNames': 'false'}

    # sending post request and saving response as response object
    r = requests.post(url=url, data=data)

    # extracting data in json format
    extract = r.json()
    sequences = extract['osv']['sequences']

    # store sequence id of each nearby tract in a list
    for i in range(len(sequences)):
        sequence_ids.append(sequences[i]['sequence_id'])

    return sequence_ids

getNearbytracks('37.811784337568625', '-122.26420640945436') will return ['11494', '11523'], but I knew the distance from the Form Data after clicking on the OSC map at that lat/lng.

Thanks in advance for any insights you may have!

charlie-moffett commented 6 years ago

After testing out the function with a few other lat/lng pairs from various cities, it seems distance can be variable and doesn't necessarily need to match what the post requests show. Since getIntersection() will only provide lat/lng, I think we can keep a fixed distance in getNearbytracks() of 5 -- I haven't seen any distances show up in OSC over 4.545.

dmarulli commented 6 years ago

Okay, awesome.