instructions and/or script to pre-fetch GeoJSON for a canned/offline demo

[nvkelso]

@patriciogonzalezvivo Can you document your steps to create the "static" GeoJSON, please?

[kevinkreiser]

it was kind of documented here: https://github.com/mapzen/open-traffic/issues/37 but just note that this doesnt get real speed data, it takes osmlr and adds fake speed data to it.

• download the osmlr geojson from the provided s3 location (aws s3 cp s3://somebucket somepath/tosomefile.tgz some_dir)
• unpack it (tar pxvf whateverfile.tgz)
• make a python script that looks like the one below
• then run it like python script.py some_dir 4 where 4 is the number of threads

your python script should look like this:

#!/usr/bin/env python
import os
import sys
import json
from datetime import datetime
from Queue import Queue
from threading import Thread

#hack json dump precision
json.encoder.FLOAT_REPR = lambda o: "%.5f" % o

#pick jobs off the work queue
def work(thread_id, work_queue):
  #pick a number from 20 to 100kph
  hacked_speed = 0
  while True:
    file_path = work_queue.get()
    sys.stderr.write('%s Thread %d will amend %s' % (datetime.utcnow().strftime('%Y.%m.%d %H:%M:%S'), thread_id, file_path) + os.linesep)
    #open the geojson and update the speeds
    with open(file_path, 'r+') as handle:
      feature_collection = json.load(handle)
      for feature in feature_collection[u'features']:
        #TODO: its not just speed anymore its an array or dict, ask @gknisely for info
        feature[u'properties'][u'speed'] = (hacked_speed % 80) + 20
        hacked_speed += 1
      #overwrite the file
      handle.seek(0)
      json.dump(feature_collection, handle, separators=(',', ':'))
      handle.truncate()
    sys.stderr.write('%s Thread %d has amended %s' % (datetime.utcnow().strftime('%Y.%m.%d %H:%M:%S'), thread_id, file_path) + os.linesep)
    work_queue.task_done()

#a place to store outstanding work
work_queue = Queue(maxsize=int(sys.argv[2]))

#workers to work on them
for thread_id in range(0, int(sys.argv[2])):
  worker = Thread(target=work, args=(thread_id, work_queue,))
  worker.setDaemon(True)
  worker.start()

#recurse over files putting onto the work queue
for path, dirs, files in os.walk(sys.argv[1]):
  for file_name in files:
    file_path = os.path.join(path, file_name)
    work_queue.put(file_path)

#wait for it to finish
work_queue.join()

[drewda]

Now that we have a working API, it should no longer be necessary to use your script to generate "synthesized"/faked speeds, @kevinkreiser.

I think the process we're going for is:

1) Construct appropriate query to the Open Traffic v2 API (using the available query parameters: https://github.com/opentraffic/api/blob/dev/py/query.py#L8-L21 ) 2) Save the results of the query as a local GeoJSON file 3) Adjust Tangram source to point to that GeoJSON file (in https://github.com/opentraffic/tangram-viz-experiments/blob/gh-pages/scene.yaml#L9-L12 )

At a minimum, we can document these steps in the readme.

In the future, @nvkelso suggests it could also be useful to allow multiple GeoJSON files/extracts to be placed in a directory and selected by a query parameter or a drop-down in the UI.