Consider debug/test flag in Python report service

[dnesbitt61]

Add a "debug/test" flag to the Reporter Python so that the returned data reflects what is reported to Datastore (and this flag disables sending to the Datastore). This would be a list of the following:

OSMLR segment Id
segment length
next OSMLR segment Id (can be empty or 0)
start time
end time (time at the start of the next OSMLR segment or time at the end of this segment if no next)
queue length (TBD) - Length from the end of the OSMLR segment where the queue begins.

vehicle type and producer Id will be in the return as well (common over the entire list)

The default returned from the Python report method needs to be information that the Kafka/Java code uses to "trim" portions of the trace data. It will have index (trace index) information that allows association of the segment with the input trace. This information is not needed for the Datastore.

It could be that the "test/debug" mode returns both sets of information.

[kevinkreiser]

just a couple FYIs

sending to the datastore will already be disabled by not defining the datstore_url env var. we do lack any way of doing this on a per request basis though, so its important to note that this is to be added to the request.

the code already does report everything it reports to the datastore verbatim. it needs to do this so that the kafka worker can trim the batch windows accordingly. so we dont need to add this so much as keep doing it.

[dnesbitt61]

Design discussion: Datastore does not need all fields in the data (shape indexes) and needs to add next segment logic (merging across internal segments, etc.). Trimming logic perhaps only needs to know the shape index (index into the reported trace) that can be trimmed - might be best to put that logic in the Python code. Where to decide whether to report on local segments (and local next segments)? Could be cone with logic in the Reporter (based on config) or could be done at Datastore (or both). Doing in Reporter could keep sensitive(privacy) concerns of residential areas within the TNC and would reduce amount of data sent to the Datastore. Doing at Datastore would allow easier future updates to handle local roads reporting (since centralized at the Datastore rather the distributed across different TNCs).

[dnesbitt61]

Debug flag is not needed since other Reporter stages will now aggregate and send to Datastore. One can simply call the Python report method to test individual traces and receive results about segment matches, next segment transitions, queue_lengths, etc.

[drewda]

closed by #71