Response to Request

[awk11]

Keep Track of each response to each batch separately rather than replacing. Thus, we to supplement response data with the batch number. The intent is to support roll-up performance evaluation system can assess improvements over subsequent submissions to prior data as a way to measure performance improvement. There are three fundamental ways to do this (or more?): (1) Add an initial '0' column with batch id prior to saving and just keep appending to the result CSV file. (2) Keep appending the result CSV file and skip adding batch id. It is irrelevant. (3) Make the batch responses into separate CSV files with batch id in the file name. The choice is based on making eval easier. Thus, #2 (closed) is likely the best option allowing eval to rollup in different ways: for standard eval, use the first instance of an example id, for performance improvement look at the last. The first two will support panda time roll up. #1 and #2 (closed) are common in transaction systems with advanced analytic queries where the the chosen value for any given entry (example id) is where the max(insert date) == row.insert data grouped by example id.

[rwgdrummer]

To confirm, I think we decided replacement is no longer supported.