Use `copyfrom` for updating vehicles

[cedarbaum]

Overview

Use copyfrom when updating vehicle entities. This has 2 advantages:

• Simpler implementation
• The NYC bus system vehicle updates are faster (see below performance comparison)

This change also adds additional tests to further verify multi-feed update behavior (i.e. that multiple feeds updating a system's vehicles don't interfere with each other).

Performance comparison

Below shows metrics from before/after this change over the course of 10 feed updates.

Single insert/update commands (current implementation)

transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.005"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.01"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.025"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.05"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.1"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.25"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.5"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="1"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="2.5"} 8
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="5"} 9
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="10"} 10
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="+Inf"} 10
transiter_feed_update_database_latency_sum{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses"} 21.973
transiter_feed_update_database_latency_count{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses"} 10

Using copyfrom:

transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.005"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.01"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.025"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.05"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.1"} 0
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.25"} 8
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="0.5"} 8
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="1"} 9
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="2.5"} 10
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="5"} 10
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="10"} 10
transiter_feed_update_database_latency_bucket{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses",le="+Inf"} 10
transiter_feed_update_database_latency_sum{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses"} 3.8310000000000004
transiter_feed_update_database_latency_count{feed_id="vehicles",feed_type="GTFS_REALTIME",system_id="us-ny-buses"} 10

[jamespfennell]

Awesome! Also fewer SQL queries to maintain!

[jamespfennell]

Thanks for all the unit tests, too! I'm planning on maybe doing some refactoring of that unit test file to be a little simpler - my sense is that there should actually be only a single parameterized test, and not separate tests for each entity. It's a good suite of unit tests to invest time in because it's quite load bearing!