m-lab / etl-gardener

Gardener provides services for maintaining and reprocessing mlab data.
Apache License 2.0
13 stars 5 forks source link

Add v2 Jobs API #384

Closed stephen-soltesz closed 2 years ago

stephen-soltesz commented 2 years ago

This change adds new, v2 Jobs API resources based around the Job ID and JobWithTarget resource type.

Previously, the JobWithTarget.Job was not an explicit field and the NextJob handler returned only the Job fields. This behavior is preserved for current clients (the original /job resource). With this change, the /v2/job/next resource returns a complete JobWithTarget type that includes an ID and Job subfield. Once the parser client migrates to this v2 API, we will be able to add new fields as needed to support additional functionality (configured output datasets, daily only, etc) without violating the contract between the parser and gardener JobsAPI.

Because of the different handling of the Job or JobWithTarget return type by the "v1" and "v2" APIs, there are two versions of the "next Job" handler. However, all other handlers for Update, Heartbeat and Error should be forward and backward compatible, first looking for an "id" parameter (v2 API), and falling back for earlier clients to a "job" parameter (v1 API).

The next change will add a new v2/client.go package for calling these v2 API resources, which will allow us to easily migrate the parser to this new interface.

Part of:


This change is Reviewable

coveralls commented 2 years ago

Coverage Status

Coverage increased (+0.3%) to 71.585% when pulling efc774712348e5450125255ae4a3234e7cf79318 on sandbox-soltesz-add-v2-api-and-client into 9a396452cb13716e60956e0aa8aa3560bd869d7d on master.

stephen-soltesz commented 2 years ago

Thank you!