Matatika / tap-spotify

Singer tap for Spotify
GNU Affero General Public License v3.0
6 stars 2 forks source link

ci: Warn when top items streams return no records, rather than fail #55

Closed ReubenFrankel closed 6 months ago

ReubenFrankel commented 6 months ago

From the Web API docs:

Over what time frame the affinities are computed. Valid values: long_term (calculated from ~1 year of data and including all new data as it becomes available), medium_term (approximately last 6 months), short_term (approximately last 4 weeks).

At the moment, I am manually streaming some songs on the test account in order to generate some data that the API will return. Recently, streams with a short_term time range started failing because I hadn't done this in a while - I imagine leaving it longer would have meant medium_term and evenutally long_term would have also failed. For the tests, it would be better to warn about no records for these streams rather than fail, as it gives a false impression of the tap being broken at a glance.

Affected streams: https://github.com/Matatika/tap-spotify/blob/1568027dd73435a61650141b6f0d99867cf9bc27/tap_spotify/tap.py#L10-L15

ReubenFrankel commented 6 months ago

Looks like ignore_no_records_for_streams is exactly what we want.

It will warn on no records rather than completely ignore: https://github.com/meltano/sdk/blob/7e92a28b5d0142c3b83a539b49b05d8a43656e2d/singer_sdk/testing/tap_tests.py#L82-L87