buttondown / roadmap

Buttondown's public roadmap
53 stars 0 forks source link

Add better harness for updating `django-perf-rec` snapshots #3358

Closed jmduke closed 1 month ago

jmduke commented 2 months ago

It's kind of annoying to deal with performance snapshot updates b/c django-perf-rec does not expose an "update snapshot" mechanism like pytest-snapshot does.

So let's do three things:

  1. Mark every single test that uses django_perf_rec as @pytest.mark.db_snapshot. (We might be able to DRY this a bit by creating a fixture that applies the mark.
  2. Augment app/update_snapshots to delete all .perf.yml files and then run just files tagged db_snapshot.
    ### Tasks
catdevnull commented 2 months ago

Mark every single test that uses django_perf_rec as @pytest.mark.db_snapshot. (We might be able to DRY this a bit by creating a fixture that applies the mark.

would this be by making a very simple perf_record or django_perf_rec fixture that just passes the record function?

catdevnull commented 2 months ago
FAILED emails/views/subscriber_facing/email--test.py::test_performance_with_a_handful_of_queries_without_billing - TypeError: run_performance_snapshot() missing 1 required positional argument: 'perf_record'
FAILED emails/views/subscriber_facing/subscribe--test.py::test_performance_with_a_handful_of_queries_without_billing - TypeError: run_performance_snapshot() missing 1 required positional argument: 'perf_record'
FAILED emails/views/subscriber_facing/email--test.py::test_performance_with_a_handful_of_queries_on_v2 - TypeError: run_performance_snapshot() missing 1 required positional argument: 'perf_record'

not sure what to do in these seemingly non-pytest tests