grafana / oncall

Developer-friendly incident response with brilliant Slack integration
GNU Affero General Public License v3.0
3.46k stars 279 forks source link

mobile: On-call now displayed even though I'm not On-call - ended an hour ago #2785

Closed Dieterbe closed 1 year ago

Dieterbe commented 1 year ago

What went wrong?

image

this feedback was reported by @marefr Marcus, thank you for submitting the report.

couple ideas: 1) is it possible your phone somehow lost internet connection and was no longer able to sync? 2) is it possible that you were no longer on call because a shift, or part of a shift of yours was overridden by somebody else? 3) were any schedule changes made shortly before this issue happening?

thanks!

How do we reproduce it?

..

Grafana OnCall Version

latest

Product Area

Mobile App

Grafana OnCall Platform?

I use Grafana Cloud

User's Browser?

No response

Anything else to add?

No response

marefr commented 1 year ago
  1. is it possible your phone somehow lost internet connection and was no longer able to sync?

I'm pretty sure it didn't. I restarted the app several times to double check.

  1. is it possible that you were no longer on call because a shift, or part of a shift of yours was overridden by somebody else?

I was on-call 03-15 UTC for Grafana Backend Primary and that was via an override created weeks ago. I was supposed to be On-call for Grafana-Plugins-Platform-Triage, but Timur replaced me with him early on the morning of August 7th (we don't have any override there).

  1. were any schedule changes made shortly before this issue happening?

As said in 2) there was a change on the morning of August 7th. As can be seen in the screenshot each schedule shows other people than me being on-call.

Dieterbe commented 1 year ago

full screenshot here. indeed, it shows only other people image

Dieterbe commented 1 year ago

So for the "on-call now" function, we query this every 20 seconds:

    final response = await httpClient.get<List?>(
      'api/internal/v1/users/$userId/upcoming_shifts',
      queryParameters: {'days': 31},
    );

we get 0-N shifts back, and if any of them has the "is_oncall" set to true, we set the label as "on-call now"

I would have loved to see a screenshot of the "personal current and next on call shifts" page (which you get when you tap on the header of the page), because it displays this shift data.

but at least we do have a screenshot of the schedules page here. the data shown here is fetched with:

    final response = await httpClient.get<Map<String, Object?>>(
      'api/internal/v1/schedules',
      queryParameters: {
        'page': 1,
        'mine': filter?.mine == true,
        'perpage': limit,
      },
    );

@matiasb any idea if there can be an inconsistency between these two endpoints? please check the details that marcus provided around the overrides that were scheduled.

Dieterbe commented 1 year ago

but Timur replaced me with him early on the morning of August 7th (we don't have any override there).

for the record, marcus submitted the image Aug 7, 2023 at 6:00 PM european time, so i suppose about 6-9 hours after the override was scheduled

matiasb commented 1 year ago

@matiasb any idea if there can be an inconsistency between these two endpoints? please check the details that marcus provided around the overrides that were scheduled.

I took a look and this seems related to an issue we got fixed last week. For a particular scenario combining updates to an iCal imported schedule and the way we generate the final schedule cache (used when checking your upcoming shifts), we were still considering cancelled events in the cache.

This should have been fixed and deployed.

Dieterbe commented 1 year ago

for the record:

@marefr looks like this should be fixed. if you see something like this again, please report it again (and ideally provide a screenshot of your personal on call view, which you can view by tapping on the header of the alert groups feed page or schedules page) , thank you.

thanks @marefr for reporting and @matiasb for fixing :)

marefr commented 1 year ago

Thanks :pray: