search

pytorch / vision

Datasets, Transforms and Models specific to Computer Vision
https://pytorch.org/vision
BSD 3-Clause "New" or "Revised" License
15.74k stars 6.88k forks source link

Automate closing of scheduled workflows #5290

Open oke-aditya opened 2 years ago

oke-aditya commented 2 years ago

🚀 The feature

We need to currently close the Issues raised by our automated tests for datasets manually.

https://github.com/pytorch/vision/issues?q=is%3Aissue+is%3Aopen+Scheduled+workflow+failed ?

Sometimes they get bunched up or we forget to close them.

Motivation

The cronjob runs daily. So ideally we should be closing the issue the next day if it doesn't fail. (If it does fail anyway another issue is created).

I propose a simple GitHub action that closes only this issue once the issue gets stale by 3 days. The logic is that if the issue is open for 3 days.

  1. Everything is working fine and hence we don't have a problem to close it. (It was one off error)
  2. There is another issue which is newer that supersedes this old issue.

So we should be fine closing it.

Pitch

I think a simple GitHub action can do this job. Would need to investigate but think that technically it's very much possible.

Additional context

I don't propose to extend this functionality to close all stale issues. An issue is well documented user feedback which is very valuable indeed. An old issue can be gem of resource for someone and keeping it open doesn't harm.

But for bot issues :( It isn't nice to be flooded with them being open :(

Some references

https://github.com/marketplace/actions/close-stale-issues

cc @pmeier @seemethere

pmeier commented 2 years ago

This seems like a cool idea. Given that they currently only spamming me, I don't think this is urgent. But if you (or someone else) wants to look into this, I would be happy to review a PR.

NicolasHug commented 2 years ago

Good idea, thanks @oke-aditya

Alternatively, if we have a lot of one-off, we can also decide to only open an issue if the download fails twice (or thrice) in a row

oke-aditya commented 2 years ago

It's actually rare to have lot of one-off. Also, while downloading and testing we do 5 retries I think?

I will work on this over the weekend, It's little interesting topic and been a while since I configured CI for something.

oke-aditya commented 1 year ago

I'm tempted to fix this. I really don't like

image

Lot at my plate in the office and at stuff. Will tackle this next. Especially I felt, sad that these issues annoyed phillip during his well deserved break.

pmeier commented 1 year ago

I think if we determine a server to be down there are two ways forward:

  1. Disable the download test like I did in #6389.
  2. Remove the download functionality from the dataset, which in turn also stops the issues.

My goto strategy is to do 1. first and wait for 2. until the next release. If the server comes back up, we simply reactivate the test. Otherwise we want to disable the download option to avoid more issues coming from users. Of course we don't just need to wait for the authors to fix the issue, but can actively notify them.