500 error when trying to download student repos

[djmandell]

I've been trying to download my student repos today and I keep getting a 500 error:

2023/09/16 18:56:14 HTTP 500: Server Error (https://api.github.com/assignments/475565/accepted_assignments?page=1&per_page=22)

I initially tried my default (62, which covers all of my students) but kept lowering the per_page number until I found that 20 seems to be a low enough number to allow the download to complete (though it should be noted that I still got a 500 twice trying to download 4 pages). I was able to successfully download all 4 pages in the end, but in an ideal world I would hope that I'd be able to download all 62 in one go.

[shanep]

Hi @djmandell

Can you replicate this issue outside of your Universities firewall? I encountered a similar issue at my University and it turned out that the IT department was throttling connections to github.com over multiple protocols (ssh, https, etc.). Apparently github.com has been flagged by some firewall vendors or configs tools as hosting malware and needing to be blocked. Additionally our University had rules in place to block any IP addresses that had excessive outbound ssh connections. The gh client and git are very chatty apps so they got flagged and we were experiencing similar issues.

We submitted a ticket to IT and they were able to add an exception to the firewalls for github controlled IP addresses and POOF everything magically started working perfectly :)

[djmandell]

@shanep

Thanks for the reply! I was at home when I attempted this, so there wasn't a corporate/institutional firewall to contend with. I've got a gigabit up/down and no firewall rules that should cause an issue. But I will try using this in a few different spots to see if I can isolate the issue further. As I mentioned, if I lower the per_page number enough it works, so it's not just a matter of it getting blocked.

[shanep]

OK this morning I am now seeing the same 500 error both at home AND on campus. I checked with my TA's and they are also reporting this same error in both locations, apparently they have been seeing this for a couple of days and just didn't say anything to me :) So I think it is safe to say based on what @djmandell is seeing and what I and my 3 TA's are seeing this is defiantly not a network issue on our end.

@RyanHecht can this be related to the fix that was done for issue #34 ?

[RyanHecht]

We have an internal timeout for the API call that fetches the submissions, and it seems like sufficiently large pages are hitting that timeout. We're currently looking into it, but for now I would recommend decreasing page size as @djmandell suggested

[a-tk]

Hey everyone,

I can confirm that I see this new 500 error even for --per-page 1, and this is with the latest version of gh (2.35.0)

(outside of university network too).

[oscarlevin]

I'm getting the error on all commands: list, view, assignments, etc.

[jbsooter]

Also receiving the error on all gh classroom actions, and authentication is working for non-gh classroom related actions such as cloning an individual repo.

[a-tk]

Today its working for me if I don't pass the '--per-page' parameter.

Since the paging works, I downloaded all of my repos with --page 1, then --page 2, etc.

Andre

[RyanHecht]

@a-tk @oscarlevin @jbsooter Thanks for the heads up, a webserver configuration issue was causing those! That's since been fixed.

We're still working on a solution for sufficiently large pages/downloading all submissions at once! Appreciate your patience :)

[zrdaley]

https://github.com/github/gh-classroom/pull/37 was merged (Thank you @shanep ✨) which should allow teachers to clone all repos using gh classroom clone student-repos --all

You will need to pull in the latest release (v.0.1.9) first using gh extension upgrade classroom

[girodg]

37 was merged (Thank you @shanep ✨) which should allow teachers to clone all repos using gh classroom clone student-repos --all

You will need to pull in the latest release (v.0.1.9) first using gh extension upgrade classroom

@zrdaley I upgraded to v.0.1.9 and ran that command but I get this failure: 2023/09/28 14:36:13 HTTP 500: Server Error (https://api.github.com/classrooms/26315/assignments?page=1&per_page=100)

It lets me select the classroom but fails before showing the assignments.

[djmandell]

I think you probably don't want to be forcing it to do 100 repos per page, because I think the underlying performance issue still hasn't been solved. If you just run gh classroom clone student-repos --all I think it defaults to 15 repos at a time, which allows it run through and grab all the repos.

[girodg]

That is the command I ran (I didn't specify a page size). I think it is getting hung up on the # of assignments instead of the # of repos.

I ran some other tests to try to list the assignments for my classroom. NOTE: the classroom has 23 assignments. I tried to run this command to list the assignments: gh classroom assignments -c 26315 --per-page 10 --page 1

it works (sometimes). But if I re-rerun with page 2, it only shows 1 more assignment when it should have shown another 10.

Also, a per page larger than 10 never works.

[zrdaley]

@girodg @djmandell we made a large internal performance improvement that should resolve these issues! I was personally able to test on a large classroom and successfully clone 25 repositories after the fix.

Please reopen this issue if you are still having problems.

[girodg]

@zrdaley everything seems to be working great today! thanks!!

[sdziallas]

I'm still encountering this issue (with an up to date version of gh and the classroom extension). We're using repositories for lab submissions and students' final project. The lab repositories seem to clone fine, but the final project ones don't (even when I set --per-page 1). Does this have to do with the size of the repositories and is there a way around this?

[mariolpantunes]

I am still having that issue. Any luck with it?

[frakkerman]

I also still experience this issue, tried the various suggestions in this thread, but without success.

[stevecassidy]

Seeing consistent 500 errors trying to download around 120 assignment submissions. Get nothing at all on Windows. On MacOS I got 90 downloaded but then 500 errors. Updated to latest version of gh classroom.

[frakkerman]

For the time being, I use a workaround, which might be useful for some. First I download the 'gradelist csv' (you can find it somewhere in the assignment dashboard). Next, I use this script to clone all student folders: https://github.com/frakkerman/vrp_competition_github_classroom/blob/main/server/repos/clone_assignment_repos.py