gl.mathhub.info reorganization: getting rid of MiKo{MH,Problems,Courses}

[kohlhase]

I want to get rid of https://gl.mathhub.info/MiKo{MH,Problems,Courses}, they were good when my courses were the only thing around, but now they are a liability. I have already moved out my old courses to https://gl.mathhub.info/courses/Jacobs and https://gl.mathhub.info/courses/FAU/{LBS,KRMT} and now I want to do the same with the rest (most importantly AI and IWGS which are in current use. The rest (bits and pieces) will follow when I have time.

The target structure for active courses is the following (here for AI):

• the group https://gl.mathhub.info/courses/FAU/AI
  • the public archive course with all the slides.
  • the public archive problems with all the homework (subdir source/prob) and quiz problems (subdir source quiz)
  • the public archive hwexam with all the (old) homework assignments and exams organized by semester
  • the private repository (not an archive) grades for all the grade information.
  • the private forks problems-private and hwexam-private which extend the contents of the public archives problems and hwexam with the upcoming problems and exams. These forks have a much restricted visibility (that's their raison d'être). The instructors should clone them in place of the public variants (so that the paths/URIs in META-INF/MANIFEST.MF work). Then the private problems/exams can be published by a pull request upstream.

Less active courses have a subset of this setup. This seems to be working well for LBS and KRMT so far. For the reorganization we have to

• [x] move the repositories on gl.mathhub.info
• [x] fix up all the relative paths in the sTeX source code accordingly and test/fix on my local MathHub.
• [x] archive MiKo{MH,Problems,Courses} (see below)
• [x] move all the grade information from the courses/FAU/*/hwexam archives into their sister grades repository.
• [x] remove all grade information from the history of courses/FAU/*/hwexam, with the git-filter-repo tool (see comment below).
• [x] commit the whole change and run the build system and fix all the errors there.
• [x] give all the necessary people access to the moved repositories (inherited rights are gone now).
• [x] Make all the corresponding URI updates in the LMS and the BoF here.
• [x] migrate to production.
• [ ] run a linter over the whole web site.

While we are doing all of this the system is in a limbo. So I will try to get this going as fast as possible.

I will keep track of the replacements here (all under the prefix https://gl.mathhub.info/

MiKoMH/{AI,IWGS} --> courses/FAU/{AI,IWGS}/course
MiKoProblems/{AI,IWGS} --> courses/FAU/{AI,IWGS}/problems
MiKoCourses/{AI,IWGS} --> courses/FAU/{AI,IWGS}/hwexam
courses/FAU/{AI,KRMT,IWGS,LBS}/problems/*/mod --> courses/FAU/{AI,KRMT,IWGS,LBS}/problems/*/prob
MiKoCourses/AIquiz --manually merged into--> courses/FAU/AI/problems (a bit messy)
MiKoMH/talks --> mkohlhase/talks
MiKoMH/GenCS --> courses/Jacobs/GenCS/course
MiKoMH/GenICT --> courses/Jacobs/GenICT/course
MiKoMH/TDM --> courses/Jacobs/TDM/course
MiKoMH/ComSem --> courses/Jacobs/ComSem
MiKoMH/CompLog --> courses/Jacobs/CompLog
MiKoMH/ACS --> courses/Jacobs/ACS/course

I do not think we need to do anything about the problem URIs, right?

[Jazzpirate]

• [ ] also replace the archive names in all metadata etc, e.g. MANIFEST.MF-files (dependencies), the course json files,... what else?

[kohlhase]

So, here is the summary of the files that should be on the build server:

• [x] courses/Jacobs/ACS/{course,problems}
• [x] courses/Jacobs/GenCS/{course,problems}
• [x] courses/Jacobs/GenICT/{course,hwexam,problems}
• [x] courses/Jacobs/TDM/{course,hwexam,/problems}
• [x] courses/Jacobs/ComSem
• [x] courses/Jacobs/CompLog
• [x] courses/FAU/AI/{course,hwexam,problems}
• [x] courses/FAU/GDP/{course,problems}
• [x] courses/FAU/IWGS/{course,hwexam,problems}
• [x] courses/FAU/KRMT/{course,hwexam,problems}
• [x] courses/FAU/LBS/{course,hwexam,problems}
• [x] courses/FAU/{AISysProj,EiDA,meta-inf,TheoCS}
• [x] problems/{meta-inf,smglom,maai-test}
• [x] MiKoMH/{KWARC,talks,meta-inf}
• [x] MiKoProblems/{AISysProj,KWARC,meta-inf}
• [x] MiKoCourses/{AISysProj,meta-inf,AISysProjAssignments}

[kohlhase]

And another batch of build server maintenance things:

• [x] remove the whole group MathHub/exams
• [x] courses/Jacobs/ACS is messed up please completely delete it and clone courses/Jacobs/ACS/{course,problems}.
• [x] MiKoMH, MiKoCourses, und MiKoProblems löschen

[kohlhase]

I figured out how to use git-filter-repo to remove the *.xlsx and *.odsfiles from history of hwexam and deposit them in a new repository grades.

1. make sure that the private fork courses/FAU/LBS/hwexam-private of courses/FAU/LBS/hwexam are synchronized.
2. make a new clone of the archive with e.g. git clone git@gl.mathhub.info:courses/FAU/LBS/hwexam.git in ~/tmp/hwexam.
3. make a new public repository hwexam-new on GitLab, but do not clone it yet, and unprotect the main branch: https://gl.mathhub.info/courses/FAU/LBS/hwexam-new/-/settings/repository#js-protected-branches-settings
4. cd ~/tmp/hwexam (this will only work on a new clone) and then
   • git filter-repo --invert-paths --path-glob '*/*.xlsx' --path-glob '*/*.ods' --path-glob '*/*.csv' (this keeps all files but (cf. --invert-paths) the spreadsheets)
   • git remote add new git@gl.mathhub.info:courses/FAU/LBS/hwexam-new.git
   • git push new main
   • check that everything but the spreadsheets is there on https://gl.mathhub.info/courses/FAU/LBS/hwexam-new`
   • if so, rm -Rf ~/tmp/hwexam
5. make a new repository courses/FAU/LBS/grades, clone it in the right place, and unprotect the main branch: https://gl.mathhub.info/courses/FAU/LBS/grades/-/settings/repository#js-protected-branches-settings
6. re-clone the archive git clone git@gl.mathhub.info:courses/FAU/LBS/hwexam.git in ~/tmp/hwexam.
7. cd ~/tmp/hwexam (this will only work on a new clone) and then
   • git filter-repo --path-glob '*/*.xlsx' --path-glob '*/*.ods' --path-glob '*/*.csv' (this keeps only the spreadsheets)
   • git remote add new git@gl.mathhub.info:courses/FAU/LBS/grades.git
   • git push new main
   • check that the spreadsheets are there on https://gl.mathhub.info/courses/FAU/LBS/grades`
   • if so, rm -Rf ~/tmp/hwexam
8. manually delete the hwexam and the hwexam-private repositories (in gitlab; see https://gl.mathhub.info/courses/FAU/KRMT/hwexam/edit --> General --> Advanced --> delete project)
9. in gitlab rename courses/FAU/LBS/hwexam-new to hwexam and make it public
   • in Settings --> General --> Naming, topics, avatar (do not forget to save)
   • in Settings --> General --> Visibility set to public (do not forget to save)
   • in Settings --> General --> Advanced --> Change Path
10. in gitlab make a new private fork courses/FAU/LBS/hwexam-private of hwexam.