Flex: detect empty/broken media repositories

[a-teammate]

Problem 1

@terencode and @obstriegel had the issue when installing inexor that the media folders where there but not correct git repos and empty. We should detect that.

Problem 2

(copied from #434)

Right now when a media repository is not properly initialized the whole thing will blow up with:

Jun 20 19:52:04 @inexor-game/flex/server[28817] INFO:  Cannot read property 'substr' of undefined
err: TypeError: Cannot read property 'substr' of undefined
at GitRepositoryManager.mergeBranches (./node_modules/@inexor-game/media/Repository.js:513:29)
at ./node_modules/@inexor-game/media/Repository.js:459:23
at tryCallOne (./node_modules/nodegit-promise/lib/core.js:37:12)
at ./node_modules/nodegit-promise/lib/core.js:103:15
at flush (./node_modules/asap/raw.js:50:29)
at _combinedTickCallback (internal/process/next_tick.js:73:7)
at process._tickCallback (internal/process/next_tick.js:104:9)

It would be good to also spawn the git cloning in a dedicated process, so it will not be killed when flex is killed.

Tasks

• [x] detect empty/broken media repositories
• [x] detect failed clone/update
• [x] check integrity of a media repository
• [x] restart (or continue) clone/update

[aschaeffer]

Probably because the cloning the repository has started but not completed. The folder may have been created but is empty.

A solution would be to check if repository.json or repository.toml exists.

[Fohlen]

It will probably be useful to implement a pipeline library (small) to do the following tasks for us

• download media repositories
• download core releases

the pipeline should preferably spawn a dedicated, new process (shell/bash/dos) and launch the download there, so in any case the download operation is continued. For the validity part, the following can be used

• https://git-scm.com/docs/git-fsck for git repos
• the checksums of the zip files for the core zip files

[aschaeffer]

1. Prevent crashes

A first effective and simple solution is already done: In https://github.com/inexorgame/inexor-flex/commit/51dc8081177529800ce13ce3a441db943e8c035e we prevent Flex from exiting cause of unhandled exceptions. This way the stability has been improved a lot. The remaining unwanted exits are segfaults in native nodejs extension (for example in grpc).

2. Prevent exit during updates or other transactional tasks

An additional task is to block Flex from exiting if there are transactional tasks (like git clone/git pull) still running. We could add a service for this.

3. Better use an integrated API instead of external programs

There's a little problem with Node.js and external tasks: In order to get any response from the external task you have to connect to it's stdout. But with stdout connected and Flex exits or crashs the external child process is also terminated. Without stdout connected there's a black box running but at least the external process won't be terminated.

Also you run into more problems:

• git as end user dependency. Now we are using Nodegit which uses or bundles the native libgit2 library.
• We can't profit anymore from the tight integration of the API provided by nodegit: http://www.nodegit.org/api/.
• I really don't want to parse the console output of git commands. It's a huge amount of unnecessary and stupid work. Also, the console output of git commands changes with new versions.

[aschaeffer]

Since this is implemented in Inexor Flex but not tested by others than myself, please give me feedback.

1. Delete your media repository
2. Start Flex with the dev profile to get verbose logging
3. Kill Flex while cloning a repository for the first time
4. Restart Flex with the dev profile
5. Check if it result in a successfully cloned repository after a while

[Fohlen]

I can verify that this works

[Croydon]

@a-teammate Does this only DETECT corrupt media repository or also remove/reset corrupt media repositories?