Size limits for commits #5

Open evanbiederstedt opened 4 years ago

I was able to commit a ~700MB Loom file: http://celltype.info/git/evanbiederstedt/test_files/src/branch/master/Aerts_Fly_AdultBrain_Filtered_57k.loom

~Check what the upper limit is, if there is one~

https://github.com/go-gitea/gitea/blob/master/custom/conf/app.example.ini#L77-L87

Revise app.ini here. Should be max file size 30MB to begin with, I think.

The idea above isn't correct. It doesn't work locally for us on apache.ini

I guess we could set this up with a prereceive hook? https://gist.github.com/nauar/826f85d25d692d9bc009312cb71577dd

But then: (1) I don't know how this works with lfs (2) Not sure how this would applies to all repos on the server, i.e. in gitea-repositories.

e.g.

#!/bin/bash 

GITCMD="git"
NULLSHA="0000000000000000000000000000000000000000"
EMPTYTREESHA=$($GITCMD hash-object -t tree /dev/null) # SHA1: "4b825dc642cb6eb9a060e54bf8d69288fbee4904"
MAXSIZE="50"
MAXBYTES=$(( $MAXSIZE * 1048576 ))
EXIT=0
PRIVATELOGFILE="/tmp/git_private.log"

function private_log() {
    moment=`date '+%d/%m/%Y %H:%M:%S'` 
    echo "[ $moment ] [ POLICY CHECK ] $1" >> $PRIVATELOGFILE
}

function log() {
    moment=`date '+%d/%m/%Y %H:%M:%S'` 
    echo "[ $moment ] [ POLICY CHECK ] $1"
}

log "Starting validation..." 
while read oldref newref refname; do

    private_log "OLDREF: $oldref NEWREF: $newref REFNAME: $refname"

    # Avoid removed branches
    if [ "${newref}" = "${NULLSHA}" ]; then
        continue
    fi

    # Set oldref properly if this is branch creation.
    if [ "${oldref}" = "${NULLSHA}" ]; then
        oldref=$EMPTYTREESHA
    fi

    # Ignore case
    shopt -s nocaseglob

    newFiles=$($GITCMD diff --stat --name-only --diff-filter=ACMRT ${oldref}..${newref})

    if [[ $? -ne 0 ]]; then 
        log "Error 101: Repository incosistency. Cancelling push..."
        exit 1;
    fi

    old_IFS=$IFS
    IFS=''    
    for filename in $newFiles; do
        private_log "Filename: $filename"
        filesize=$($GITCMD cat-file -s "${newref}:${filename}") 2> $PRIVATELOGFILE

        if [[ -z $filesize  ]]; then filesize=0; fi
        filesize_mb=$(($filesize / 1048576))

        if [ "${filesize}" -gt "${MAXBYTES}" ]; then
            log "File $filename is greater than $MAXSIZE MB. Its size is $filesize_mb MB."  
            exit 1
        fi

    done
    IFS=$old_IFS
done%

Gitlab offers server side hooks: https://docs.gitlab.com/ee/administration/server_hooks.html

Explore how this works for Gitea

Update:

Ok, this is useful: https://coderwall.com/p/jp7d5q/create-a-global-git-commit-hook

I think this should work: https://coderwall.com/p/jp7d5q/create-a-global-git-commit-hook

In principle, if we set up this up, the hooks will be automatically copied to each new repo created

I need to try this out.

kharchenkolab / gitea

Size limits for commits #5