sindresorhus / gulp-changed

Only pass through changed files
MIT License
742 stars 44 forks source link

hasChanged function using Git log #80

Open CraigChamberlain opened 4 years ago

CraigChamberlain commented 4 years ago

I guess this is a feature request but I'll try and implement myself. Want to document use case and my progress, see if anyone else has ideas, suggestions or is interested.

If I store binary files, such as images in Git LFS. When I checkout the files they all get a modified/created date of the time the files are checked out.

Two solutions:

  1. Update the timestamp.
    git ls-tree -r --name-only HEAD | while read filename; do 
    unixtime=$(git log -1 --format="%at" -- "${filename}")
    touchtime=$(date -d @$unixtime +'%Y%m%d%H%M.%S')
    touch -t ${touchtime} "${filename}"
    done
  2. Use gulp-git in git-changed.

I think the second option might be worth perusing and profiling for relative performance? There is potential for half the read I/O none of the write I/O.

CraigChamberlain commented 4 years ago

Early indications suggest the following will work pretty quickly. If a file exists in the destination but has not yet been committed it is forwarded for processing again.

gulp-git does not support git log but gulp-exec might be a better match anyway.

const {promisify} = require('util');
const fileSystem = require('fs')
const stat = promisify(fileSystem.stat);
const exec = require('child_process').exec;
const execProm = promisify(exec);

async function getTimeStamp(filename){
  let result;
  try{
    result = (await execProm(`git log -1 --format="%at" -- "${filename}"`)).stdout.trim()

  } catch(ex){
    result = ex;
  }

  if ( Error[Symbol.hasInstance](result) )
       return ;

  return result
}

async function compareLastModifiedTimeGit(stream,sourceFile, targetPath) {
        if(targetPath == "/home/craig/Documents/src/a-tent-with-a-view-LFS/processedimages/images/02-horombo-hut-marangu-route-3 copy.jpg"){
          var x = "sdfsdf";
        }
        const targetStat = await stat(targetPath);
        if (targetStat === undefined) {
            stream.push(sourceFile);
        } else {
            var sourceMTime = await getTimeStamp(sourceFile.path)
            var targetMTime = await getTimeStamp(targetPath)

            if (sourceMTime > targetMTime) {
                stream.push(sourceFile);
            }
        }
};
CraigChamberlain commented 4 years ago

Together with another function to selectively pull the requested lfs files, I have a pretty usable solution. It's certainly not gulp ready and is probably pretty ugly code but it's a proof of concept if anyone has an interest contributing to make if more idiomatic or otherwise more fit for production.

async function pullFile(filename) {
  await execProm(`git lfs fetch -I "${filename}"`)

};