Snippet for tracking accross Renames ie. log --follow

[AlphaJuliettOmega]

I'm dying trying to figure this one out, I'm going to share my progress, just in case it helps you make a snippet for it - it's very difficult for me to keep track of which methods/datatypes to use with the jGit Api

Please excuse the doubling of this function, I made a second function (overloading for second parameter is almost the only difference) to try to walk through commits related to the file i'm targeting but after a file is moved, its history is still not tracked, I feel i've taken it too far but I can't stop now

` package org.xxx.xxx.common; import java.util.ArrayList; import java.io.ByteArrayOutputStream; import java.io.File; import java.io.FileNotFoundException; import java.io.IOException; import java.io.InputStream; import java.io.PrintWriter; import java.sql.Timestamp; import java.util.List; import java.util.logging.Level; import java.util.logging.Logger; import org.adeptnet.guardian.ex.GitException; import org.apache.commons.io.FileUtils; import org.eclipse.jgit.api.Git; import org.eclipse.jgit.api.errors.GitAPIException; import org.eclipse.jgit.diff.DiffEntry; import org.eclipse.jgit.diff.DiffFormatter; import org.eclipse.jgit.diff.RenameDetector; import org.eclipse.jgit.errors.CorruptObjectException; import org.eclipse.jgit.errors.IncorrectObjectTypeException; import org.eclipse.jgit.errors.NoWorkTreeException; import org.eclipse.jgit.errors.RevisionSyntaxException; import org.eclipse.jgit.lib.ObjectId; import org.eclipse.jgit.lib.ObjectLoader; import org.eclipse.jgit.lib.ObjectReader; import org.eclipse.jgit.lib.Repository; import org.eclipse.jgit.revwalk.RevCommit; import org.eclipse.jgit.revwalk.RevTree; import org.eclipse.jgit.revwalk.RevWalk; import org.eclipse.jgit.storage.file.FileRepositoryBuilder; import org.eclipse.jgit.treewalk.CanonicalTreeParser; import org.eclipse.jgit.treewalk.TreeWalk; import org.eclipse.jgit.treewalk.FileTreeIterator; import org.eclipse.jgit.treewalk.filter.AndTreeFilter; import org.eclipse.jgit.treewalk.filter.PathFilter; import org.eclipse.jgit.treewalk.filter.PathFilterGroup; import org.eclipse.jgit.treewalk.filter.TreeFilter; import org.gitective.core.CommitUtils;

public List<FileVersion> getHistoryByFilename(String dir) throws FileException {
    List<RevCommit> commitsList = new ArrayList<>();
    List<FileVersion> fileList = new ArrayList<>();
    try {
        LOG.log(Level.INFO, dir);
        RevWalk revWalk = new RevWalk(repo);
        revWalk.setTreeFilter(
                AndTreeFilter.create(
                        //a possibility could be to add the 'old path' here and get those commits too
                        PathFilterGroup.createFromStrings(dir),
                        TreeFilter.ALL
                )
        );
        LogFollowCommand betterLogCommand = new LogFollowCommand(repo, dir);
        ArrayList<String> oldPaths = betterLogCommand.getOldPaths();
        oldPaths.forEach((t) -> {
            try {
                fileList.addAll(getHistoryByFilename(t, fileList));
            } catch (FileException ex) {
                LOG.log(Level.WARNING, "Epic fail on inner rename recursion loop", ex);
            }
        });

        for (RevCommit revCommit : betterLogCommand.call()) {
            LOG.log(Level.INFO, revCommit.getFullMessage());

            RevTree tree = revCommit.getTree();
            try (TreeWalk treeWalk = new TreeWalk(repo)) {
                treeWalk.addTree(tree);
                treeWalk.setRecursive(true);
                treeWalk.setFilter(PathFilter.create(dir)); 

                while (treeWalk.next()) {

                    ObjectId objectId = treeWalk.getObjectId(0); 
                    ObjectLoader loader = repo.open(objectId);
                    InputStream is = loader.openStream();
                    FileVersion pew = new FileVersion(convertStreamToString(is), String.valueOf(loader.getType()), new Timestamp(revCommit.getCommitTime() * 1000L));
                    fileList.add(pew);
                    // LOG.info("found a file at " + treeWalk.getPathString() + ": with commit type of" + tree.);
                    is.close();
                }
            }
        }
    } catch (IncorrectObjectTypeException ex) {
        LOG.log(Level.SEVERE, "Found a broken object while attempting to find history of File ", ex);
    } catch (IOException | GitAPIException ex) {
        LOG.log(Level.SEVERE, "Failed to read from disk or jGit broke", ex);
    }
    fileList.sort((o1, o2) -> {
        return o2.getVersionDate().compareTo(o1.getVersionDate());
    });
    return fileList;
}

public List<FileVersion> getHistoryByFilename(String dir, List<FileVersion> fileList) throws FileException {

    try {

        RevWalk revWalk = new RevWalk(repo);
        revWalk.setTreeFilter(
                AndTreeFilter.create(
                        //a possibility could be to add the 'old path' here and get those commits too
                        PathFilterGroup.createFromStrings(dir),
                        TreeFilter.ALL
                )
        );
        //+"^1"

        LogFollowCommand betterLogCommand = new LogFollowCommand(repo, dir);
        ArrayList<String> oldPaths = betterLogCommand.getOldPaths();
        oldPaths.forEach((t) -> {
            try {
                fileList.addAll(getHistoryByFilename(t, fileList));
            } catch (FileException ex) {
                LOG.log(Level.WARNING, "Epic fail on inner rename recursion loop", ex);
            }
        });
        for (RevCommit revCommit : betterLogCommand.call()) {
            LOG.log(Level.INFO, revCommit.getFullMessage());

            RevTree tree = revCommit.getTree();
            try (TreeWalk treeWalk = new TreeWalk(repo)) {
                treeWalk.addTree(tree);
                treeWalk.setRecursive(true);
                treeWalk.setFilter(PathFilter.create(dir));   //WHY DID I DO THIS

                while (treeWalk.next()) {

                    ObjectId objectId = treeWalk.getObjectId(0);  //WHY DID I DO THIS
                    ObjectLoader loader = repo.open(objectId);

                    InputStream is = loader.openStream();

                    // and then one can the loader to read the file
                    //System.out.println(loader.toString());
                    //loader.copyTo(System.out);
                    FileVersion pew = new FileVersion(convertStreamToString(is), String.valueOf(loader.getType()), new Timestamp(revCommit.getCommitTime() * 1000L));
                    //LOG.info(pew.toString());
                    fileList.add(pew);
                    LOG.info("found a file at " + treeWalk.getPathString() + ": \n" + pew.getConfig());
                    is.close();
                }
            }
        }
    } catch (IncorrectObjectTypeException ex) {
        LOG.log(Level.SEVERE, "Incorrect object type found in Git, search failed.", ex);
    } catch (IOException | GitAPIException ex) {
        LOG.log(Level.SEVERE, "IO failure or Git Exception.", ex);
    }
    return configList;
}

static String convertStreamToString(InputStream is) {
    java.util.Scanner s = new java.util.Scanner(is).useDelimiter("\\A");
    return s.hasNext() ? s.next() : "";
}

`

I tried modifying the LogCommand to track renames for use in the above code: ` package org.xxx.xxx.common;

import org.eclipse.jgit.api.Git; import org.eclipse.jgit.api.errors.GitAPIException; import org.eclipse.jgit.diff.DiffEntry; import org.eclipse.jgit.diff.RenameDetector; import org.eclipse.jgit.errors.MissingObjectException; import org.eclipse.jgit.lib.Repository; import org.eclipse.jgit.revwalk.RevCommit; import org.eclipse.jgit.treewalk.TreeWalk; import org.eclipse.jgit.lib.ObjectLoader;

import java.io.IOException; import java.io.InputStream; import java.util.ArrayList; import java.util.List; import java.util.logging.Logger; import org.eclipse.jgit.treewalk.filter.PathFilter;

/*

• @author AJ found here
• http://stackoverflow.com/questions/11471836/how-to-git-log-follow-path-in-jgit-to-retrieve-the-full-history-includi

• */ /**

• Create a Log command that enables the follow option: git log --follow -- < path
• User: OneWorld Example for usage: ArrayList commits = new

• LogFollowCommand(repo,"src/com/mycompany/myfile.java").call(); */ public class LogFollowCommand {

  private static final Logger LOG = Logger.getLogger(LogFollowCommand.class.getName()); private final Repository repository; private String path; private Git git; private ArrayList oldPaths;

  /**

  • Create a Log command that enables the follow option: git log --follow -- < path
  • @param repository
  • @param path */ public LogFollowCommand(Repository repository, String path) { this.repository = repository; this.path = path; oldPaths = new ArrayList(); }

  /**

  • Returns the result of a git log --follow -- < path >
  • @return
  • @throws IOException
  • @throws MissingObjectException

  • @throws GitAPIException */ public ArrayList call() throws IOException, MissingObjectException, GitAPIException { oldPaths.add(path); ArrayList commits = new ArrayList(); git = new Git(repository); RevCommit start = null; do { Iterable log = git.log().addPath(path).call(); //tip from stackoverflow question, not sure how to implement this //But I think you should also add the start ObjectId (if !=null) to the log command in call(). What now happens is that when the file with the old name is being added again AFTER the rename, it will show up in the log of the new file. for (RevCommit commit : log) { if (commits.contains(commit)) { start = null; } else { start = commit; commits.add(commit); } } if (start == null) { return commits; } } while ((path = getRenamedPath(start)) != null);

    return commits; }

  /**

  • Checks for renames in history of a certain file. Returns null, if no
  • rename was found. Can take some seconds, especially if nothing is
  • found... Here might be some tweaking necessary or the LogFollowCommand
  • must be run in a thread.
  • @param start
  • @return String or null
  • @throws IOException
  • @throws MissingObjectException

  • @throws GitAPIException */ private String getRenamedPath(RevCommit start) throws IOException, MissingObjectException, GitAPIException { Iterable allCommitsLater = git.log().add(start).call(); for (RevCommit commit : allCommitsLater) {

    TreeWalk tw = new TreeWalk(repository);
    //have to split off path to config and only search for filename, or file moves are lost
    tw.setFilter(PathFilter.create(path.split("/")[path.split("/").length - 1]));
    tw.addTree(commit.getTree());
    tw.addTree(start.getTree());
    tw.setRecursive(true);
    RenameDetector rd = new RenameDetector(repository);
    rd.addAll(DiffEntry.scan(tw));
    List<DiffEntry> files = rd.compute();
    for (DiffEntry diffEntry : files) {
    
        if ((diffEntry.getChangeType() == DiffEntry.ChangeType.RENAME || diffEntry.getChangeType() == DiffEntry.ChangeType.COPY) && diffEntry.getNewPath().contains(path)) {
            LOG.info("Found: " + diffEntry.toString() + " return " + diffEntry.getOldPath());
            oldPaths.add(diffEntry.getOldPath());
            ObjectLoader miniloader = repository.open(diffEntry.getId(DiffEntry.Side.OLD).toObjectId());
            InputStream mini_in = miniloader.openStream();
            LOG.info(convertStreamToString(mini_in));
    
            return diffEntry.getOldPath();
        }
    }

    } return null; }

  public ArrayList getOldPaths() {

  return oldPaths;

  }

  static String convertStreamToString(InputStream is) { java.util.Scanner s = new java.util.Scanner(is).useDelimiter("\A"); return s.hasNext() ? s.next() : ""; } } ` and finally, for ease of use, here's the class I created for storing the different versions (to loop over and display on an interface)

` package org.adeptnet.guardian.entity;

import java.sql.Timestamp;

/*

• @author AJ */ public class FileVersion implements FileVersionInterface {

  private final String fileText; private final int userId = 0; private final Timestamp versionDate; private final String changeType;

  public FileVersion(String fileText, String changeType, Timestamp versionDate) { this.fileText= fileText; this.versionDate = versionDate; this.changeType = changeType; }

  @Override public String toString() { return "@" + versionDate.toString() + " file length:" + fileText.length() + " change type: " + changeType; }

  @Override public int getUserId() { return userId; }

  @Override public Timestamp getVersionDate() { return versionDate; }

  @Override public String getFile() { return fileText; }

}

`

I'm very sorry, i'm struggling to get the markdown to work correctly with these code blocks

[centic9]

As stated in the README it is not useful to raise questions here as my time is quite limited. Please post on Stackoverflow and use the JGit-tag, which will cause many more very knowledgeable people to potentially take a look at your problem.

[AlphaJuliettOmega]

I have not found anything useful on Stackoverflow working on this for months. My implementation has been completely re-written twice since this ticket to no avail,

doing a git log --follow with jGit is absolute hell

[AlphaJuliettOmega]

Yeah I'm still stuck on this... I've never regretted using a library so much.

[centic9]

That's unfortunate to hear. However many people seem to be enjoying using jgit for many different tasks, so it seems you hit a specific roadblock here.

Did you try to post your actual problem on stackoverflow? Some of the core jgit developers are listening in there and you are likely to get at least some response.

Otherwise you could try to post a bug-report at eclipse.org if you think that it is an incorrect behavior of the library.