Thoughts on generate this data

[sarcasticadmin]

Description

@dlangille following up from our conversation earlier today: https://twitter.com/sarcasticadmin/status/1278075104417603584

I know in your initial examples you have the commit hash that youll be updating each time you run this script. Wanted to also point out that you could alternatively leverage some of the built-in functionality in git to do this without having to embedded a commit SHA in the code.

Below is a script for updating freesbd-ports. It assumes that the repo is already cloned down although that could be handled here too if desired. But the idea is that since you have a local copy of the remote master branch and a local master you can diff them to see where you last left on the git log to xml conversion and then just run the conversion as you typically would.

git-delta.sh:

# Assumes you already cloned down the repo once
# Leverage the fact that we already have the new starting
# point since we know the position of the local master branch
# the last time we ran this

REMOTE='origin'

# Update local copies of remote branches
git fetch $REMOTE

# Make sure branch is clean and on master
git reset --hard HEAD
git checkout master

# Get the first commit for our starting point
STARTPOINT=$(git log master..$REMOTE/master --oneline --reverse | head -n 1 | cut -d' ' -f1)

# Bring local branch up-to-date with the local remote
git rebase $REMOTE/master

# Call xml conversion with starting point
#python3 git-to-freshports.py -c $STARTPOINT
echo "Would have ran git-to-freshport.py starting at: $STARTPOINT"

Example

The following is an example I had from a local ports tree. I have git-delta.sh in the same dir as the rest of the ports tree. Set ports tree master so some old commit. This commits from Jan:

$ git checkout master && git reset --hard d3b56095dd0e

Run the delta:

$ sh git-delta.sh  
HEAD is now at d3b56095dd0e Update to 0.11.2.
Already on 'master'
Your branch is behind 'origin/master' by 47065 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)
Successfully rebased and updated refs/heads/master.
Would have ran git-to-freshport.py starting at: d475beb9e713

Looking at d475beb9e713 its the next commit in the tree after d3b56095dd0e:

commit d475beb9e7130c8bc2ccf4108229932285f52f84
Author: jbeich <jbeich@FreeBSD.org>
Date:   Wed Jan 30 05:28:56 2019 +0000

    multimedia/libva-intel-media-driver: update to 18.4.0 (release)

    Changes:        https://github.com/intel/media-driver/compare/7e966d06...intel-media-18.4.0
    Changes:        https://github.com/intel/media-driver/releases/tag/intel-media-18.4.0
    MFH:            2019Q1 (stabilization)

Hope this is helpful!

[dlangille]

My test run:

[freshports@devgit-ingress01 ~/ports-jail/var/db/repos/PORTS-head-git]$ git fetch origin
remote: Enumerating objects: 284, done.
remote: Counting objects: 100% (284/284), done.
remote: Compressing objects: 100% (153/153), done.
remote: Total 300 (delta 147), reused 268 (delta 131), pack-reused 16
Receiving objects: 100% (300/300), 601.62 KiB | 7.16 MiB/s, done.
Resolving deltas: 100% (147/147), completed with 69 local objects.
From https://github.com/freebsd/freebsd-ports
   008e30ab98fb..51d2aed475f4  master          -> origin/master
   f412caa04aa1..07bb2c7e524a  branches/2020Q2 -> origin/branches/2020Q2
   a70ca3962b88..94111c0e989c  svn_head        -> origin/svn_head

$ STARTPOINT=$(git log master..origin/master --oneline --reverse | head -n 1 | cut -d' ' -f1)

$ echo $STARTPOINT
01347222477d

$ git rebase origin/master
Successfully rebased and updated refs/heads/master.

The next test, is the revlist:

$ git rev-list $STARTPOINT..HEAD
51d2aed475f44b3835a3c180266636257f9c7037
944d2f0c310bc89399f00aca0a1b635cb1d41c6d
83a5c9b40d076bb9562ee48d748eb8d3e22c5d81
70ede099c6a26bf0f19c0e816aa9d8bbd8b1b961
3dc471ded667ab45e7be0314b1cf6db44cc901b3
373c576ab3099536a3e0fce79bf73ac4c00fa1ba
385e1ace7c5fab291f2590ccd934b4fb7c76fa06
f4fc414bacd7980e7dd6b3d9e68dd8e211c2e046
d21e632df7c2517268875d349e5a7966e78a3efd
e3c8eb9ccab4bf3d11ac83ad42d559b332f51f6c
112b8bf9408d27a7cae294620649fde510e650bd
2f2a0769a4470d1acb8afe33583ed876f3864521
0e07387774643c088ede7a6b4026a6b997dd92a7
27d1aaca54cb95bafa07a24729d04ba0e3df8dd7
738cc68d6faca71656ed04ae4bd2eca9bac8b6ae
6e2f80f27c0a9df22740d4dccf364eec97f0d974
d84306dddb0ef1bb2c248b7816b9e4aa10caa573
1e038b0fb77d6ab3f961c292bc029404c4c8ec2f
31c682c7046cca03cce6a3d9693d6b66a548ba86
a43594e02bf66ea831f3f6007a6af849e0e38557
abedb312dc01495d6e51cc1e061a1168e2751226
2b9459769919758413beea86bda9d1f67fa48752
1ffec389b40b928d1699c9a277a64ffeb62730c4
e0284c5c3e092606af58489042035aa50d324f8f
348e5320ff8048ca88a2651927d0a299ea278a42
4bd67a5b7556e4228aa02e68bb5ed0fdb171d298
4654270415f661c68dc2a71958115af6b576ca7c
f2c790d8d52aad3d7aac3021637d308dc47f9a52
e6b3f87f95478ffb64692d1b8a3ff0ffc1b89079
3c6614cb0266bd8336bd7e7c241b1e717fcc58d4

Thanks.

BTW, you mentioned "without having to embedded a commit SHA in the code " - git-show-commit.sh was a proof of concept to verify iteration could happen. When expanded, it would not have had a hardcoded hash.

With your example, I see how we can use master and $REMOTE/master to leap-frog ahead.

Thank you.

[sarcasticadmin]

@dlangille awesome, happy to help!

BTW, you mentioned "without having to embedded a commit SHA in the code " - git-show-commit.sh was a proof of concept to verify iteration could happen. When expanded, it would not have had a hardcoded hash.

Ya, understood. I just wanted to point out that the repo has all the info you need.

Thanks for all the work you've done with Freshports, I use it daily!

[dlangille]

For testing purposes, how can I reset this? I could use zfs snapshot/rollback but is there a way to reset my working copy repo back so I can do the same thing again?

[sarcasticadmin]

Absolutely @dlangille you can the following:

# Make sure we are on the master branch
git checkout master
# Go back to a specific hash
git reset --hard COMMITHASH

Just substitute the COMMITHASH for the hash you originally started from.

[dlangille]

Confirmed. Just did this. Thanks.

[dlangille]

@sarcasticadmin Thank you for your help. I appreciate it.