Split the parameter files into an independent repo?

[paulmking]

For maintainability reasons, it may be a good idea to import all the parameter files into a separate repo, and than possibly include that as a dependency. If we don't include that as a dependency, we should leave a few example parameter files in this one.

We should then make sure there's a command line argument to define the parameter file path. Perhaps if the "config" argument contains a path, it would be included as the first search path for all other parameter files?

Rationale: the parameter files change much more often than the actual source code, and so their changes can clutter the real source file changes. Also, we could easily make branches or forks representing different experiments (PREX vs. MOLLER, for example).

[wdconinc]

This is a possible use case for git submodule, https://git-scm.com/book/en/v2/Git-Tools-Submodules. The prminput directory would be its own repository (e.g. japan-prminput-prexcrex) which is included as a submodule at Parity/prminput for installations. That submodule can point to the repository of the current commonly used experiment (i.e. prex/crex now). Export of the current content of prminput, with history, into a new repository is possible (if not trivial).

[paulmking]

I was wondering about using a submodule, but thought I understood that it would require an extra step for users to checkout a working copy. I didn't read enough to understand the trade-offs.

Although, if we just had a separate repo (not a submodule) a user still needs to do a separate checkouot and then needs to set up their directories correctly, so a submodule is likely better than a fully independent repo.

[wdconinc]

The first level submodule is transparent upon checkout. Nested submodules is a pain.

Submodules are pinned at a specific commit hash of the included repository. E.g. I think it would go something like this: $ git clone japan -> gets submodule at pinned prminput hash $ git pull Parity/prminput -> updates prminput to its master $ git commit Parity/prminput && git push Parity/prminput -> pushes to prminput $ git submodule update && git push -> updates prminput to point to new hash

Risks: changes in submodule directories are not ever pushed anywhere since git diff in japan/ doesn't show them.

[wdconinc]

Following commands to extract Parity/prminput into https://github.com/JeffersonLab/japan-prminput-prexcrex as a proof of concept (probably doesn't need all branches)

# Clone, no tags, track all branches
git clone --no-tags git@github.com:JeffersonLab/japan japan-prminput-prexcrex
cd japan-prminput-prexcrex
for BRANCH in $(git branch -a | grep remotes | grep -v HEAD | grep -v master); do
  git branch --track "${BRANCH#remotes/origin/}" "${BRANCH}"
done
# Remove origin to prevent errors
git remote rm origin

# Filter out directories
git filter-branch --prune-empty -f --index-filter 'git rm --cached --ignore-unmatch -r bin cmake Doxygen evio rootScripts SetupFiles Tests' -- --all

# Filter out files
git filter-branch --prune-empty -f --index-filter 'git rm --cached --ignore-unmatch -r CMakeLists.txt Dockerfile Doxyfile PackageInfo.dox README.md' -- --all
git filter-branch --prune-empty -f --index-filter 'git rm --cached --ignore-unmatch -r qweak-config.in INSTALL .gitignore GNUmakefile' -- --all

# Filter out directories
git filter-branch --prune-empty -f --index-filter 'git rm --cached --ignore-unmatch -r Analysis Parity/include Parity/main Parity/src' -- --all

# Move prminput into root
git filter-branch --prune-empty -f --tree-filter '
  git ls-tree --name-only $GIT_COMMIT Parity/prminput/ | xargs -I files mv files .
' -- --all
# Remove Parity/prminput
git filter-branch --prune-empty -f --index-filter 'git rm --cached --ignore-unmatch -r Parity/prminput' -- --all

# Apply dos2unix
git filter-branch --prune-empty -f --tree-filter 'git ls-files -z | xargs -0 dos2unix' -- --all

# Apply chmod a-x
git filter-branch --prune-empty -f --tree-filter 'git ls-files -z | xargs -0 chmod a-x' -- --all

# Remove backup heads
git for-each-ref --format="%(refname)" refs/original/ | xargs -r -n 1 git update-ref -d

# Garbage clean
git gc --aggressive
git prune

# Garbage clean
git gc --aggressive
git prune

# Connect to new remote
git remote add origin git@github.com:JeffersonLab/japan-prminput-prexcrex.git

# Push all to upstream
git push -u origin --all

[wdconinc]

Proof of concept on submodule at https://github.com/JeffersonLab/japan/tree/submodule-prminput.