Script changes having no effect

[McFateM]

This is baffling... I've forked this repo and modified my ISLE .env file to include:

## ISLE Drupal Build tools:

# Pull build tools from git repo?  bool (true|false), default: true
PULL_ISLE_BUILD_TOOLS=true

# Repo and branch to pull?
ISLE_BUILD_TOOLS_REPO=https://github.com/DigitalGrinnell/ISLE-Drupal-Build-Tools.git
ISLE_BUILD_TOOLS_BRANCH=Digital.Grinnell

So my environment points to my intended repo (the fork) and branch. I've made changes to this repo/branch by adding a new folder, custom.d and contents, as well as changes to isle_islandora_installer.sh. When I spin up a new isle.localdomain and drop into the Apache container I can see the new custom.d folder, but my changes to isle_islandora_installer.sh are nowhere to be seen.

I'm stumped. Anyone have a clue what might cause such a disconnect?

[g7morris]

@McFateM Good question. I think we'll need to do some further testing.

What isn't clear to me re syntax are the confd variable settings within the apache image settings

Specifically Lines 3 -9:

ENV INITRD=no \
    ISLANDORA_UID=${ISLANDORA_UID:-1000} \
    ENABLE_XDEBUG=${ENABLE_XDEBUG:-false} \
    PULL_ISLE_BUILD_TOOLS=${PULL_ISLE_BUILD_TOOLS:-true} \
    ISLE_BUILD_TOOLS_REPO=${ISLE_BUILD_TOOLS_REPO:-https://github.com/Islandora-Collaboration-Group/isle_drupal_build_tools.git} \
    ISLE_BUILD_TOOLS_BRANCH=${ISLE_BUILD_TOOLS_BRANCH:-master}
    ## @TODO: add GH creds to container for private repo pulls.

Not only do the variables give the appearance of being hardcoded there is a TO DO at the end. Me thinks we need to ensure this ticket gets into the next ISLE feature sprint or in the ISLE Phase II - Sprint for "cleanup".

[g7morris]

It is possible the appearance of hardcoding is simply setting ENV values temporarily for (and during) the build process itself but that doesn't explain confd ignoring the new scripts or changes.

[g7morris]

@McFateM I think this appears to be build process only at this point in development but I'm not sure.

When I review here: https://github.com/Islandora-Collaboration-Group/isle-apache/blob/7be53f841b8b8de2f202c60c19a0b6cb3b867ede/rootfs/etc/confd/conf.d/apache-isle-installer.toml

This appears to overwrite settings in the /utility-scripts/isle_drupal_build_tools/isle_islandora_installer.sh file which would seem to overwrite your file as well. Seems like a collision of sorts at first glance.

Additionally though here: https://github.com/Islandora-Collaboration-Group/isle-apache/blob/7be53f841b8b8de2f202c60c19a0b6cb3b867ede/rootfs/etc/cont-init.d/01-confd-site-enable

# Pending composer rework of ISLE BUILD TOOLS.
if [ "$PULL_ISLE_BUILD_TOOLS" = true ] ; then
    echo "Pulling newest version of ISLE Drupal Islandora build tools."
    cd /utility-scripts/isle_drupal_build_tools || exit
    EXISTING_BUILD=`git rev-parse --short HEAD`
    echo "Updating to $ISLE_BUILD_TOOLS_BRANCH from $ISLE_BUILD_TOOLS_REPO."
    git remote rm origin
    git remote add origin $ISLE_BUILD_TOOLS_REPO
    git fetch --all && git reset origin/$ISLE_BUILD_TOOLS_BRANCH --hard
    NEW_BUILD_HASH=`git rev-parse --short HEAD`
    echo "Done updating build_tools;\nOld build: $EXISTING_BUILD -> New build: $NEW_BUILD_HASH"
fi

This seems to suggest the actual mechanics of pulling your remote git repo are in place but that comment above is throwing me off.

To be clear the code above would run everytime a) your apache container is started, created, restarted etc and the appropriate values within your .env so I'm not clear on what is happening.

[g7morris]

@McFateM Could you post your script etc (obviously scrape out any not for public line items etc) It would be good to see if ISLE maintainers can replicate what is happening here.

[McFateM]

@g7morris Sorry I didn't respond sooner, the @marksandford tag had me confused (should be @McFateM). I think you are correct about the script being overwritten. Ben's instructions earlier were to use isle_islandora_installer.sh as a template of sorts, and initially I replaced it with grinnell_installer.sh, and that worked. But I have made so many changes to my installer script that I thought it better to take a different approach, namely to have the installer script check for numbered scripts that I drop into a new custom.d folder. The intent was to be able to leave isle_islandora_installer.sh as-is (after my addition of code to pull in the custom.d contents) so that there are no custom 'changes' to the repo, only additions.

So, what I'm trying to do is introduce logic like this...

# ----------
echo "Checking for 01-* scripts in ISLE-Drupal-Build-Tools/custom.d"
for f in /utility-scripts/isle_drupal_build_tools/custom.d/01-*.sh; do
  bash "$f" -H || break
  echo "  $f finished."
done

echo "Using Drush makefile to create sample Drupal site within /tmp/drupal_install"
drush make --prepare-install /utility-scripts/isle_drupal_build_tools/isle-drush_make/drupal.drush.make /tmp/drupal_install

# ----------
echo "Checking for 02-* scripts in ISLE-Drupal-Build-Tools/custom.d"
for f in /utility-scripts/isle_drupal_build_tools/custom.d/02-*.sh; do
  bash "$f" -H || break
  echo "  $f finished."
done

echo "Using Islandora makefile for Islandora Modules for sample Drupal site within /tmp/drupal_install"
drush make --no-core /utility-scripts/isle_drupal_build_tools/isle-drush_make/islandora.drush.make /tmp/drupal_install

# ----------
echo "Checking for 03-* scripts in ISLE-Drupal-Build-Tools/custom.d"
for f in /utility-scripts/isle_drupal_build_tools/custom.d/03-*.sh; do
  bash "$f" -H || break
  echo "  $f finished."
done

Basically, there's a new for loop in between the major elements of the original script. Not sure this is a good idea? But thought I would give it a try and see how it plays.

I'm a little concerned with how maintainers might address updates to ISLE-Drupal-Build-Tools in its current form.

[br2490]

I'm on my phone, please forgive brevity and autocorrect doing it's thing.

@McFateM this had me confused for a bit. There it a file in the templates folder for the install script. If you update that (but pay attention to the place where confd vars are pulled in) you should be able to modify all the things. In theory you may only need to update that file only, and you can leave the normal bash script empty. Let us know, add apologies for any confusion ;)

[g7morris]

@McFateM My apologies on the wrong tag. 3 times!! I'm the worst.

Yes ultimately ISLE maintainers need to revisit this as we need to allow for more user customization. Additionally, some of the vsets are great when installing and tickering with isle.localdomain but would be considered destructive to a migrated site.

Lastly with respect to the drush make scripts, I think a general review of what is in there should also be reviewed. A documented chart / matrix with columns of Drupal module versions plugins etc should be made. (this time I'm really tagging @marksandford ;)

[g7morris]

@McFateM I also think more of a "how to" should exist for folks other than "be careful" when updating a single file, this spartan approach for such an important installation and migration should be less IKEA and more Sharper Image. ;). This is not a critique of @br2490 's hard work more of laying down a challenge to documenters and ISLE maintainers to pick up the mantle and streamline this.

[g7morris]

@br2490 you are referring to this template? https://github.com/Islandora-Collaboration-Group/ISLE-Drupal-Build-Tools/blob/master/templates/isle_islandora_installer.sh.tpl

I know you're on your phone just trying to help @McFateM pinpoint the resources.

Also from line #7 of that file:

# Composer will be next, but the files commited here are a direct lift of Mark's build tools from the Alpha.

^ I like this and feel this should be the next direction ISLE maintainers go in for improving this process.

[McFateM]

OK, thanks for all the info guys. I see what had me so confused here, but I still worry that I've made LOTS of changes to the *_installer.sh script, and to the *.make files too. So many changes that I'm worried I won't be able to reconcile them all if/when the underlying canonical files are modified as part of a maintenance release.

That's why I've started playing with the approach of only "adding" my customization into new files, without touching the existing stuff at all... sort of a "hook" approach. I am finding that is working nicely thus far, now that I understand what's happening to isle_islandora_installer.sh.

I think what I will do is continue on this path a bit farther using a new_installer.sh script that I created to replace my grinnell_installer.sh.

I also like the notion of trying to address this part of ISLE in phase 2 if time/resources allow. At the very least, some additional "how-to" documentation around the build tools would be welcome. Thanks.

[SummittDweller]

I think I've found an approach I like, but working with this in the .env...

# Repo and branch to pull?
ISLE_BUILD_TOOLS_REPO=https://github.com/DigitalGrinnell/ISLE-Drupal-Build-Tools.git
ISLE_BUILD_TOOLS_BRANCH=Digital.Grinnell

...is a real pain during development. Is there any way I can rewrite the ISLE_BUILD_TOOLS_REPO line to pull from a local repo? As it is, I make a change and I have to push it back to the named Github repo just to spin it up and test.