Update info about what happens when not specifying a branch when running script that deploys Dataverse on AWS

[jggautier]

If we don't specify a branch with the -b option, the script for spinning up an instance of Dataverse on AWS doesn't default to the develop branch but to the latest version of Dataverse, such as v6.2 if that's the latest version.

In order to correct what users are told about what happens when -b is not used when running the script for creating a Dataverse instance on AWS, I'm proposing in a PR I'll open a bit that we remove the line * the default branch is the develop from the documentation at https://github.com/GlobalDataverseCommunityConsortium/dataverse-ansible/tree/develop/ec2.

Running the script with the -h option also tells the user that the develop branch is used when the -b option isn't used to specify a branch. This needs to be up updated, too, but I'm not sure how to do that. Does the usage part of the ec2-create-instance.sh script need to be changed?

The documentation in the Dataverse Developers Guide is also being updated. See https://github.com/IQSS/dataverse/issues/10572 and conversation in the PR at https://github.com/IQSS/dataverse/pull/10589.

[jggautier]

Scratch that, I can't create a branch in this repo in order to create a PR to remove the line * the default branch is the develop from the documentation at https://github.com/GlobalDataverseCommunityConsortium/dataverse-ansible/tree/develop/ec2.

Could someone else do this? @donsizemore or @pdurbin maybe?

[pallinger]

But you can create a pull request, you just need to fork the repo, and create your own branch there. Or not even create your own branch and create a PR from your develop branch!

[pallinger]

Moreover, as far as I see in ec2-create-instance.sh:

BRANCH_DEFAULT="develop"

so it should deploy the develop branch. Which may say about itself, that it is v6.2. You should actually login to the ec2 instance, and see the checked-out dataverse directory and see which branch is checked out. I unfortunately cannot try this out, as I do not have an aws subscription.

[jggautier]

Thanks for looking into what should be deployed by default when the -b option isn't included when we run ec2-create-instance.sh.

On May 17 my colleague @landreev created an instance this way, that is without including -b, and we saw "v 6.2 build 1603-a218417" on the bottom right of the webpage where the Dataverse version is indicated. Is that what you mean by seeing the checked-out dataverse directory?

v6.2 was released on April 2, a month before that instance was created. And the develop branch doesn't really have a version number yet, right? So would we really expect to see v6.2 at the bottom of an instance that would be based on the develop branch?

[donsizemore]

@jggautier my addled memory is hazy, but due to Ansible's group_var priorities, if you give the script a -g (group_vars file) and a -b (Dataverse branch), group_vars will win out? I forget. I haven't had time to dig into this or anything fun, but pull requests are most welcome.

[jggautier]

Hi @donsizemore. I'm not sure if the group_vars will win out but I can try to find out.

I haven't had luck spinning up instances, even when running ec2-create-instance.sh alone (without -b, -g or any other options), which is why @landreev created the instance on May 17, (which I think he did without -g and -b).

[donsizemore]

@jggautier do you have the current version of the script? (and how far do you get? error message?)

[jggautier]

Hey @donsizemore. Yeah I always sync my copy of the dataverse-ansible repo so that I have the latest version of the script.

When I run the script with no arguments, it seems to finish. At the end it tells me the URL of the instance, how to ssh into it and how to delete it, but the URL leads to a page that says that "This site can’t be reached" and it "refused to connect."

Before that, the last error message I see is:

rsync error: some files could not be transferred (code 23) at /AppleInternal/Library/BuildRoots/8e85887f-758c-11ee-a904-2a65a1af8551/Library/Caches/com.apple.xbs/Sources/rsync/rsync/main.c(1404) [receiver=2.6.9]

And before that I see other errors like NO MORE HOSTS LEFT, "ERROR: compilation failed for package ‘DescTools’" and failed: [localhost] (item=DescTools). The last two seem similar to what @pdurbin wrote in https://github.com/GlobalDataverseCommunityConsortium/dataverse-ansible/issues/357.

There are other things that looks like errors and warnings. I can Slack you the complete output if you'd like.

[pallinger]

we saw "v 6.2 build 1603-a218417"

a218417 is really the commit for the v6.2 tag, so that is really not develop. So the script has a bug, we should fix it! After inspecting ec2/ec2-create-instance.sh I cannot really find where v6.2 would come from. It seems to me that dataverse_branch is correctly overridden on the ansible-playbook command line. This needs a thorough debugging...

[donsizemore]

@jggautier you're seeing this: https://github.com/IQSS/dataverse/issues/10452 which is dependent on DescTools being fixed upstream (and likely won't happen any time soon).

I suggest switching to Rocky 9: https://raw.githubusercontent.com/GlobalDataverseCommunityConsortium/dataverse-ansible/243_rocky_9/ec2/ec2-create-instance.sh (which will be updated to 9.4 once AWS publishes Rocky's new images)

[jggautier]

a218417 is really the commit for the v6.2 tag, so that is really not develop. So the script has a bug, we should fix it!

@pallinger I've been assuming that we intend for the instance to use the latest released version of Dataverse when no branch is specified. So it's the documentation that's wrong and not a bug with the script. Does that make sense? That's why this GitHub issue is about updating documentation to match what's happening and not about changing what's happening.

Do you think the develop branch should be used when the script is run with no arguments?

@donsizemore and @pdurbin what you think?

I don't mind either way, as long as it's documented of course.

@jggautier you're seeing this: https://github.com/IQSS/dataverse/issues/10452 which is dependent on DescTools being fixed upstream (and likely won't happen any time soon).

I suggest switching to Rocky 9: https://raw.githubusercontent.com/GlobalDataverseCommunityConsortium/dataverse-ansible/243_rocky_9/ec2/ec2-create-instance.sh (which will be updated to 9.4 once AWS publishes Rocky's new images)

Ah okay. Thanks @donsizemore. I suppose that's what @landreev did? I'll try running the script you linked to without any arguments and if it works see what version of Dataverse is used.

[donsizemore]

@jggautier I bet with no arguments it's finding the default main.yml - I remember some screwiness around this, but it's years past. As a workaround, passing it a group_vars file should always work.

[pallinger]

@jggautier

Do you think the develop branch should be used when the script is run with no arguments?

I do not think that, but reading the script, it seems to use the develop branch by default, or the parameter after -b otherwise. My main concern is that if it does not use the develop branch by default, then maybe the -b option also does not work...

[jggautier]

I was able to spin up an instance using the script you shared @donsizemore, and v6.2 was on the bottom right. Seems like more evidence for your bet about it finding the default main.yml.

@pallinger Using -b has worked for me, although it's been a while since I've needed to use it.

I'm getting less certain about what to do now.

Should I create a PR that does the following?:

• edit the documentation at https://github.com/GlobalDataverseCommunityConsortium/dataverse-ansible/tree/develop/ec2 so that the line * the default branch is develop is changed to if you run the script without any arguments, it should spin up the latest version of Dataverse
• edit the usage part of the ec2-create-instance.sh script so that the line echo "default branch is develop" is changed to echo "if you run the script without any arguments, it should spin up the latest version of Dataverse"

[donsizemore]

@jggautier last time I ran into this I just added an exit 0 in the script before it actually did anything, which along with a few echo commands told me what I needed. Let me tinker for a minute.

[pdurbin]

v6.2 was on the bottom right

Was there a build number? Did it match the one from the 6.2 release?

[jggautier]

Yeah there was more that came after the v6.2 but I don't remember what it was and if it matches the one from the v6.2 release.

I could spin up an instance again and see.

"build 1603-a218417" is the build number of the 6.2 release, right?

[pdurbin]

Yep, I unzipped the 6.2 war and it's the same build number:

% grep build.number WEB-INF/classes/BuildNumber.properties
build.number=1603-a218417

[jggautier]

I spun up an instance again and saw build 1603-a218417, so it matches the one from the 6.2 release.

[pdurbin]

Thanks, I went ahead and merged the guides PR:

• https://github.com/IQSS/dataverse/pull/10589

[donsizemore]

@jggautier @pdurbin I'm finally looking at this (sorry). Looks like Ansible is being launched on the instance as /usr/bin/python3.12 /usr/bin/ansible-playbook -i dataverse/inventory dataverse/dataverse.pb --connection=local -e dataverse_branch=develop which according to https://docs.ansible.com/ansible/latest/reference_appendices/general_precedence.html#command-line-options should override the default group_vars file. Still poking.

[donsizemore]

@jggautier @pdurbin I just launched the script with

$ ec2/ec2-create-instance.sh -b develop

and I'm seeing

{"version":"6.2","build":"develop-"}

but on the instance, in /opt/dataverse/dataverse I issue sudo -u dataverse git status and I see

On branch develop
Your branch is up to date with 'origin/develop'