Closed jrwdunham closed 5 years ago
The following features under core/ were skipped for the reasons supplied in parentheses after the feature name.
@unexecutable
)STATUS: PASSING
$ behave \
--tags=mo-aip-reingest \
--no-skipped \
-v \
-D am_username=test \
-D am_password=test \
-D am_url=http://127.0.0.1:62080/ \
-D am_version=1.7 \
-D am_api_key=test \
-D ss_username=test \
-D ss_password=test \
-D ss_url=http://127.0.0.1:62081/ \
-D ss_api_key=test \
-D home=archivematica \
-D driver_name=Firefox
1 feature passed, 0 failed, 19 skipped
1 scenario passed, 0 failed, 51 skipped
36 steps passed, 0 failed, 522 skipped, 0 undefined
Took 4m35.652s
STATUS: PASSING
$ behave \
--tags=ipc \
-D am_url=http://127.0.0.1:62080/ \
-D ss_url=http://127.0.0.1:62081/ \
-D am_username=test \
-D am_password=test \
-D ss_username=test \
-D ss_password=test \
--no-skipped \
-D driver_name=Firefox \
-D transfer_source_path=archivematica/archivematica-sampledata/TestTransfers/acceptance-tests \
-D am_version=1.7
On the first run, the @preservation @nonmanual
feature failed on the all-conform-policy
because of a circumstantial failure of the tests, viz. RecursionError: maximum recursion depth exceeded while calling a Python object
was triggered by the recursive am_browser_jobs_tasks_ability.py::get_job_uuid
. This is reflected in the failed scenario below:
0 features passed, 1 failed, 19 skipped
7 scenarios passed, 1 failed, 44 skipped
108 steps passed, 1 failed, 449 skipped, 0 undefined
Took 22m1.490s
On a second run (i.e., adding --tags=preservation --tags=nonmanual
to the above command), the failing scenario passed:
1 feature passed, 0 failed, 19 skipped
2 scenarios passed, 0 failed, 50 skipped
32 steps passed, 0 failed, 526 skipped, 0 undefined
Took 4m55.505s
STATUS: PASSING
$ behave \
--tags=icc \
-D am_url=http://127.0.0.1:62080/ \
-D ss_url=http://127.0.0.1:62081/ \
-D am_username=test \
-D am_password=test \
-D ss_username=test \
-D ss_password=test \
--no-skipped \
-D driver_name=Firefox \
-D transfer_source_path=archivematica/archivematica-sampledata/TestTransfers/acceptance-tests \
-D am_version=1.7
1 feature passed, 0 failed, 19 skipped
2 scenarios passed, 0 failed, 50 skipped
19 steps passed, 0 failed, 539 skipped, 0 undefined
Took 4m23.470s
STATUS: PASSING
Note: the AM/SS URLs, which are different from those in the commands above. This is a result of updating my am.git.
$ behave \
--tags=tpc \
-D am_url=http://127.0.0.1:61780/ \
-D ss_url=http://127.0.0.1:61781/ \
-D am_username=test \
-D am_password=test \
-D ss_username=test \
-D ss_password=test \
--no-skipped \
-D driver_name=Firefox \
-D transfer_source_path=archivematica/archivematica-sampledata/TestTransfers/acceptance-tests \
-D am_version=1.7
1 feature passed, 0 failed, 19 skipped
2 scenarios passed, 0 failed, 50 skipped
26 steps passed, 0 failed, 532 skipped, 0 undefined
Took 4m27.059s
STATUS: PASSING
$ behave \
--tags=picc \
-D am_url=http://127.0.0.1:61780/ \
-D ss_url=http://127.0.0.1:61781/ \
-D am_username=test \
-D am_password=test \
-D ss_username=test \
-D ss_password=test \
--no-skipped \
-D driver_name=Firefox \
-D transfer_source_path=archivematica/archivematica-sampledata/TestTransfers/acceptance-tests \
-D am_version=1.7
1 feature passed, 0 failed, 19 skipped
2 scenarios passed, 0 failed, 50 skipped
14 steps passed, 0 failed, 544 skipped, 0 undefined
Took 3m43.780s
STATUS: PASSING
$ behave \
--tags=uuids-dirs \
-D am_url=http://127.0.0.1:61780/ \
-D ss_url=http://127.0.0.1:61781/ \
-D am_username=test \
-D am_password=test \
-D ss_username=test \
-D ss_password=test \
--no-skipped \
-D driver_name=Firefox \
-D transfer_source_path=archivematica/archivematica-sampledata/TestTransfers/acceptance-tests \
-D am_version=1.7 \
-D docker_compose_path=/path/to/dir/containing/docker/compose/file/ \
-D home=archivematica
The uuids-for-directories.feature is not idempotent. That is, you cannot run it more than once and expect it to pass. This is because it involves starting a Zipped Bag type transfer and such transfers cannot be given unique names using Archivematica: their names are based on the basename of the .zip file used as the transfer source. Therefore, in order to run this feature multiple times, you have to close all transfers and ingests (or at least the ones named "BagTransfer"). You can do this with the following two commands:
Close all transfers:
$ ./close_all_transfers.sh \
-D am_url=http://127.0.0.1:61780/ \
-D ss_url=http://127.0.0.1:61781/ \
-D am_username=test \
-D am_password=test \
-D ss_username=test \
-D ss_password=test \
-D driver_name=Firefox \
-D am_version=1.7
Close all ingests:
$ ./close_all_ingests.sh \
-D am_url=http://127.0.0.1:61780/ \
-D ss_url=http://127.0.0.1:61781/ \
-D am_username=test \
-D am_password=test \
-D ss_username=test \
-D ss_password=test \
-D driver_name=Firefox \
-D am_version=1.7
1 feature passed, 0 failed, 19 skipped
2 scenarios passed, 0 failed, 50 skipped
12 steps passed, 0 failed, 546 skipped, 0 undefined
Took 3m25.053s
STATUS: PASSING
$ behave \
--tags=premis-events \
--no-skipped \
-v \
-D am_username=test \
-D am_password=test \
-D am_url=http://127.0.0.1:61780/ \
-D am_version=1.7 \
-D am_api_key=test \
-D ss_username=test \
-D ss_password=test \
-D ss_url=http://127.0.0.1:61781/ \
-D ss_api_key=test \
-D home=archivematica \
-D driver_name=Firefox
1 feature passed, 0 failed, 19 skipped
5 scenarios passed, 0 failed, 47 skipped
86 steps passed, 0 failed, 472 skipped, 0 undefined
Took 9m58.158s
STATUS: FAILING
$ behave \
--tags=pid-binding \
--no-skipped \
-D am_username=test \
-D am_password=test \
-D am_url=http://127.0.0.1:61780/ \
-D am_api_key=test \
-D ss_username=test \
-D ss_password=test \
-D ss_url=http://127.0.0.1:61781/ \
-D ss_api_key=test \
-D home=archivematica \
-D driver_name=Firefox \
-D am_version=1.7 \
-D docker_compose_path=/path/to/dir/containing/docker/compose/file/ \
-D pid_web_service_endpoint=<SOME_URL> \
-D pid_web_service_key=<SOME_SECRET> \
-D handle_resolver_url=<SOME_RESOLVER_URL> \
-D base_resolve_url=<SOME_RESOLVE_URL> \
-D pid_xml_namespace=<SOME_NAMESPACE>
0 features passed, 1 failed, 19 skipped
0 scenarios passed, 1 failed, 51 skipped
7 steps passed, 1 failed, 550 skipped, 0 undefined
Took 1m59.168s
The failure is presumably caused by the supplied handle credentials being invalid:
Then the AIP METS file documents PIDs, PURLs, and UUIDs for all files, directories and the package itself
# features/steps/pid_binding_steps.py:97 12.018s
Assertion Failed: Unable to find an identifier of type hdl for entity hierarchy_with_empty_dir_1536189083-f45125dc-69a7-42ac-8246-7e88ed519a91
The following AMAUAT features were run against an Archivematica instance deployed on OVH (Ubuntu Xenial) using deploy-pub (Vagrant/Ansible). The steps used to do this are given below.
First, install the vagrant openstack provider::
$ vagrant plugin install vagrant-openstack-provider
Then checkout the dev/vagrant-openstack
branch of deploy-pub::
$ pwd
/.../deploy-pub/playbooks/archivematica-xenial
$ git st
On branch dev/vagrant-openstack
Make some mods to the Vagrantfile. (Note: using the "xenial_template1" makes the provisioning much faster, but it failed for me so that's why the change to "Ubuntu 16.04".)::
$ git diff
diff --git a/playbooks/archivematica-xenial/Vagrantfile b/playbooks/archivematica-xenial/Vagrantfile
index c8b8329..bdc7bfe 100644
--- a/playbooks/archivematica-xenial/Vagrantfile
+++ b/playbooks/archivematica-xenial/Vagrantfile
@@ -22,8 +22,10 @@ Vagrant.configure('2') do |config|
os.password = ENV['OS_PASSWORD']
os.region = ENV['OS_REGION_NAME']
os.flavor = "b2-7"
- os.image = "xenial_template1"
+ #os.image = "xenial_template1"
+ os.image = "Ubuntu 16.04"
ov.nfs.functional = false
+ ov.vm.allowed_synced_folder_types = :rsync
end
Download the Ansible roles::
$ ansible-galaxy install -f -p roles/ -r requirements.yml
Bring up the OVH VM and provision it::
$ vagrant up
Get the SSH details::
$ vagrant ssh-config
Host default
HostName XXX.XX.XXX.XXX
User ubuntu
IdentityFile /.../deploy-pub/playbooks/archivematica-xenial/.vagrant/machines/default/openstack/vagrant-generated-eh6rotde
Add the ubuntu user to the archivematica group::
ubuntu@test-xenial:~$ sudo adduser ubuntu archivematica
Adding user `ubuntu' to group `archivematica' ...
Adding user ubuntu to group archivematica
Done.
STATUS: PASSING
$ behave \
--tags=aip-encrypt-mirror \
--no-skipped \
-D am_version=1.7 \
-D driver_name=Firefox \
-D am_username=admin \
-D am_password=archivematica \
-D am_url=http://XXX.XX.XXX.XXX/ \
-D ss_username=admin \
-D ss_password=archivematica \
-D ss_url=http://XXX.XX.XXX.XXX:8000/ \
-D home=ubuntu \
-D server_user=ubuntu \
-D ssh_identity_file=/.../deploy-pub/playbooks/archivematica-xenial/.vagrant/machines/default/openstack/vagrant-generated-eh6rotde
1 feature passed, 0 failed, 19 skipped
1 scenario passed, 0 failed, 51 skipped
25 steps passed, 0 failed, 533 skipped, 0 undefined
Took 3m8.262s
The first attempt to run this feature failed on the second step of the sole scenario And there is a standard GPG-encrypted Replicator location in the storage service
with an error in the GUI involving a missing value for a required field. @scollazo saw the same issue in a separate run of this feature.
STATUS: FAILING
$ behave \
--tags=aip-encrypt \
--no-skipped \
-D am_version=1.7 \
-D driver_name=Firefox \
-D am_username=admin \
-D am_password=archivematica \
-D am_url=http://XXX.XX.XXX.XXX/ \
-D ss_username=admin \
-D ss_password=archivematica \
-D ss_url=http://XXX.XX.XXX.XXX:8000/ \
-D home=ubuntu \
-D server_user=ubuntu \
-D ssh_identity_file=/.../deploy-pub/playbooks/archivematica-xenial/.vagrant/machines/default/openstack/vagrant-generated-eh6rotde
Attempting to store a (compressed) AIP in a GPG space will fail at Store AIP: Store the AIP.
STATUS: PASSING
Same as above.
1 feature passed, 0 failed, 19 skipped
7 scenarios passed, 0 failed, 45 skipped
59 steps passed, 0 failed, 499 skipped, 0 undefined
Took 22m5.523s
STATUS: PASSING
With output capturing enabled in AM:
$ behave -v --tags=capture-stdout \
--tags=deploy.output_capture_state.on \
--no-skipped \
-D am_username=admin \
-D am_password=archivematica \
-D am_url=http://XXX.XX.XXX.XXX/ \
-D am_version=1.7 \
-D ss_username=admin \
-D ss_password=archivematica \
-D ss_url=http://XXX.XX.XXX.XXX:8000/ \
-D home=ubuntu \
-D driver_name=Firefox
With output capturing disabled in AM:
$ behave -v --tags=capture-stdout \
--tags=deploy.output_capture_state.off \
--no-skipped \
-D am_username=admin \
-D am_password=archivematica \
-D am_url=http://XXX.XX.XXX.XXX/ \
-D am_version=1.7 \
-D ss_username=admin \
-D ss_password=archivematica \
-D ss_url=http://XXX.XX.XXX.XXX:8000/ \
-D home=ubuntu \
-D driver_name=Firefox
Note: to disable output capturing in this type of OVH Vagrant/Ansible Ubuntu 16 deploy, add
ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_CAPTURE_CLIENT_SCRIPT_OUTPUT=false
to
/etc/default/archivematica-mcp-client
:
$ cat /etc/default/archivematica-mcp-client
...
ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_CAPTURE_CLIENT_SCRIPT_OUTPUT=false
...
and restart the MCP client service:
$ sudo service archivematica-mcp-client stop
$ sudo service archivematica-mcp-client start
With output capturing enabled in AM:
1 feature passed, 0 failed, 19 skipped
1 scenario passed, 0 failed, 51 skipped
4 steps passed, 0 failed, 554 skipped, 0 undefined
Took 1m1.208s
With output capturing disabled in AM:
1 feature passed, 0 failed, 19 skipped
1 scenario passed, 0 failed, 51 skipped
4 steps passed, 0 failed, 554 skipped, 0 undefined
Took 1m1.962s
STATUS: PASSING
$ behave \
--tags=deploy.method.ansible,same-aip,gui \
--no-skipped \
-D am_version=1.7 \
-D driver_name=Firefox \
-D am_username=admin \
-D am_password=archivematica \
-D am_url=http://XXX.XX.XXX.XXX/ \
-D ss_username=admin \
-D ss_password=archivematica \
-D ss_url=http://XXX.XX.XXX.XXX:8000/ \
-D home=ubuntu
To put this OVH Vagrant/Ansible Ubuntu 16 deploy in indexless mode, set the following env vars as described and restart the relevant services:
$ cat /etc/default/archivematica-dashboard
ARCHIVEMATICA_DASHBOARD_DASHBOARD_SEARCH_ENABLED=false
$ cat /etc/default/archivematica-mcp-server
ARCHIVEMATICA_MCPSERVER_MCPSERVER_SEARCH_ENABLED=false
$ cat /etc/default/archivematica-mcp-client
ARCHIVEMATICA_MCPCLIENT_MCPCLIENT_SEARCH_ENABLED=false
$ sudo service archivematica-dashboard stop
$ sudo service archivematica-dashboard start
$ sudo service archivematica-mcp-server stop
$ sudo service archivematica-mcp-server start
$ sudo service archivematica-mcp-client stop
$ sudo service archivematica-mcp-client start
Since we are simply disabling indexing on a deploy that already has ES installed, it does not make sense for the feature to assert that ES is not installed. Therefore, comment out the step that does that:
$ git diff features/
diff --git a/features/core/indexless.feature b/features/core/indexless.feature
index 09f729e..365c53e 100644
--- a/features/core/indexless.feature
+++ b/features/core/indexless.feature
@@ -66,7 +66,7 @@ Feature: Archivematica is deployed without Indexing feature
Given Archivematica deployment method <method> which provides an option to exclude Elasticsearch
When Archivematica is deployed without Elasticsearch using method <method>
Then the installation has Indexing disabled
- And Elasticsearch is not installed
+ #And Elasticsearch is not installed
1 feature passed, 0 failed, 19 skipped
3 scenarios passed, 0 failed, 49 skipped
33 steps passed, 0 failed, 522 skipped, 0 undefined
Took 4m27.164s
$ behave --tags=man-norm \
--no-skipped \
-D am_version=1.7 \
-D driver_name=Firefox \
-D am_username=admin \
-D am_password=archivematica \
-D am_url=http://XXX.XX.XXX.XXX/ \
-D ss_username=admin \
-D ss_password=archivematica \
-D ss_url=http://XXX.XX.XXX.XXX:8000/ \
-D home=ubuntu
STATUS: FAILING
0 features passed, 1 failed, 19 skipped
0 scenarios passed, 1 failed, 51 skipped
5 steps passed, 1 failed, 549 skipped, 0 undefined
Took 19m52.671s
AM (normalize.py) is failing to recognize that %SIPDirectory%objects/image.bmp has a manually normalized preservation derivative in %SIPDirectory%objects/manualNormalization/preservation/image.tif. See https://github.com/archivematica/Issues/issues/150.
STATUS: PASSING
This is with cherry-picking the fix from https://github.com/artefactual/archivematica/pull/973 into stable/1.7.x.
1 feature passed, 0 failed, 19 skipped
1 scenario passed, 0 failed, 51 skipped
11 steps passed, 0 failed, 547 skipped, 0 undefined
Took 5m38.030s
This requires a fix to the manual-normalization.feature
so that the "Document empty directories" decision is provided. This fix is provided in https://github.com/artefactual-labs/archivematica-acceptance-tests/pull/128.
STATUS: PASSING
$ behave \
--tags=mediaconch-challenging-mkv \
--no-skipped \
-D am_version=1.7 \
-D driver_name=Firefox \
-D am_username=admin \
-D am_password=archivematica \
-D am_url=http://XXX.XX.XXX.XXX/ \
-D ss_username=admin \
-D ss_password=archivematica \
-D ss_url=http://XXX.XX.XXX.XXX:8000/ \
-D home=ubuntu
1 feature passed, 0 failed, 19 skipped
1 scenario passed, 0 failed, 51 skipped
10 steps passed, 0 failed, 548 skipped, 0 undefined
Took 6m37.841s
On the first run, this failed at Then all PREMIS implementation-check-type validation events have eventOutcome = pass
. This seems to have been an issue with the tests themselves and not with AM.
Please describe the problem you'd like to be solved.
Archivematica's automated user acceptance test suite (AMAUAT) should be run against the latest Archivematica v.1.7.2 release candidate prior to cutting the release. This will help us to discover regressions. Cf. https://github.com/artefactual/archivematica/issues/942.
Describe the solution you'd like to see implemented.
I would like to see the following Archivematica deploy tested with AMAUAT and the results documented in this issue:
The following tests were run against an Archivematica instance deployed locally with the above two commits using the am.git Docker Compose method. The AMAUAT tests were run from the host machine. Here are the details of the testing environment: