artefactual / automation-tools

Tools to aid automation of Archivematica and AtoM.
GNU Affero General Public License v3.0
46 stars 33 forks source link

Add create_dip module and scripts #35

Closed jraddaoui closed 6 years ago

jraddaoui commented 7 years ago

Creates a new module to create a DIP from an AIP in the SS and a Bash script with an use example. Check the README file for more information.

joel-simpson commented 7 years ago

The requirements for these scripts are currently in draft in this google doc: https://docs.google.com/document/d/1UH6IbmEIj_D1fdOzEx5ThntpjEpRZlOao7euRwx3mXU/edit#heading=h.h6s5mhw97nml I will be updating these in the next few days and creating proper feature files in the acceptance test repo (in a dev branch).

sevein commented 7 years ago

@jraddaoui, I noticed you pushed directly to GH but I pushed the same branch to the local server to avoid the branch being closed. I've also added Travis CI (runs tests and linter) to master and rebased your branch. You can see now the output of the build and see why flake8 is complaining - they're quick cosmetic fixes. You can also run it locally:

$ pip install tox
$ tox
./aips/create_dip.py:14:1: I100 Import statements are in the wrong order. import re should be before import requests
./aips/create_dip.py:97:15: E201 whitespace after '{'
./aips/create_dip.py:97:58: E202 whitespace before '}'

More about import statements here: https://www.python.org/dev/peps/pep-0008/#imports.

jraddaoui commented 6 years ago

Hi @sevein, I still need to add some documentation in the README but, could you please take another look?

There is a related PR in the acceptace-tests repository that is assigned to @jrwdunham, but maybe it's better if you take care of both.

https://github.com/artefactual-labs/archivematica-acceptance-tests/pull/35

Thanks!

jraddaoui commented 6 years ago

It looks like the checks failed in here, but not locally:

(env) radda@thebeast:~/artefactual/automation-tools$ tox
py27 installed: certifi==2017.11.5,chardet==3.0.4,configparser==3.5.0,contextlib2==0.5.5,enum34==1.1.6,flake8==3.4.1,flake8-import-order==0.13,funcsigs==1.0.2,idna==2.6,lxml==4.1.1,mccabe==0.6.1,metsrw==0.2.0,mock==2.0.0,pbr==3.1.1,py==1.4.34,pycodestyle==2.3.1,pyflakes==1.5.0,pytest==3.2.3,PyYAML==3.12,requests==2.18.4,six==1.11.0,SQLAlchemy==1.1.15,urllib3==1.22,vcrpy==1.11.1,wrapt==1.10.11
py27 runtests: PYTHONHASHSEED='3266699143'
py27 runtests: commands[0] | pytest
========================================== test session starts ===========================================
platform linux2 -- Python 2.7.12, pytest-3.2.3, py-1.4.34, pluggy-0.4.0
rootdir: /home/radda/artefactual/automation-tools, inifile:
collected 42 items                                                                                        

tests/test_amclient.py ........................
tests/test_create_dip.py ....
tests/test_transfers.py ..............

======================================= 42 passed in 0.71 seconds ========================================
py27-flake8 installed: certifi==2017.11.5,chardet==3.0.4,configparser==3.5.0,contextlib2==0.5.5,enum34==1.1.6,flake8==3.4.1,flake8-import-order==0.13,funcsigs==1.0.2,idna==2.6,lxml==4.1.1,mccabe==0.6.1,metsrw==0.2.0,mock==2.0.0,pbr==3.1.1,py==1.4.34,pycodestyle==2.3.1,pyflakes==1.5.0,pytest==3.2.3,PyYAML==3.12,requests==2.18.4,six==1.11.0,SQLAlchemy==1.1.15,urllib3==1.22,vcrpy==1.11.1,wrapt==1.10.11
py27-flake8 runtests: PYTHONHASHSEED='3266699143'
py27-flake8 runtests: commands[0] | flake8 .
py36 installed: certifi==2017.11.5,chardet==3.0.4,flake8==3.4.1,flake8-import-order==0.13,idna==2.6,lxml==4.1.1,mccabe==0.6.1,metsrw==0.2.0,multidict==3.3.2,py==1.4.34,pycodestyle==2.3.1,pyflakes==1.5.0,pytest==3.2.3,PyYAML==3.12,requests==2.18.4,six==1.11.0,SQLAlchemy==1.1.15,urllib3==1.22,vcrpy==1.11.1,wrapt==1.10.11,yarl==0.13.0
py36 runtests: PYTHONHASHSEED='3266699143'
py36 runtests: commands[0] | pytest
========================================== test session starts ===========================================
platform linux -- Python 3.6.3, pytest-3.2.3, py-1.4.34, pluggy-0.4.0
rootdir: /home/radda/artefactual/automation-tools, inifile:
collected 42 items                                                                                        

tests/test_amclient.py ........................
tests/test_create_dip.py ....
tests/test_transfers.py ..............

======================================= 42 passed in 0.85 seconds ========================================
py36-flake8 installed: certifi==2017.11.5,chardet==3.0.4,flake8==3.4.1,flake8-import-order==0.13,idna==2.6,lxml==4.1.1,mccabe==0.6.1,metsrw==0.2.0,multidict==3.3.2,py==1.4.34,pycodestyle==2.3.1,pyflakes==1.5.0,pytest==3.2.3,PyYAML==3.12,requests==2.18.4,six==1.11.0,SQLAlchemy==1.1.15,urllib3==1.22,vcrpy==1.11.1,wrapt==1.10.11,yarl==0.13.0
py36-flake8 runtests: PYTHONHASHSEED='3266699143'
py36-flake8 runtests: commands[0] | flake8 .
________________________________________________ summary _________________________________________________
  py27: commands succeeded
  py27-flake8: commands succeeded
  py36: commands succeeded
  py36-flake8: commands succeeded
  congratulations :)
sevein commented 6 years ago

@jraddaoui this is probably happening because the new dep (7z) has not been added to .travis.yml, can you do that please? This is an example that you can use: https://github.com/artefactual/archivematica-storage-service/blob/4ad0f042039810849e3f12047a1fc12cdb3b78c4/.travis.yml#L7-L12.

jraddaoui commented 6 years ago

Thanks @sevein!!

jraddaoui commented 6 years ago

Merged in 3fcda1b644450152c2d6a76b51328480d1036381