= Introduction = This is the full MiG project code released at the MiGrid project at SourceForge:

https://sourceforge.net/projects/migrid/

and in the process of moving to github

MiG is Free Software originally designed and developed by the MiG Project lead by Brian Vinter (currently brian DOT vinter AT au DOT dk). Development and maintenance continues with the Science HPC Center at University of Copenhagen taking over since Brian left.

Please refer to the COPYING file in this directory for further information about the GPL v2 license under which MiG is distributed.

= Getting Started = Please refer to the information available at the aforementioned URL especially the wiki pages including:

https://sourceforge.net/p/migrid/wiki/GettingStarted/

= Requirements = A MiG server basically requires an Apache web server, the OpenSSH client tools and a Python interpreter with a few external modules. -Apache 2.x (http://httpd.apache.org/) -Apache SSL module (http://httpd.apache.org/docs/current/mod/mod_ssl.html) -Apache proxy module (http://httpd.apache.org/docs/current/mod/mod_proxy.html) -Apache Rewrite module (http://httpd.apache.org/docs/current/mod/mod_rewrite.html) -OpenSSH clients (https://www.openssh.org/) -Python 2.7 or later (https://www.python.org/) -Python future module (https://pypi.org/project/future/)

Since early 2023 the Python 3.x support is stabilizing and getting real world testing. The trunk / master branch for the time being still concentrates on the old stable 2.x version, while the edge and experimental branches are used to polish 3.x support. At some point in mid 2024 we will completely make the switch to 3.x when popular distributions having python-2.x as default reach their End-of-Life.

On Debian/Ubuntu servers the corresponding basic packages can be installed with: sudo apt install apache2 openssh-client python python-pip python-setuptools \ python-dev build-essential and most of the optional dependencies similarly with: sudo apt install libapache2-mod-wsgi libapache2-mod-auth-openid \ libapache2-mod-auth-openidc python-irclib python-paramiko python-enchant \ python-jsonrpclib python-requests python-psutil python-future \ python-cracklib python-yaml python-cffi python-sendfile openssh-server \ libpam0g-dev mercurial trac trac-mercurial lftp rsync fail2ban ipset

On RedHat/CentOS servers the basic packages can be installed with: sudo yum install epel-release sudo yum install httpd mod_ssl openssh-clients python python-pip \ python-setuptools python-devel gcc

and most of the optional dependencies similarly with: sudo yum install mod_wsgi mod_auth_openid mod_auth_openidc python-paramiko \ python-enchant python-jsonrpclib python-requests python2-psutil \ python-future python2-cffi PyYAML pysendfile cracklib-python \ openssh-server pam-devel mercurial lftp rsync fail2ban ipset

while the remaining ones need to be installed from EPEL sudo yum install trac trac-mercurial-plugin or alternatively directly from pypi: sudo pip install trac tracmercurial

Apache comes with a number of modules natively included, so it is usually not necessary to explicitly install e.g. mod-proxy and mod-rewrite, only the modules explicitly packaged separately.

We highly recommend installing the optional python openid module directly from upstream, because packaged versions are generally outdated even on pypi. The packaged openid module lacks some security fixes and the upstream one can easily be installed with pip: sudo pip install https://github.com/openid/python-openid/archive/master.zip

The stable wsgidav version used to lack proper chrooting support and a fix for upload/write access for OSX clients but it should have been fixed in pypi releases. We're currently in the process of migrating to a modern 3.x+ version of wsgidav in order to also support python 3 use. For the time being we recommend running the old and thoroughly tested legacy 1.3.x version for production setups on python 2.x and it can easily be installed with pip: sudo pip install 'wsgidav<2'

On python 2 setups one can also try out the more recent 3.x series of wsgidav still maintained as stable and supported there. It can be installed with pip: sudo pip install 'more-itertools<6' 'jaraco.functools<3' 'jinja2<3' \ 'markupsafe<2' 'pyyaml<6' 'importlib-metadata<3' cheroot 'wsgidav<4'

On python 3 setups a more recent wsgidav 3.x or later is required. It can easily be installed with pip: sudo pip install cheroot wsgidav

Additional packaged Trac extensions can be installed with: sudo aptitude install trac-customfieldadmin trac-graphviz \ trac-mastertickets trac-wikiprint trac-wikirename trac-wysiwyg

and the unpackaged ones can be grabbed from trac-hacks.org with pip and easy_install: sudo easy_install https://trac-hacks.org/svn/wikicssplugin sudo easy_install https://trac-hacks.org/svn/fullblogplugin sudo easy_install https://trac-hacks.org/svn/discussionplugin sudo easy_install https://trac-hacks.org/svn/tracpasteplugin sudo easy_install https://trac-hacks.org/svn/downloadsplugin sudo pip install TracStats

where 0.11 in the URLs may need to be changed to fit your particular version of Trac. Please note that the source code stats in the TracStats plugin do not currently work for Mercurial repositories! Additional plugins are available from http://trac-hacks.org/

The downloads plugin currently needs patching to work. Please refer to the notes in the [downloads] section of the generated MiG trac.ini file.

Please note that there may be subtle internal plugin dependencies and conflicts that e.g. can cause problems if plugins are not loaded in the right order. We have seen database upgrade problems if FullBlog, Watchlist and Discussion are enabled but not loaded in an order where Watchlist is loaded in between the other two. Similar issues appeared when we enabled the Pastebin and Downloads plugins in one step. It was necessary to either patch tracdownloads/db/db1.py to ignore errors for existing tables or to enable one plugin and upgrade all Trac environments before enabling the other plugin and upgrading again. Thus you may have to experiment with the installed plugins in a conservative way.

With the inclusion of Trac we no longer rely on the MoinMoin software for stand-alone wikis.

The optional grid_ftps daemon requires the pyftpdlib module in a recent version, so it is easiest to install it with pip: sudo pip install pyftpdlib Please note that a recent pyopenssl module is required for TLS elliptic curve cipher support, which may be needed by some clients like FileZilla. At least if MiGserver.conf does not explicitly configure grid_ftps to allow strong legacy ciphers with the enable_ftps_legacy_tls option. The ancient pyOpenSSL package in CentOS 7 is too old and pyopenssl from pip pulls in a recent cryptography dependency, which breaks the native python-paramiko package there. So in short on CentOS 7 we recommend either sticking with the old pyOpenSSL package and limited ftps cipher support or installing the centos-openstack-RELEASE package (where RELEASE is 'stein' or later) to enable the official extra openstack repo with newer cryptography, pyopenssl and paramiko packages without these conflicts. Please carefully read https://docs.openstack.org/install-guide/environment-packages-rdo.html for the implications like potential EPEL conflicts if going with the latter.

The optional grid_webdavs daemon requires the wsgidav module in a specific version (1.3.0+ or 3.x for python 2 and 3.x+ for python 3), so it is easiest to install it with pip as mentioned above. We also rely on either the sslkeylog library or our own custom _sslsession module in grid_webdavs in order to efficiently reuse client sessions rather than repeating the auth handshake for every single DAV operation. The former requires python 2.7.9+ and can be installed with pip install sslkeylog and the latter which is supported for older python versions but is limited to OpenSSL versions prior to 1.1 can similarly be installed with cd mig/src/sslsession && pip install .

The optional grid_openid daemon requires the openid module in a recent version, so it is easiest to install it with pip as mentioned above.

The optional grid_events daemon requires the watchdog and scandir modules which may be installed with: sudo pip install watchdog scandir it is likely that the inotify default settings are too low to handle any serious number of vgrid shares so you may have to additionally tune the sysctl settings e.g. by adding the following to your /etc/sysctl.conf :

For grid_events daemon

It may be necessary to increase the number of watched files

fs.inotify.max_user_watches=1048576 fs.inotify.max_user_instances=1024

The optional grid_transfers daemon requires lftp and rsync clients to handle the background data transfers. Most Linux distributions come with versions that can be used right out of the box. However, the lftp 4.4.8 in CentOS/RHEL 7 appears to have an annoying bug, so that it loops forever instead of just failing if the username/password is incorrect. From the changelog it sounds like it was fixed in version 4.4.12, and neither the 4.4.13 version available in Ubuntu LTS (14.04) nor the 4.6 version on Debian stable exhibit this issue. Luckily the upstream project provides RPM's directly installable with yum as in: sudo yum install http://lftp.yar.ru/ftp/binaries/lftp-4.6.5-1.x86_64.rpm The only downside is that it then requires manual updates until the distro catches up.

The optional interactive computing environment relies on one or more separate Jupyter hosts. The easiest way of setting that up is to use the existing jupyter swarm-spawner with docker packs to launch all user sessions in isolated containers. This part is still work-in-progress so please get in touch for details. Only the python-requests module is needed for the actual MiG server-side integration.

The optional PDF generator environment relies on python pdfkit and xvfbwrapper which may be installed with: sudo pip install xvfbwrapper sudo pip install pip install git+https://github.com/Martin-Rehr/python-pdfkit.git xvfbwrapper require xorg-x11-server-Xvfb and pdfkit require wkhtmltopdf. If links are NOT working in the generated PDF's then try to install the latest stable version of wkhtmltopdf from: https://wkhtmltopdf.org/downloads.html

If the generated PDFs are gibberish then xorg-x11-fonts are most likely missing on the system.

The optional PyOTP module used for 2-factor authentication is not necessarily recent in distributions but then readily available with pip: sudo pip install pyotp

The optional cloud integration relies on the Python openstackclient package for communicating with the associated OpenStack provider. On CentOS 7 it can be installed from upstream repos with: yum install https://rdoproject.org/repos/rdo-release.rpm yum install python-openstackclient in line with https://docs.openstack.org/install-guide/environment-packages-rdo.html If your platform is not listed there you might also have luck with pip: sudo pip install openstackclient but we experienced a bunch of package conflicts in that setup.

The optional country code validation in certificate and OpenID account request backends requires the iso3166 module which may be installed with: sudo pip install iso3166

The optional pygdb module used for debugging is available with pip: sudo pip install pygdb

The optional but recommended email-validator module used for verification e.g. of user email addresses during sign up is available with pip. It relies on the similarly pip installable dnspython library, which dropped python2 support in the 2.x series. Furthermore email-validator itself silently started migrating to python 3 in the 1.3.x and officially in 2.x version. Thus, for it to work with python2 versions the pip installation command requires quite explicit versions as in: sudo pip install 'dnspython<2.0' 'email-validator<1.3' For python 3 one can simply install with pip3 without such workarounds: sudo pip3 install email-validator

The optional workflows module relies on the python special modules nbformat, nbconvert to validate and produce correct Jupyter Notebook formatted files. Nbformat specifically requires that the more_itertools module is both installed beforehand and that it is pinned to version 5.0.0 to ensure Python 2.7 support. Other versioned dependencies were needed, too, on CentOS 7: sudo pip install more_itertools==5.0.0 sudo pip install zipp==0.6.0 sudo pip install nbformat==4.4.0 sudo pip install pygments==2.4.2 sudo pip install nbconvert==5.6.1 In addition the workflows module creates yaml parameter files via the pyyaml module. To schedule and execute workflow tasks, the execution nodes are required to provide the papermill module via the PAPERMILL environment variable, the notebook_parameterizer module via the NOTEBOOK_PARAMETERIZER environment variable, and the sshfs command via the SSHFS_MOUNT environment variable.

= Installing MiG = If you want to run your own MiG server for your own grid or to develop MiG you should checkout / clone or download and unpack the source code (including this file) on a UNIX compatible computer as described below.

The MiG core services are provided by the MiG daemons from the mig/server directory and they can simply be run directly from the unpacked source code directory when a suitable server configuration is added.

For the web interfaces to work you will need to run an apache server as described in the mig/install directory. Grid job handout relies on OpenSSH client commands like ssh and scp. MiG does not include the actual Apache web server or OpenSSH clients, so you will need to install those using either packages provided by your distribution or install it from source.

MiG is tested on Debian/Ubuntu and Redhat/CentOS Linux using Apache 1.3 or 2.X with mod-ssl respectively but other distribution and apache combinations should also work. MiG relies on apache's mod-ssl for automatic certificate validation and access control. Furthermore quite a bit of rewrite rules are used for access and convenience so the mod-rewrite apache module is required too.

You can read more about the apache configuration in the provided mig/install/README.Debian file.

This server documentation expects the MiG code to run as a separate 'mig' user on the UNIX system, but this naming is not a requirement. Just modify your apache and MiG configurations appropriately if you want to run MiG as a different user or with other paths. It is important to configure apache so that the MiG web interfaces can read and write the files created by the MiG daemons and vice versa. This may require extra care if the MiG installation and apache run as different system users. If you use the default setup you do not need to worry about this.

As root you can create an ordinary user, mig, for running the MiG server:

su -

useradd -m -U mig

su - mig

To avoid other processes from tampering it is a good idea to set either the permissions on the entire mig user home very restrictively:

chmod 700 ~mig

or at least set the umask tight enough to avoid unauthorized access to the MiG server files. If you run MiG with different apache and mig users, you will most likely need to provide both users write access to the mig user home, though.

Download and unpack the MiG source or make a checkout from svn at

https://sourceforge.net/p/migrid/code/HEAD/tree/trunk/

or a clone from github at

https://github.com/ucphhpc/migrid-sync

At this point it may be comfortable to copy some of the basic account configuration files from mig/install/mig-user to ~/ but this is not mandatory.

Now you are ready to actually configure your installation. The easiest way to do that is to use the configuration generator in mig/install/generateconfs.py to create configurations that match your setup. For the default settings it could just be done as: cd mig/install/ ./generateconfs.py

If your setup uses custom paths or settings just provide them on the commandline like the command help indicates: ~/mig/install > ./generateconfs.py -h Usage: ./generateconfs.py [OPTIONS] Where supported options include -h/--help for this help or the conf settings: --source=SOURCE --destination=DESTINATION --destination_suffix=DESTINATION_SUFFIX --auto_add_filter_fields=AUTO_ADD_FILTER_FIELDS --auto_add_filter_method=AUTO_ADD_FILTER_METHOD --base_fqdn=BASE_FQDN --public_fqdn=PUBLIC_FQDN --public_alias_fqdn=PUBLIC_ALIAS_FQDN --public_sec_fqdn=PUBLIC_SEC_FQDN --mig_cert_fqdn=MIG_CERT_FQDN --ext_cert_fqdn=EXT_CERT_FQDN --mig_oid_fqdn=MIG_OID_FQDN --ext_oid_fqdn=EXT_OID_FQDN --mig_oidc_fqdn=MIG_OIDC_FQDN --ext_oidc_fqdn=EXT_OIDC_FQDN --sid_fqdn=SID_FQDN --io_fqdn=IO_FQDN --seafile_fqdn=SEAFILE_FQDN --seafile_base=SEAFILE_BASE --seafmedia_base=SEAFMEDIA_BASE --seafhttp_base=SEAFHTTP_BASE --openid_address=OPENID_ADDRESS --sftp_address=SFTP_ADDRESS --sftp_subsys_address=SFTP_SUBSYS_ADDRESS --ftps_address=FTPS_ADDRESS --davs_address=DAVS_ADDRESS --jupyter_services=JUPYTER_SERVICES --jupyter_services_desc=JUPYTER_SERVICES_DESC --cloud_fqdn=CLOUD_FQDN --cloud_services=CLOUD_SERVICES --cloud_services_desc=CLOUD_SERVICES_DESC --user=USER --group=GROUP --apache_version=APACHE_VERSION --apache_etc=APACHE_ETC --apache_run=APACHE_RUN --apache_lock=APACHE_LOCK --apache_log=APACHE_LOG --openssh_version=OPENSSH_VERSION --mig_code=MIG_CODE --mig_state=MIG_STATE --mig_certs=MIG_CERTS --mig_oid_title=MIG_OID_TITLE --mig_oid_provider=MIG_OID_PROVIDER --ext_oid_title=EXT_OID_TITLE --ext_oid_provider=EXT_OID_PROVIDER --mig_oidc_provider_meta_url=MIG_OIDC_PROVIDER_META_URL --ext_oidc_provider_meta_url=EXT_OIDC_PROVIDER_META_URL --ext_oidc_client_name=EXT_OIDC_CLIENT_NAME --ext_oidc_client_id=EXT_OIDC_CLIENT_ID --ext_oidc_scope=EXT_OIDC_SCOPE --ext_oidc_remote_user_claim=EXT_OIDC_REMOTE_USER_CLAIM --dhparams_path=DHPARAMS_PATH --daemon_keycert=DAEMON_KEYCERT --daemon_pubkey=DAEMON_PUBKEY --daemon_show_address=DAEMON_SHOW_ADDRESS --alias_field=ALIAS_FIELD --peers_permit=PEERS_PERMIT --vgrid_creators=VGRID_CREATORS --vgrid_managers=VGRID_MANAGERS --signup_methods=SIGNUP_METHODS --login_methods=LOGIN_METHODS --csrf_protection=CSRF_PROTECTION --password_policy=PASSWORD_POLICY --password_legacy_policy=PASSWORD_LEGACY_POLICY --hg_path=HG_PATH --hgweb_scripts=HGWEB_SCRIPTS --trac_admin_path=TRAC_ADMIN_PATH --trac_ini_path=TRAC_INI_PATH --user_clause=USER_CLAUSE --group_clause=GROUP_CLAUSE --listen_clause=LISTEN_CLAUSE --serveralias_clause=SERVERALIAS_CLAUSE --distro=DISTRO --autolaunch_page=AUTOLAUNCH_PAGE --landing_page=LANDING_PAGE --skin=SKIN --title=TITLE --short_title=SHORT_TITLE --peers_explicit_fields=PEERS_EXPLICIT_FIELDS --peers_contact_hint=PEERS_CONTACT_HINT --external_doc=EXTERNAL_DOC --secscan_addr=SECSCAN_ADDR --user_interface=USER_INTERFACE --vgrid_label=VGRID_LABEL --default_menu=DEFAULT_MENU --user_menu=USER_MENU --collaboration_links=COLLABORATION_LINKS --default_vgrid_links=DEFAULT_VGRID_LINKS --advanced_vgrid_links=ADVANCED_VGRID_LINKS --admin_email=ADMIN_EMAIL --admin_list=ADMIN_LIST --log_level=LOG_LEVEL --twofactor_mandatory_protos=TWOFACTOR_MANDATORY_PROTOS --freeze_to_tape=FREEZE_TO_TAPE --status_system_match=STATUS_SYSTEM_MATCH --duplicati_protocols=DUPLICATI_PROTOCOLS --gdp_data_categories=GDP_DATA_CATEGORIES --cert_valid_days=CERT_VALID_DAYS --oid_valid_days=OID_VALID_DAYS --generic_valid_days=GENERIC_VALID_DAYS --apache_worker_procs=APACHE_WORKER_PROCS --sftp_subsys_auth_procs=SFTP_SUBSYS_AUTH_PROCS --wsgi_procs=WSGI_PROCS --public_port=PUBLIC_PORT --public_http_port=PUBLIC_HTTP_PORT --public_https_port=PUBLIC_HTTPS_PORT --mig_cert_port=MIG_CERT_PORT --ext_cert_port=EXT_CERT_PORT --mig_oid_port=MIG_OID_PORT --ext_oid_port=EXT_OID_PORT --mig_oidc_port=MIG_OIDC_PORT --ext_oidc_port=EXT_OIDC_PORT --sid_port=SID_PORT --sftp_port=SFTP_PORT --sftp_show_port=SFTP_SHOW_PORT --sftp_subsys_port=SFTP_SUBSYS_PORT --sftp_subsys_show_port=SFTP_SUBSYS_SHOW_PORT --davs_port=DAVS_PORT --davs_show_port=DAVS_SHOW_PORT --ftps_ctrl_port=FTPS_CTRL_PORT --ftps_ctrl_show_port=FTPS_CTRL_SHOW_PORT --openid_port=OPENID_PORT --openid_show_port=OPENID_SHOW_PORT --openid_session_lifetime=OPENID_SESSION_LIFETIME --seafile_seahub_port=SEAFILE_SEAHUB_PORT --seafile_seafhttp_port=SEAFILE_SEAFHTTP_PORT --seafile_client_port=SEAFILE_CLIENT_PORT --seafile_quota=SEAFILE_QUOTA --auto_add_cert_user=AUTO_ADD_CERT_USER --auto_add_oid_user=AUTO_ADD_OID_USER --auto_add_oidc_user=AUTO_ADD_OIDC_USER --enable_migadmin=ENABLE_MIGADMIN --enable_sftp=ENABLE_SFTP --enable_sftp_subsys=ENABLE_SFTP_SUBSYS --enable_davs=ENABLE_DAVS --enable_ftps=ENABLE_FTPS --enable_wsgi=ENABLE_WSGI --enable_jobs=ENABLE_JOBS --enable_resources=ENABLE_RESOURCES --enable_workflows=ENABLE_WORKFLOWS --enable_events=ENABLE_EVENTS --enable_sharelinks=ENABLE_SHARELINKS --enable_transfers=ENABLE_TRANSFERS --enable_freeze=ENABLE_FREEZE --enable_sandboxes=ENABLE_SANDBOXES --enable_vmachines=ENABLE_VMACHINES --enable_preview=ENABLE_PREVIEW --enable_jupyter=ENABLE_JUPYTER --enable_cloud=ENABLE_CLOUD --enable_gdp=ENABLE_GDP --enable_hsts=ENABLE_HSTS --enable_vhost_certs=ENABLE_VHOST_CERTS --enable_verify_certs=ENABLE_VERIFY_CERTS --enable_seafile=ENABLE_SEAFILE --enable_duplicati=ENABLE_DUPLICATI --enable_crontab=ENABLE_CRONTAB --enable_notify=ENABLE_NOTIFY --enable_imnotify=ENABLE_IMNOTIFY --enable_dev_accounts=ENABLE_DEV_ACCOUNTS --enable_twofactor=ENABLE_TWOFACTOR --enable_twofactor_strict_address=ENABLE_TWOFACTOR_STRICT_ADDRESS --enable_peers=ENABLE_PEERS --peers_mandatory=PEERS_MANDATORY --enable_cracklib=ENABLE_CRACKLIB --enable_openid=ENABLE_OPENID --enable_gravatars=ENABLE_GRAVATARS --enable_sitestatus=ENABLE_SITESTATUS --daemon_pubkey_from_dns=DAEMON_PUBKEY_FROM_DNS --seafile_ro_access=SEAFILE_RO_ACCESS --public_use_https=PUBLIC_USE_HTTPS --prefer_python3=PREFER_PYTHON3 --io_account_expire=IO_ACCOUNT_EXPIRE --gdp_email_notify=GDP_EMAIL_NOTIFY

All those values can also be set via environment variable, by setting the corresponding MIG_X environment variable where X is the option name in upper case. That is, instead of passing --enable_transfers=True as argument one could set MIG_ENABLE_TRANSFERS=True in the environment. If the same option is set both as environment variable and CLI parameter, then the CLI parameter takes precedence.

For a server running MiG as the 'mig' user with the code checked out directly in the home directory and Debian apache 2.4 without OpenID but with full grid jobs and resources and efficient data access services: ./generateconfs.py --source=. --destination=generated-confs \ --base_fqdn=migrid.org \ --public_fqdn=www.migrid.org \ --mig_cert_fqdn=dk-cert.migrid.org \ --ext_cert_fqdn= \ --mig_oid_fqdn=dk-ext.migrid.org \ --ext_oid_fqdn=dk-oid.migrid.org \ --sid_fqdn=dk-sid.migrid.org \ --io_fqdn=dk-io.migrid.org \ --user=mig --group=mig \ --apache_version=2.4 \ --apache_etc=/etc/apache2 \ --apache_run=/var/run/apache2 \ --apache_lock=/var/lock/apache2 \ --apache_log=/var/log/apache2 \ --openssh_version=7.4 \ --mig_code=/home/mig/mig \ --mig_state=/home/mig/state \ --mig_certs=/etc/apache2/MiG-certificates \ --hg_path=/usr/bin/hg \ --hgweb_scripts=/usr/share/doc/mercurial-common/examples \ --trac_admin_path=/usr/bin/trac-admin \ --trac_ini_path=/home/mig/mig/server/trac.ini \ --public_http_port=80 --mig_cert_port=443 --mig_oid_port=443 \ --ext_oid_port=443 --sid_port=443 \ --enable_jobs=True --enable_resources=True \ --enable_ftps=True --enable_sftp_subsys=True \ --enable_webdavs=True --enable_transfers=True \ --enable_sandboxes=True --enable_vmachines=True \ --user_clause=User --group_clause=Group \ --listen_clause='#Listen' \ --serveralias_clause='ServerAlias' \ --signup_methods="migcert" \ --login_methods="migcert" \ --skin=migrid-basic \ --short_title=MiG

or a similar setup with vhost-specific certificates from LetsEncrypt, additional web apps and OpenID on CentOS: ./generateconfs.py --source=. --destination=generated-confs \ --destination_suffix="_svn$(svnversion -n ~/)" \ --base_fqdn=migrid.org \ --public_fqdn=www.migrid.org \ --public_alias_fqdn=dk-www.migrid.org \ --mig_cert_fqdn=dk-cert.migrid.org \ --ext_cert_fqdn= \ --mig_oid_fqdn=dk-ext.migrid.org \ --ext_oid_fqdn=dk-oid.migrid.org \ --sid_fqdn=dk-sid.migrid.org \ --io_fqdn=dk-io.migrid.org \ --daemon_show_address=dk-io.migrid.org \ --user=mig --group=mig \ --apache_version=2.4 \ --apache_etc=/etc/httpd \ --apache_run=/var/run/httpd \ --apache_lock=/var/lock/subsys/httpd \ --apache_log=/var/log/httpd \ --openssh_version=7.4 \ --mig_code=/home/mig/mig \ --mig_state=/home/mig/state \ --mig_certs=/etc/httpd/MiG-certificates \ --hg_path=/usr/bin/hg \ --hgweb_scripts=/usr/share/doc/mercurial-2.6.2 \ --trac_admin_path=/usr/bin/trac-admin \ --trac_ini_path=/home/mig/mig/server/trac.ini \ --public_http_port=80 --public_https_port=443 \ --ext_cert_port=443 --mig_oid_port=443 \ --ext_oid_port=443 --sid_port=443 \ --mig_oid_provider=https://dk-ext.migrid.org/openid/ \ --ext_oid_provider=https://openid.ku.dk/ \ --enable_openid=True --enable_sftp_subsys=True \ --enable_davs=True --enable_ftps=True \ --enable_sandboxes=True --enable_jobs=True \ --enable_resources=True --enable_notify=True \ --enable_events=True --enable_imnotify=True \ --enable_cracklib=True --enable_freeze=False \ --enable_transfers=True --enable_gravatars=True \ --enable_vhost_certs=True --enable_verify_certs=True \ --enable_migadmin=True --enable_peers=True \ --peers_mandatory=True --peers_explicit_fields='full_name email' \ --peers_contact_hint='employed at UCPH and authorized to invite external users' \ --user_clause=User --group_clause=Group \ --listen_clause='#Listen' \ --serveralias_clause='ServerAlias' --alias_field=email \ --dhparams_path=~/certs/dhparams.pem \ --daemon_keycert=~/certs/combined.pem \ --daemon_pubkey=~/certs/combined.pub \ --daemon_pubkey_from_dns=True \ --signup_methods="extoid migoid migcert" \ --login_methods="extoid migoid migcert" \ --distro=centos --skin=migrid-basic \ --collaboration_links="default advanced" \ --default_vgrid_links="files web" \ --advanced_vgrid_links="files web scm tracker workflows monitor" \ --smtp_sender="Do Not Reply no-reply@migrid.org" \ --support_email="MiGrid Support support@migrid.org" \ --admin_email="MiGrid Info info@migrid.org" --log_level=info \ --title="Minimum intrusion Grid" \ --short_title="MiG" \ --external_doc=https://www.migrid.org \ --mig_oid_title="Non-KU/UCPH" --ext_oid_title="KU/UCPH" \ --default_menu="home files submitjob jobs vgrids settings setup logout" \ --user_menu="sharelinks people cloud crontab transfers runtimeenvs resources peers downloads docs dashboard migadmin" \ --wsgi_procs=25 --sftp_subsys_auth_procs=20 \ --sftp_max_sessions=16 \ --auto_add_oid_user=True --auto_add_cert_user=True \ --auto_add_filter_fields=full_name --auto_add_filter_method=skip \ --io_account_expire=True \ --password_policy="MODERN:12" \ --password_legacy_policy=MEDIUM \ --peers_permit="role:.*(vip|tap)" \ --status_system_match="MiGrid ALL" \ --digest_salt="FILE::/home/mig/state/secrets/digest_salt.hex" \ --crypto_salt="FILE::/home/mig/state/secrets/crypto_salt.hex" \ --imnotify_address="FILE::/home/mig/state/secrets/imnotify_address.txt" \ --imnotify_channel="FILE::/home/mig/state/secrets/imnotify_channel.txt" \ --imnotify_username="FILE::/home/mig/state/secrets/imnotify_username.txt" \ --imnotify_password="FILE::/home/mig/state/secrets/imnotify_password.txt" \ --secscan_addr="130.226.158.3 130.225.213.72 192.38.10.137"

and a storage-only setup with CentOS 7.x, apache 2.4, WSGI (default web), optimized SFTP, WebDAVS FTPS, Data Transfers, external Seafile integration, local OpenID login and added Jupyter+cloud integration for data analysis: ./generateconfs.py --source=. --destination=generated-confs \ --destination_suffix="_svn$(svnversion -n ~/)" \ --base_fqdn=erda.dk \ --public_fqdn=www.erda.dk \ --public_alias_fqdn=www.erda.dk \ --public_sec_fqdn=erda.ku.dk \ --mig_cert_fqdn= \ --ext_cert_fqdn=cert.erda.dk \ --mig_oid_fqdn=ext.erda.dk \ --ext_oid_fqdn=erda.dk \ --sid_fqdn=sid.erda.dk \ --io_fqdn=io.erda.dk \ --seafile_fqdn=sid.erda.dk \ --daemon_show_address=io.erda.dk \ --sftp_show_port=22 --ftps_ctrl_show_port=21 \ --ftps_pasv_ports=8030:8100 \ --davs_show_port=443 --openid_show_port=443 \ --user=mig --group=mig \ --apache_version=2.4 \ --apache_etc=/etc/httpd \ --apache_run=/var/run/httpd \ --apache_lock=/var/lock/subsys/httpd \ --apache_log=/var/log/httpd \ --openssh_version=7.4 \ --mig_code=/home/mig/mig \ --mig_state=/home/mig/state \ --mig_certs=/etc/httpd/MiG-certificates \ --hg_path=/usr/bin/hg \ --hgweb_scripts=/usr/share/doc/mercurial-2.6.2 \ --trac_admin_path='' --trac_ini_path='' \ --public_http_port=80 --public_https_port=443 \ --ext_cert_port=443 --mig_oid_port=443 \ --ext_oid_port=443 --sid_port=443 \ --mig_oid_provider=https://ext.erda.dk/openid/ \ --ext_oid_provider=https://openid.ku.dk/ \ --enable_openid=True --enable_sftp_subsys=True \ --enable_davs=True --enable_ftps=True \ --enable_duplicati=True --enable_seafile=True \ --seafile_fqdn=seafile.erda.dk \ --seafile_ro_access=False --enable_cracklib=True \ --enable_transfers=True --enable_gravatars=True \ --enable_vhost_certs=True --enable_verify_certs=True \ --enable_notify=True --enable_jupyter=True \ --jupyter_services='DAG.https://dag002.science DAG.https://dag003.science DAG.https://dag004.science DAG.https://dag005.science DAG.https://dag006.science DAG.https://dag007.science DAG.https://dag008.science DAG.https://dag009.science DAG.https://dag010.science DAG.https://dag203.science DAG.https://dag204.science MODI.https://dag100.science' \ --jupyter_services_desc="{'DAG': '/home/mig/state/wwwpublic/dag_desc.html', 'MODI': '/home/mig/state/wwwpublic/modi_desc.html'}" \ --enable_cloud=True --enable_migadmin=True \ --enable_peers=True --peers_mandatory=True \ --peers_explicit_fields='full_name email' \ --peers_contact_hint='employed at UCPH and authorized to invite external users' \ --user_clause=User --group_clause=Group \ --listen_clause='#Listen' \ --serveralias_clause='#ServerAlias' --alias_field=email \ --dhparams_path=~/certs/dhparams.pem \ --daemon_keycert=~/certs/combined.pem \ --daemon_pubkey=~/certs/combined.pub \ --daemon_pubkey_from_dns=True \ --signup_methods="extoid migoid extcert" \ --login_methods="extoid migoid extcert" \ --distro=centos --skin=erda-ucph-science \ --vgrid_label=Workgroup --apache_worker_procs=2048 \ --davs_port=8020 --openid_port=8001 \ --wsgi_procs=100 --sftp_subsys_auth_procs=50 \ --sftp_max_sessions=16 \ --default_menu="home files vgrids archives jupyter settings setup logout" \ --user_menu="sharelinks seafile crontab transfers cloud people downloads peers docs migadmin" \ --collaboration_links="default advanced" \ --default_vgrid_links="files web" \ --advanced_vgrid_links="files web scm workflows monitor" \ --smtp_sender="Do Not Reply no-reply@erda.dk" \ --support_email="ERDA Support support@erda.dk" \ --admin_email="ERDA Info info@erda.dk" --log_level=info \ --title="University of Copenhagen - Electronic Research Data Archive" \ --short_title="UCPH ERDA" \ --external_doc=https://erda.ku.dk \ --mig_oid_title="Non-KU/UCPH" --ext_oid_title="KU/UCPH" \ --auto_add_oid_user=True --auto_add_cert_user=True \ --auto_add_filter_fields=full_name --auto_add_filter_method=skip \ --permanent_freeze="freeze phd backup" --freeze_to_tape="4w" \ --io_account_expire=True \ --password_policy="MODERN:12" \ --password_legacy_policy=MEDIUM \ --peers_permit="role:.(vip|tap)" \ --vgrid_creators="role:.(vip|tap)" \ --status_system_match="ERDA IDMC SIF ALL" \ --digest_salt="FILE::/home/mig/state/secrets/digest_salt.hex" \ --crypto_salt="FILE::/home/mig/state/secrets/crypto_salt.hex" \ --secscan_addr="130.226.158.3 130.225.213.72 192.38.10.137"

and a similar setup with CentOS 7.x, apache 2.4, WSGI (default web), optimized SFTP, WebDAVS, FTPS, job execution, Jupyter integration, previews and local OpenID login and support for legacy sftp clients: ./generateconfs.py --source=. --destination=generated-confs \ --destination_suffix="_svn$(svnversion -n ~/)" \ --base_fqdn=idmc.dk \ --public_fqdn=www.idmc.dk \ --mig_cert_fqdn= \ --ext_cert_fqdn=cert.idmc.dk \ --mig_oid_fqdn=ext.idmc.dk \ --ext_oid_fqdn=oid.idmc.dk \ --sid_fqdn=sid.idmc.dk \ --io_fqdn=io.idmc.dk \ --user=mig --group=mig \ --apache_version=2.4 \ --apache_etc=/etc/httpd \ --apache_run=/var/run/httpd \ --apache_lock=/var/lock/subsys/httpd \ --apache_log=/var/log/httpd \ --openssh_version=7.3 \ --mig_code=/home/mig/mig \ --mig_state=/home/mig/state \ --mig_certs=/etc/httpd/MiG-certificates \ --hg_path=/usr/bin/hg \ --hgweb_scripts=/usr/share/doc/mercurial-2.6.2 \ --trac_admin_path='' --trac_ini_path='' \ --public_http_port=80 --public_https_port=443 \ --ext_cert_port=443 --mig_oid_port=443 \ --ext_oid_port=443 --sid_port=443 \ --mig_oid_provider=https://ext.idmc.dk/openid/ \ --ext_oid_provider=https://openid.ku.dk/ \ --enable_openid=True --enable_sftp_subsys=True \ --enable_davs=True --enable_ftps=True \ --enable_transfers=True --enable_gravatars=True \ --enable_jobs=True --enable_resources=True \ --enable_events=True --enable_cracklib=True \ --enable_notify=True --enable_preview=True \ --enable_workflows=True --enable_freeze=False \ --enable_vhost_certs=True --enable_verify_certs=True \ --enable_jupyter=True --enable_migadmin=True \ --jupyter_services='DAG.https://dag002.science DAG.https://dag003.science DAG.https://dag004.science DAG.https://dag005.science DAG.https://dag006.science DAG.https://dag007.science DAG.https://dag008.science DAG.https://dag009.science DAG.https://dag010.science DAG.https://dag203.science DAG.https://dag204.science MODI.https://dag100.science' \ --jupyter_services_desc="{'DAG': '/home/mig/state/wwwpublic/dag_desc.html', 'MODI': '/home/mig/state/wwwpublic/modi_desc.html'}" \ --enable_peers=True --peers_mandatory=True \ --peers_explicit_fields='full_name email' \ --peers_contact_hint='employed at UCPH and authorized to invite external users' \ --user_clause=User --group_clause=Group \ --listen_clause='#Listen' \ --serveralias_clause='#ServerAlias' --alias_field=email \ --dhparams_path=~/certs/dhparams.pem \ --daemon_keycert=~/certs/combined.pem \ --daemon_pubkey=~/certs/combined.pub \ --daemon_pubkey_from_dns=False \ --daemon_show_address=io.idmc.dk \ --signup_methods="extoid migoid extcert" \ --login_methods="extoid migoid extcert" \ --distro=centos --skin=idmc-basic \ --vgrid_label=Workgroup --apache_worker_procs=512 \ --wsgi_procs=25 --sftp_subsys_auth_procs=25 \ --sftp_max_sessions=16 \ --davs_port=8020 --openid_port=8001 \ --default_menu="home files submitjob jobs vgrids jupyter settings setup logout" \ --user_menu="sharelinks people cloud crontab transfers runtimeenvs resources downloads peers docs migadmin" \ --collaboration_links="default advanced" \ --default_vgrid_links="files web" \ --advanced_vgrid_links="files web scm workflows monitor" \ --smtp_sender="Do Not Reply no-reply@idmc.dk" \ --support_email="IDMC Support support@idmc.dk" \ --admin_email="IDMC Info info@idmc.dk" --log_level=info \ --title="Imaging Data Management Center" \ --short_title="IDMC" \ --external_doc=https://www.idmc.dk \ --mig_oid_title="Non-KU/UCPH" --ext_oid_title="KU/UCPH" \ --auto_add_oid_user=True --auto_add_cert_user=True \ --auto_add_filter_fields=full_name --auto_add_filter_method=skip \ --io_account_expire=True \ --password_policy="MODERN:12" \ --password_legacy_policy=MEDIUM \ --peers_permit="role:.(vip|tap)" \ --vgrid_creators="role:.(vip|tap)" \ --status_system_match="IDMC ERDA ALL" \ --digest_salt="FILE::/home/mig/state/secrets/digest_salt.hex" \ --crypto_salt="FILE::/home/mig/state/secrets/crypto_salt.hex" \ --secscan_addr="130.226.158.3 130.225.213.72 192.38.10.137"

Finally a storage-only with CentOS 7.x, apache 2.4, WSGI (default web), optimized SFTP, WebDAVS, strict access control and extensive logging to comply with the General Data Protection Regulation (GDPR) imposed by EU: https://en.wikipedia.org/wiki/General_Data_Protection_Regulation ./generateconfs.py --source=. --destination=generated-confs \ --destination_suffix="_svn$(svnversion -n ~/)" \ --support_email="SIF Support support@sif.erda.dk" \ --admin_email="SIF admin info@sif.erda.dk" --admin_list="/C=DK/ST=NA/L=NA/O=NBI/OU=NA/CN=Jonas Bardino/emailAddress=bardino@nbi.ku.dk , C=DK/ST=NA/L=NA/O=NBI/OU=NA/CN=Martin Rehr/emailAddress=rehr@nbi.ku.dk" \ --auto_add_cert_user=False \ --auto_add_oid_user=True \ --auto_add_oidc_user=False \ --auto_add_filter_fields=full_name --auto_add_filter_method=skip \ --oid_valid_days=180 \ --base_fqdn=sif.erda.dk \ --public_fqdn=sif-www.erda.dk \ --public_alias_fqdn=sif.ku.dk \ --public_sec_fqdn=sif.ku.dk \ --public_use_https=True \ --mig_cert_fqdn= \ --ext_cert_fqdn= \ --mig_oid_fqdn=sif-ext.erda.dk \ --ext_oid_fqdn=sif.erda.dk \ --sid_fqdn=sif-sid.erda.dk \ --io_fqdn=sif-io.erda.dk \ --user=mig --group=mig \ --apache_version=2.4 \ --apache_etc=/etc/httpd \ --apache_run=/var/run/httpd \ --apache_lock=/var/lock/subsys/httpd \ --apache_log=/var/log/httpd \ --openssh_version=7.4 \ --mig_code=/home/mig/mig \ --mig_state=/home/mig/state \ --mig_certs=/etc/httpd/MiG-certificates \ --hg_path='' \ --hgweb_scripts='' \ --trac_admin_path='' --trac_ini_path='' \ --public_http_port=80 --public_https_port=443 \ --ext_cert_port=443 --mig_oid_port=443 \ --ext_oid_port=443 --sid_port=443 \ --mig_oid_provider=https://sif-ext.erda.dk/openid/ \ --ext_oid_provider=https://openid.ku.dk/ \ --enable_openid=True \ --enable_sftp=True --enable_sftp_subsys=False \ --enable_davs=True --enable_ftps=False \ --enable_sharelinks=False --enable_transfers=False \ --enable_duplicati=False --enable_seafile=False \ --enable_sandboxes=False --enable_vmachines=False \ --enable_crontab=False --enable_jobs=False \ --enable_resources=False --enable_events=False \ --enable_freeze=False --enable_preview=False \ --enable_gdp=True --gdp_email_notify=True \ --enable_notify=True --enable_twofactor=True \ --enable_twofactor_strict_address=True \ --enable_cracklib=True --enable_hsts=True \ --enable_vhost_certs=True --enable_verify_certs=True \ --enable_migadmin=False --enable_peers=True \ --peers_mandatory=True --peers_explicit_fields='full_name email' \ --peers_contact_hint='employed at UCPH and authorized to invite external users' \ --user_clause=User --group_clause=Group \ --listen_clause='#Listen' \ --serveralias_clause='#ServerAlias' --alias_field=email \ --dhparams_path=~/certs/dhparams.pem \ --daemon_keycert=~/certs/combined.pem \ --daemon_pubkey=~/certs/combined.pub \ --daemon_pubkey_from_dns=True \ --daemon_show_address=sif-io.erda.dk \ --signup_methods="extoid migoid" \ --login_methods="extoid migoid" \ --password_policy=MODERN:12 --password_legacy_policy=HIGH \ --distro=centos --skin=sif-ucph-science \ --title="Sensitive Information Facility" \ --short_title="SIF" \ --external_doc=https://sif.ku.dk \ --mig_oid_title="External" \ --ext_oid_title="KU/UCPH" \ --vgrid_label="Project" \ --vgrid_creators="role:.*(vip|tap)" \ --csrf_protection="FULL" \ --io_account_expire=True \ --digest_salt="FILE::/home/mig/state/secrets/digest_salt.hex" \ --crypto_salt="FILE::/home/mig/state/secrets/crypto_salt.hex" \ --wsgi_procs=25 --user_interface='V2' \ --default_menu="files setup close logout" \ --secscan_addr="130.226.158.3 130.225.213.72 192.38.10.137"

Most of the arguments should be relatively straight forward, but you need to provide the MIG_CERTS path where your apache server key and certificates are available along with optional MiG x509 server certificates (used for MiG server to server communication). The actual keys and certificates can be added later, so you can just choose a suitable directory path at first. In practice one can purchase server certificates or generate them with LetsEncrypt.

The hg and trac path pairs are optional and can be set to the empty string if mercurial/trac is not available or if VGrid wikis, SCMs and trackers should simply not be enabled. If you want VGrid trackers including mercurial integration, but don't want the direct VGrid SCM links, you can set the trac_X and hg_X options but leave out the scm entry in the ordered list of vgrid_links in the SITE section. The same procedure applies for visibility of the other VGrid components.

Similarly the mercurial package provides all required components for VGrid SCMs on Debian/Ubuntu or RedHat/CentOS. The same applies for the trac and trac-mercurial packages. Paramiko is required for the optional grid_sftp daemon to work and the python-paramiko package provides all required components for it on recent Debian/Ubuntu. You want to use 1.15+ because it added significant performance improvements. Just pull it in from pip if your distribution only provides an older version. In case you want to run a high-performance SFTP frontend you can combine OpenSSH with the same paramiko SFTP backend through the use of the sftp_subsys.py module. You need to compile and configure the PAM and NSS modules in mig/src/libpam-mig and mig/src/libnss-mig as described in the README files there. Benchmarks show that it performs and scales far better to multiple clients. Python FTP server library (pyftpdlib) in a recent version (1.x+) is required for the optional grid_ftps daemon to work and the python-pyftpdlib package provides all required components for it on recent Debian/Ubuntu. If no recent version is available, it can still easily be installed with pip instead: pip install pyftpdlib Please note that a version before 2.x is needed for python2 support so the pip command would in that case be: pip install 'pyftpdlib<2.0' WsgiDAV is required for the optional grid_webdavs daemon to work and a recent version is needed for full OSX client support. Thus it is recommended to install directly from github or with pip. All optional file server services like sftp, ftps and davs rely on one of the python pbkdf2 modules for password auth support and it is distributed with the MiG code base.

The four CLAUSE arguments can be used to comment out the explicit setting of user, group, serveralias and ports in the apache conf by providing a '# User', '# Group', '# ServerAlias' and '# Listen'. This is mostly relevant if using apache2 with WSGI.

The generator will inform you about the steps to install your configuration files in the right locations.

== Running a MiG Server == Before you run the MiG daemons you need to have a working configuration for your daemons in mig/server/MiGserver.conf or another location you can specify in the MIG_CONF environment variable. Please note that if you want to use this environment variable, it must be available to all MiG components to work. You can use the generator as mentioned above or manually modify e.g. the localhost example configuration in MiGserver-localhost.conf. At any time you can verify the validity of your configuration with the checkconf.py script in the same directory.

Once set up you can use the provided init script to manage all services. It is also possible to run the services individually as explained below.

In full grid compute setups the central daemon is grid_script.py, which takes care of all job management on the server. If you want to include grid monitor web pages you should additionally run the grid_monitor.py daemon. The optional job notifications and ssh multiplexing daemons are available as grid_imnotify.py / grid_imnotify_stdout.py and grid_sshmux.py in the same location. In case you don't know what they do, you can most likely safely ignore them and just run the grid_script.py daemon. In storage-centric setups the job handling can be completely disabled and grid_script left not running.

All the daemons and scripts can be launched from their unpacked location as long as python knows where the package is hosted e.g. as in: export PYTHONPATH=/path/to/unpacked/mig/source python /path/to/unpacked/mig/source/mig/server/grid_script.py

They rely on the conf file in the mig/server subdir but can be told to look for it in another locations provided in the environment: export MIG_CONF="/path/to/MiGserver.conf" python /path/to/unpacked/mig/source/mig/server/grid_script.py

Each daemon will keep running until you actively stop it, so you probably need individual shell sessions for each daemon.

For testing purposes this interactive execution is fine, but in more permanent setups you will probably want to run the MiG daemons as true daemons so that you can disconnect from the server and leave them running. One solution is to run the daemons inside a GNU Screen session: screen -S MiG python /path/to/unpacked/mig/source/mig/server/grid_script.py [ctrl-a d to disconnect]

Then you can disconnect from the server and resume the session any time later by reattaching the screen session: screen -S MiG -R

Please refer to 'man screen' or other screen documentation for further details.

For production use we also provide a classic init script, useful for managing the services completely as background daemons. It is included as a part of the configuration generator output and can be copied/moved into place from there. When located in /etc/init.d it's easy to manage all enabled services as one or each individual one as described below: /etc/init.d/migrid Usage: migrid {start|stop|status|restart|reload}[daemon DAEMON] where daemon is left out for all or given along with DAEMON as one of the following (script|monitor|sshmux|events|cron|transfers|openid|sftp|sftpsubsys|webdavs|ftps|notify|imnotify|vmproxy|all)

== Adding users == You need a either an OpenID 2.0, OpenID Connect login or a MiG certificate+key to fully interact with any MiG server as a user.

For local username+password logins you can run the built-in OpenID 2.0 service and have users register there (https://SERVER/cgi-sid/reqoid.py) to combine login and account creation. You can also configure login with any similar remote OpenID 2.0 or Connect service as long as it offers the required information for sign up and login. We do that to ease user management on a number of sites with the automatic signup backend (https://SERVER/wsgi-bin/autocreate.py).

If you want certificate login but don't want to set up your own Certificate Authority (CA) you might be able to use e.g. our certificates. Please use the certificate request link from https://www.migrid.org if you haven't got a certificate yet.

If you run your own CA you can simply use the certificate request mechanism included in MiG (https://SERVER/cgi-sid/reqcert.py) to combine certificate and user creation. Certificate requests will automatically result in an email with full certificate and MiG user creation instructions to the configured MiG admins. Otherwise you can use the external certificate sign up request mechanism included in MiG (https://SERVER/cgi-sid/extcert.py).

Finally you can simple run the MiG user creation or import commands directly as described below.

To manually add a user to your MiG server you need to have the account fields at hand and run the createuser script:

cd ~/mig/server ./createuser.py

You will be prompted for user details one by one before the user is added to the local MiG user database. Any user added to this database can access your MiG server and manage his/her MiG jobs and files. The user must present a MiG certificate with the exact same Distinguished Name or login through OpenID with the associated ID to get access, however. If you do not use the MiG CA or another CA with the same Distinguished Name format (/C=./ST=./O=./CN=./emailAddress=.*) you have to supply the -i DN option for the user to work.

Example: adding myself as a user on a MiG server:

./createuser.py

Please enter the details for the new user: Full Name: Jonas Bardino Organization: NBI State: 2-letter Country Code: DK Email: bardino@nbi.ku.dk Comment: This is my own MiG user Password: using user dict: {'comment': 'This is my own MiG user', 'country': 'DK', 'state': '', 'full_name': 'Jonas Bardino', 'organization': 'NBI', 'password': '*****', 'email': 'bardino@nbi.ku.dk'} logging to: server.log ; level: info Creating dirs and files for new user: Jonas Bardino User name without spaces: Jonas_Bardino

User Jonas Bardino was successfully added to user DB! DB entry and dirs for Jonas Bardino were created or updated

My MiG certificate with Full Name Jonas Bardino and so on will now give me access to this development server and any others where the MiG CA is configured. You can find the certificate field details using openssl or by viewing the certificate imported in a browser.

Users can also be mass created e.g. from a csv file with our importusers helper: cd ~/mig/server python parsecsvusers.py add-users.csv > add-users.list ./importusers.py -v -p AUTO add-users.list

The csv file must have a header line to specify the field layout and then one line per user using that layout. It could look something like this:

full_name;organization;email;country

Jonas Bardino;NBI;bardino@nbi.ku.dk;DK Martin Garrigues Rehr;NBI;rehr@nbi.ku.dk;DK ...

== Adding resources == When you have added yourself as a user on your MiG server, you can open your personal Resources page on the corresponding web interface and add resources of all kinds there. Please refer to the wiki pages online for explanations on each kind of resource and some examples of setups.

== Stopping a MiG server == All the daemons can be stopped with ctrl-c and most also support a SHUTDOWN message through the named input pipe defined in the configuration file: echo SHUTDOWN >> /path/to/server.stdin

To completely stop MiG you need to stop all the MiG daemons and the apache server.

We provide migrid service scripts for easy integration on the common Linux distributions. In that way management of the services is just a matter of sudo service migrid ACTION and individual daemons can be controlled with: sudo service migrid ACTIONdaemon DAEMON

= Uninstalling MiG = The default server configuration template keeps all MiG files installed under the single directory where the MiG source code is unpacked, so uninstalling is simply a matter of deleting that directory. If you change your server configuration to save e.g. state files outside this directory you will have to manually clean up those directories as well to completely uninstall MiG.

ucphhpc / migrid-sync

readme