User
Authentication
Registration
New password
Generate API access token
Admin
Confirm registrations
Set a user's document conversion rate
Delete user
Edit User
System log viewer
Manager
Receives conversion jobs
Job list
Handles job distribution to queues
PathFinder
DocxConversion
WpPdfConversion
NlmxmlConversion
Cermine
Grobid
ReferenceConversion
BibtexConversion
BibtexreferenceConversion
HtmlConversion
CitationStyleConversion
PdfConversion
XmpConversion
EpubConversion
MergeXMLOutputs
ZipConversion
API
Apache mod_headers needs to be installed and enabled
meTypeset needs the python lxml library
Java VM needs to be installed
sendmail needs to be installed and configured
zip and unzip need to be installed, and I have to mention this because they aren’t by default on Ubuntu for some reason.
citation parsing has a variety of requirements, as listed in the ParsCit documentation. At minimum, you need Perl XML Twig and XML Writer, a working Ruby installation, and CRF++ binaries.
xml2bib needs to be installed
Pandoc, pandoc-citeproc, and libghc-citeproc-hs-data need to be installed
The XMP conversion needs Exiftool to be installed
The docX conversion needs LibreOffice with unoconv installed. The server is tested to work with LibreOffice 4.2.4 through 5.x.
The PHP5 xsl and curl modules must be installed.
# git clone https://github.com/pkp/xmlps.git # cd xmlps
# php composer.phar self-update # php composer.phar install
Ensure there is a MySQL database called xmlps
, and a MySQL user with access to it.
Configure the environment:
Copy the local.php
template, then edit it:
# cp config/autoload/local.php.dist config/autoload/local.php
Provide the MySQL user and password.
If you want debug messages to be displayed to the user (i.e., if developing locally), set these options:
'view_manager' => array(
'display_not_found_reason' => true,
'display_exceptions' => true,
),
If using a version of unoconv
installed via package manager (such as apt), you may need to override the command, e.g.:
'conversion' => array(
'docx' => array(
'unoconv' => array(
'command' => 'unoconv',
),
),
),
Make sure the cache directories are writable by your Web server process, and empty the cache directory, e.g:
# chmod -R go+w var # rm var/cache/zfcache-ea/*
You need to manually set the path to the MITIE libraries and the ner_stream binary from https://github.com/mit-nlp/MITIE as in https://github.com/pkp/xmlps/commit/63963bfc75f0ebb3be2bfcfca44865a15700b5e0 in order for the Named Entity Recognition module to work.
Initialize the database (you may need to manually create an 'xmlps' database first)
# vendor/doctrine/doctrine-module/bin/doctrine-module orm:schema-tool:update --force
Input an email address for CrossRef's API in https://github.com/pkp/ots/blob/master/config/autoload/global.php#L112
Run the shell script that starts the conversion queues
# ./start_queues.sh
<VirtualHost *:80>
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html/public
<Directory />
Options FollowSymLinks
AllowOverride All
</Directory>
<Directory /var/www/html/public>
Options -Indexes +FollowSymLinks +MultiViews
AllowOverride All
Order allow,deny
allow from all
</Directory>
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory "/usr/lib/cgi-bin">
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/access.log combined
Alias /doc/ "/usr/share/doc/"
<Directory "/usr/share/doc/">
Options Indexes MultiViews FollowSymLinks
AllowOverride None
Order deny,allow
Deny from all
Allow from 127.0.0.0/255.0.0.0 ::1/128
</Directory>
</VirtualHost>
Note that you will probably want to make sure that port 8080
is blocked from non-localhost connections so that external submissions can not be made directly to the Grobid module service.
After a successful installation the unit tests should complete without errors:
# ./unittest.sh
You will need to re-empty the cache directories afterward, if your Web server runs as a different user than you.
SASS compilation, CSS and Javascript compression & unification is done using Guard (http://guardgem.org)
After making changes to Javascript (javascript/
) or style files (style/scss/
) recompile/recompress the style and Javascript files by running
# guard
There is a simple REST API available to submit, view and retrieve jobs from/to the server.
Submit
Submit a job to the server. The citationStyleHash
is an internal identifier for the requested citaton style. A list of hashes can be retrieved through the citationStyleList API. The API will return the job id which can be used to retrieve the completed job later or to query the server for the job status.
URL: api/job/submit
Request type: POST
Parameters:
email
access_token
fileName
fileContent
citationStyleHash
fileMetadata
E.g.:
http://example.com/api/job/submit
POST parameters:
'email' => 'user@example.com'
'access_token' => 'access_token'
'fileName' => 'document.docx'
'citationStyleHash' => 'c6de5efe3294b26391ea343053c19a84',
'fileContent' => '...'
'fileMetadata' => *OPTIONAL* known good metadata like https://raw.githubusercontent.com/pkp/xmlps/master/module/MergeXMLOutputs/test/assets/metadata.json
Example response:
{"status":"success","id":123}
Status
Returns the current status for a job. Only completed jobs can be retrieved from the server. A full list of statuses can be found here.
URL: api/job/status
Request type: GET
Parameters:
email
access_token
id
E.g.:
http://example.com/api/job/status?email=user@example.com&access_token=access_token&id=123
Example response:
{"status":"success","jobStatus":0,"jobStatusDescription":"Pending"}
Citation Style List
Returns a list of available citation styles and their internal ids. We support all citation styles from citationstyles.org.
URL: api/job/citationStyleList
Request type: GET
E.g.:
http://example.com/api/job/citationStyleList
Example response:
{"status":"success","citationStyles":{"c6de5efe3294b26391ea343053c19a84":"ACM SIG Proceedings (\u0022et al.\u0022 for 15+ authors)"...
Retrieve
Retrieve a converted document. The jobConversionStage parameter specifies which type of conversion you want to get returned. A full list of conversion stages can be found here. The "final" XML produced by our pipeline is stage 20.
URL: api/job/retrieve
Request type: GET
Parameters:
email
access_token
id
conversionStage
E.g.:
http://example.com/api/job/retrieve?email=user@example.com&access_token=access_token&id=123&conversionStage=10
Example response:
The requested document or a JSON string with an error message.