appjsonify
: An Academic Paper PDF-to-JSON Conversion Toolkitappjsonify
[^1] is a handy PDF-to-JSON conversion tool for academic papers implemented in Python.
appjsonify
allows you to obtain a structured JSON file that can be easily used for various downstream tasks such as paper recommendation, information extraction, and information retrieval from papers.
[^1]: Academic Paper PDF jsonify
Please manually install it based on the instructions.
If your environment does not have poppler
, please install it.
This is necessary to obtain PDF images using pdf2image
.
For more details, refer to Prerequisites.
pip install appjsonify
python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'
git clone https://github.com/hitachi-nlp/appjsonify.git
python -m pip install --editable .
python -m pip install 'git+https://github.com/facebookresearch/detectron2.git'
appjsonify
offers two options to structure your paper PDF file into a JSON file.
AAAI
, ACL
, ICML
, ICLR
, NeurIPS
, IEEE
, ACM
, or Springer
styles. See Templates for more details.appjsonify
provides two types of the templates for each of the following paper types: AAAI
, ACL
, ICML
, ICLR
, NeurIPS
, IEEE
, ACM
, and Springer
.
One is more accurate but slower due to the use of machine learning based models, the other is less accurate but faster due to its rule based approach.
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type AAAI
If your environment has a GPU(s), it is better to also specify
--detectron_device_mode cuda
to speed up the process.
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type AAAI2
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type ACL
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type ACL2
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type ICML
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type ICML2
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type ICLR
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type ICLR2
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type NeurIPS
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type NeurIPS2
Currently only tested with IEEE BigData papers.
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type IEEE
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type IEEE2
Currently only tested with TALLIP papers.
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type ACM
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type ACM2
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type Springer
appjsonify /path/to/pdf/dir/or/path /path/to/output/dir --paper_type Springer2
--verbose
: If you want to check the intermediate processing results, please set this flag. The log files will be saved under output_dir
. Optionally, you can use the following four flags to add the corresponding information.
--show_pos
: Bounding box information.--show_font
: Font name and size information.--show_style
: Style information (e.g., section
, body
, abstract
, etc.)--show_meta
: Supplementary information (e.g., information on objects and footnotes.)--insert_page_break
: Insert breaks between pages.--save_image
: If you are using a more accurate but slower version of templates or load_objects_with_ml
, appjsonify
can save detected table and figure images if this flag is set. In addition to this, please also specify the output directory path as --output_image_dir
. appjsonify
also allows users to build their own academic paper PDF-to-JSON processing pipeline.
For more details, please refer to Available Modules and Document Handling in appjsonify
.
Users can add their own modules to appjsonify
for more flexible document processing.
To add modules, appjsonify
must be installed in editable mode.
See Customize appjsonify
for more details and feel free to make a PR if you wish to add your module to this repository and package!
Contributions are more than welcome! Feel free to raise an issue and/or make a PR. Possible future work is as follows:
If you use appjsonify
in your work, please cite the following.
@article{yamaguchi2023appjsonify,
title={appjsonify: An Academic Paper PDF-to-JSON Conversion Toolkit},
author={Atsuki Yamaguchi and Terufumi Morishita},
year={2023},
eprint={2310.01206},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
© 2023 Atsuki Yamaguchi and Terufumi Morishita (Hitachi, Ltd.)
This work is licensed under the MIT license unless specified.
appjsonify
uses the follwoing publicly available works.