Code, Data, and Checkpoint For Unleashing the Power of Knowledge Extraction from Scientific Literature in Catalysis
Download dataset from zenodo and follow the dataset instruction to extract the data
The reproduce.ipynb
notebook provides instructions to reproduce our main expriment result using ALL data
The train.ipynb
notebook gives examples of how to train your own model on your own data set
The prediction.ipynb
notebook gives examples of how to use our model to extract catalysis information on given text
We are sorry that we cannot make our search engine and correlation analysis system public since some articles in our collection are not open-accessed. Sharing those is against Elsevier's TDM policy. But we provide the key piece of code of our system, which should help you build a similar system on you own data. If you want to try our system personally, please get in touch with us.
streamlit_correlation.py
)Here we show you how we search query related articles from our highly relevant article collection
follow the Guide to indexing and searching English documents
section at pyserini's guideline
Here we store the metadata of each paper in MongoDB and use Pymongo to access it. Feel free change load_db
and fetch_paper
function to use other metadata storage.
streamlit_correlation.py
)Here we show you how we analyze the correlation between entities
doi
,sent_i
,label
,span
and norm_span
five columns. It should covers all extracted chemical entities, their index(which paper, which sentence), their label(which type of entity), their normalization form(mainly to aggregate alternative expressions)Ru on C
returns no result, since Ru on C
will be normlized into Ru C
. So searching Ru C
has result while searching Ru on C
has no result. One alternative is to search the raw entity, but in this way, the co-occurance patter will be greatly weakened by various alternative expression. So we decided to design the system in current form. In the furture, we may have better normalization techniques to solve this issue. Please create an issue or email to zhangyue@udel.edu should you have any questions about the code and data.