search

petermr / tigr2ess

Materials for TIGR2ESS workshop in Delhi Feb 2019 - joint UK(Cambridge) - India project on Food Security.
Other
4 stars 10 forks source link

"Test run status of ami-search-cooccur for ami20190218b" #54

Closed ambarishK closed 5 years ago

ambarishK commented 5 years ago

Test run for the ami-search-cooccur is not successful.

It raises run-time exception because of not being able to get metadata file ( most probably eupmc_results.json file).

As an output it could not get cooccurrence results.

Run time log

ambarish123@ubuntu:~$ ami-search-cooccur 18febricenew/ country drugs plantparts species 
0    [main] DEBUG org.contentmine.ami.AMIProcessor  - project name: 18febricenew/ /home/ambarish123
1546 [main] DEBUG org.contentmine.ami.plugins.CommandProcessor  - running NORMA -i fulltext.xml -o scholarly.html --transform nlm2html --project /home/ambarish123/18febricenew
PMC6331594 .PMC6337123 PMC6339371 PMC6342930 PMC6343365 PMC6343895 PMC6345848 PMC6351596 PMC6352273 PMC6357162 running: word; word([frequencies])[{xpath:@count>20}, {w.stopwords:pmcstop.txt stopwords.txt}]
.filter: word([frequencies])[{xpath:@count>20}, {w.stopwords:pmcstop.txt stopwords.txt}]
.summary: word([frequencies])[{xpath:@count>20}, {w.stopwords:pmcstop.txt stopwords.txt}]
.running: search; search([country])[]
.filter: search([country])[]
.summary: search([country])[]
.running: search; search([drugs])[]
.filter: search([drugs])[]
.summary: search([drugs])[]
.running: search; search([plantparts])[]
.filter: search([plantparts])[]
.summary: search([plantparts])[]
.running: species; species([binomial])[]
SP: /home/ambarish123/18febricenew.filter: species([binomial])[]
.summary: species([binomial])[]
.create data tables
Exception in thread "main" java.lang.RuntimeException: Null metadataByTreename
    at org.contentmine.cproject.util.DataTablesTool.setMetadataByTreename(DataTablesTool.java:463)
    at org.contentmine.ami.plugins.CommandProcessor.createDataTables(CommandProcessor.java:210)
    at org.contentmine.ami.AMIProcessor.runSearches(AMIProcessor.java:116)
    at org.contentmine.ami.AMIProcessor.runSearchesAndCooccurrence(AMIProcessor.java:191)
    at org.contentmine.ami.tools.AMISearch.runAMISearches(AMISearch.java:47)
    at org.contentmine.ami.tools.AMISearch.main(AMISearch.java:28)
ambarishK commented 5 years ago

Test run of ami-search-new for ami20190218b is successful.

It generates all cooccurences and entries.DataTables.xml file containing bibliographic information into it's second column.

Run time log

ambarish123@ubuntu:~$ ami-search-new  -p 18febricenew/ --dictionary country drugs plantparts species 

Generic values (AMISearchTool)
================================
basename            null
cproject            /home/ambarish123/18febricenew
ctree               
cTreeList           [18febricenew/PMC6331594, 18febricenew/PMC6337123, 18febricenew/PMC6339371, 18febricenew/PMC6342930, 18febricenew/PMC6343365, 18febricenew/PMC6343895, 18febricenew/PMC6345848, 18febricenew/PMC6351596, 18febricenew/PMC6352273, 18febricenew/PMC6357162]
dryrun              false
excludeBase         null
excludeTrees        null
file types          []
forceMake           false
includeBase         null
includeTrees        null
log4j               
logfile             null
verbose             0

Specific values (AMISearchTool)
================================
dictionaryList       [country, drugs, plantparts, species]
dictionaryTop        null
dictionarySuffix     [xml]
ignorePlugins        []

cProject: 18febricenew
0    [main] DEBUG org.contentmine.ami.tools.AMISearchTool  - cannot find builtin dictionary: country
cannot find dictionary: country
10   [main] DEBUG org.contentmine.ami.tools.AMISearchTool  - cannot find builtin dictionary: drugs
cannot find dictionary: drugs
14   [main] DEBUG org.contentmine.ami.tools.AMISearchTool  - cannot find builtin dictionary: plantparts
cannot find dictionary: plantparts
15   [main] DEBUG org.contentmine.ami.tools.AMISearchTool  - SEARCH running legacy processors
SEARCH running JSON bibliography
running: word; word([frequencies])[{xpath:@count>20}, {w.stopwords:pmcstop.txt stopwords.txt}]
.filter: word([frequencies])[{xpath:@count>20}, {w.stopwords:pmcstop.txt stopwords.txt}]
.summary: word([frequencies])[{xpath:@count>20}, {w.stopwords:pmcstop.txt stopwords.txt}]
.running: search; search([country])[]
.filter: search([country])[]
.summary: search([country])[]
.running: search; search([drugs])[]
.filter: search([drugs])[]
.summary: search([drugs])[]
.running: search; search([plantparts])[]
.filter: search([plantparts])[]
.summary: search([plantparts])[]
.running: species; species([binomial])[]
SP: 18febricenew.filter: species([binomial])[]
.summary: species([binomial])[]
.create data tables

Message about the built-in dictionaries persists.

 0    [main] DEBUG org.contentmine.ami.tools.AMISearchTool  - cannot find builtin dictionary: country
cannot find dictionary: country
10   [main] DEBUG org.contentmine.ami.tools.AMISearchTool  - cannot find builtin dictionary: drugs
cannot find dictionary: drugs
14   [main] DEBUG org.contentmine.ami.tools.AMISearchTool  - cannot find builtin dictionary: plantparts cannot find dictionary: plantparts

Screenshot of entries.DataTables.html

bibliography

ambarishK commented 5 years ago

Test run of the ami-dictionary is successful.

Run time log

ambarish123@ubuntu:~$ ami-dictionary create --input https://en.wikipedia.org/wiki/Ocimum --dictionary Ocimum_xml --informat wikipage --outformats xml --directory Ocimum_xml/

Generic values (AMIDictionaryTool)
================================
basename            null
cproject            
ctree               
cTreeList           null
dryrun              false
excludeBase         null
excludeTrees        null
file types          []
forceMake           false
includeBase         null
includeTrees        null
log4j               
logfile             null
verbose             0

Specific values (AMIDictionaryTool)
================================
dataCols      null
dictionary    [Ocimum_xml]
dictionaryTop     Ocimum_xml/
href          null
hrefCols      null
input         https://en.wikipedia.org/wiki/Ocimum
informat      wikipage
dictInformat  null
linkCol       null
log4j         null
nameCol       null
operation     create
outformats    [xml]
splitCol      ,
termCol       null
terms         null
wikiLinks     [wikipedia, wikidata]
0    [main] DEBUG org.contentmine.ami.tools.AMIDictionaryTool  - extracting hyperlinks
............!....!!.....!.!......!...!!.!.!!.!!!!!!!!!!!!!!!!!.!!!!.!!!!!.!!!!!!!!!!!!!!!!!.!!!!!!!.....!.!..........!...!. !WP  !WP  !WP ++++++ !WP ++ !WP ++++++++ !WP  !WP +++++++ !WP ++++ !WP  !WP + !WP  !WP + !WP + !WP  !WP + !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP + !WP  !WP  !WP  !WP + !WP  !WP  !WP  !WP  !WP + !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP  !WP + !WP  !WP  !WP  !WP  !WP  !WP  !WP ++++++++++++++++writing dictionary to Ocimum_xml/Ocimum_xml.xml

Missing wikipedia: :

output is xml formatted dictionary file.

petermr commented 5 years ago

Thank you, well done. But please keep different problems (search and dictionary) in separate issues