getpapers "exact phrase" queries not working.

[EmanuelFaria]

@petermr @deadlyvices ISSUE: getpapers is not respecting quotes for exact phrase searches (i.e. "essential oil" OR "essential oils"). (See TABLE of side-by-side tests below). (UPDATE: It is also not respecting NEAR operator (essential NEAR/1 oil)"

The query I desire should (according to EUPMC) net me 488,095 Free to read & use results. Getpapers returns just 37

What I found was that EUPMC allows me to string these together using only quotes, as above, but getpapers returned errors unless I also put round brackets around each quoted term — ("essential oil") OR ("essential oils") — but getpapers still will not treat them as exact terms.

METHOD TO REPRODUCE: In crafting a getpapers search query, I started by testing variations of multi-word query terms (a term with two or more words separated by a hyphen or space), to make sure it actually made the difference I expected. The following table is the results (SCROLL RIGHT TO SEE ALL TABLE ENTRIES, Scroll down to see all the queries)

	EUPMC (DESIRED QUERY)	getpapers (DESIRED QUERY)	EUPMC (quotes removed)	getpapers (Quotes Removed)	EUPMC	getpapers	EUPMC	getpapers	EUPMC	getpapers	EUPMC	getpapers	EUPMC	getpapers	EUPMC	getpapers
Results	1,594,159 results Free to read (860,541) Free to read & use (488,095)	37 results	3,115,207 results Free to read (1,975,960) Free to read & use (1,080,006)	3115208 results	160165 results Free to read (137,660) Free to read & use (79,119)	160165 results	160,022 results Free to read (137,517) Free to read & use (78,974)	160022 results	159,990 results Free to read (137,466) Free to read & use (78,924)	159990 results	160,165 results Free to read (137,660) Free to read & use (79,119)	160165 results	25,998 results Free to read (9,364) Free to read & use (7,783)	2096288 results	26,000 results Free to read (9,366) Free to read & use (7,788)	2096541 results
Query	((kampo) OR ("alternative-medicine") OR ("Chinese-medicine") OR ("traditional-medicine") OR ("medicinal-plants") OR ("medicinal-herbs") OR ("herbal-remedy") OR ("herbal remedies") OR (phytomed) OR (phytother) OR (phytochem) OR (phytocomp) OR ("plant-based") OR ("plant-derived") OR ("ultrasound-assisted extraction") OR ("supercritical fluid extraction") OR ("supercritical CO2 extraction") OR ("headspace solid phase microextraction") OR ("liquid-liquid extraction") OR ("essential oil") OR ("essential oils") OR ("medicinal plants") OR ("aromatic oils") OR ("plant essence") OR ("plant extract") OR ("plant extracts") OR ("cold-pressed") OR leaves OR berries OR berry OR bark OR root OR ("flower extract") OR ("bark extract") OR ("berry-oil") OR ("berry-extract") OR ("seed oil") OR ("seed extract") OR ("leaf oil") OR ("root oil") OR ("root extract") OR ("leaf extract") OR (leaves)) AND ((activit) OR (propert) OR (effect) OR (action)) OR ((inhibit) OR (promot*))	getpapers -q "((kampo) OR ("alternative-medicine") OR ("Chinese-medicine") OR ("traditional-medicine") OR ("medicinal-plants") OR ("medicinal-herbs") OR ("herbal-remedy") OR ("herbal remedies") OR (phytomed) OR (phytother) OR (phytochem) OR (phytocomp) OR ("plant-based") OR ("plant-derived") OR ("ultrasound-assisted extraction") OR ("supercritical fluid extraction") OR ("supercritical CO2 extraction") OR ("headspace solid phase microextraction") OR ("liquid-liquid extraction") OR ("essential oil") OR ("essential oils") OR ("medicinal plants") OR ("aromatic oils") OR ("plant essence") OR ("plant extract") OR ("plant extracts") OR ("cold-pressed") OR leaves OR berries OR berry OR bark OR root OR ("flower extract") OR ("bark extract") OR ("berry-oil") OR ("berry-extract") OR ("seed oil") OR ("seed extract") OR ("leaf oil") OR ("root oil") OR ("root extract") OR ("leaf extract") OR (leaves)) AND ((activit) OR (propert) OR (effect) OR (action)) OR ((inhibit) OR (promot*))" -a -n	(kampo OR (alternative-medicine) OR (Chinese-medicine) OR (traditional-medicine) OR (medicinal-plants) OR (medicinal-herbs) OR (herbal-remedy) OR (herbal remedies) OR (phytomed) OR (phytother) OR (phytochem) OR (phytocomp) OR (plant-based) OR (plant-derived) OR (ultrasound-assisted extraction) OR (supercritical fluid extraction) OR (supercritical CO2 extraction) OR (headspace solid phase microextraction) OR (liquid-liquid extraction) OR (essential oil) OR (essential oils) OR (medicinal plants) OR (aromatic oils) OR (plant essence) OR (plant extract) OR (plant extracts) OR (cold-pressed) OR leaves OR berries OR berry OR bark OR root OR (flower extract) OR (bark extract) OR (berry-oil) OR (berry-extract) OR (seed oil) OR (seed extract) OR (leaf oil) OR (root oil) OR (root extract) OR (leaf extract) OR (leaves)) AND ((activit) OR (propert) OR (effect) OR (action)) OR ((inhibit) OR (promot*))	getpapers -q "(kampo OR (alternative-medicine) OR (Chinese-medicine) OR (traditional-medicine) OR (medicinal-plants) OR (medicinal-herbs) OR (herbal-remedy) OR (herbal remedies) OR (phytomed) OR (phytother) OR (phytochem) OR (phytocomp) OR (plant-based) OR (plant-derived) OR (ultrasound-assisted extraction) OR (supercritical fluid extraction) OR (supercritical CO2 extraction) OR (headspace solid phase microextraction) OR (liquid-liquid extraction) OR (essential oil) OR (essential oils) OR (medicinal plants) OR (aromatic oils) OR (plant essence) OR (plant extract) OR (plant extracts) OR (cold-pressed) OR leaves OR berries OR berry OR bark OR root OR (flower extract) OR (bark extract) OR (berry-oil) OR (berry-extract) OR (seed oil) OR (seed extract) OR (leaf oil) OR (root oil) OR (root extract) OR (leaf extract) OR (leaves)) AND ((activit) OR (propert) OR (effect) OR (action)) OR ((inhibit) OR (promot*))" -a -n	essential AND oil	-q "essential AND oil" -a -n	essential oil	getpapers -q "essential oil" -a -n	(essential oil)	getpapers -q "(essential oil)" -a -n	((essential oil)	-q "((essential oil))" -a -n	"essential oil"	getpapers -q ""essential oil"" -a -n	("essential oil")	getpapers -q "("essential oil")" -a -n

[petermr]

These very long queries are fragile. I have a suspicion that one doesn't have balanced brackets. It's possible that they get truncated somewhere. If you can show it with much shorter queries then it could be followed. We can revisit it if any of the team also have long queries.

On Thu, Jun 18, 2020 at 8:50 PM Emanuel Faria notifications@github.com wrote:

Assigned #65 https://github.com/petermr/openVirus/issues/65 to @petermr https://github.com/petermr.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/petermr/openVirus/issues/65#event-3459813913, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFTCS32AYD3JRZDE7JT5VLRXJVZVANCNFSM4OA53TDQ .

-- Peter Murray-Rust Founder ContentMine.org and Reader Emeritus in Molecular Informatics Dept. Of Chemistry, University of Cambridge, CB2 1EW, UK

[EmanuelFaria]

Agreed. I did double and triple check my brackets every which way I could but kept getting very strange results. That's when I began testing the simplest exact phrase queries side by side, as in the table below.

I tried every combination I could think of... even double brackets, and quotes inside brackets, hoping that somehow getpapers would return what I was expecting. Nothing I tried worked. I even tried "NEAR" functions — like essential NEAR/1 oil (which would be awesome, by the way), but no luck with those either. So unless I've made a mistake in my getpapers query, it seems that the problem really lies with the quotes needed to do exact phrases.

Unfortunately, I'll need exact phrase capability to reduce the number of papers I'll download more than ever, because getpapers isn't letting me download more than 29k at a time.

EUPMC	getpapers	EUPMC	getpapers	EUPMC	getpapers	EUPMC	getpapers	EUPMC	getpapers	EUPMC	getpapers
essential AND oil	-q "essential AND oil" -a -n	essential oil	getpapers -q "essential oil" -a -n	(essential oil)	getpapers -q "(essential oil)" -a -n	((essential oil)	-q "((essential oil))" -a -n	"essential oil"	getpapers -q ""essential oil"" -a -n	("essential oil")	getpapers -q "("essential oil")" -a -n
160165 results Free to read (137,660) Free to read & use (79,119)	160165 results	160,022 results Free to read (137,517) Free to read & use (78,974)	160022 results	159,990 results Free to read (137,466) Free to read & use (78,924)	159990 results	160,165 results Free to read (137,660) Free to read & use (79,119)	160165 results	25,998 results Free to read (9,364) Free to read & use (7,783)	2096288 results	26,000 results Free to read (9,366) Free to read & use (7,788)	2096541 results