feat: Prospero SearchSource

[geritwagner]

Description

Enhance the CoLRev environment by implementing the colrev.prospero package, offering search functionality for the PROSPERO database, which specializes in systematic review protocols. This feature involves scraping search results, including detailed protocol information, and supporting load heuristics for recognizing and processing exported results.

Implementation Notes

• Use the BeautifulSoup library to develop a scraper that navigates the PROSPERO search interface, fetching search results along with detailed data from sub-pages, such as the date of registration.
• Documentation should guide users on how to perform searches and load search results.
• Implement unit tests simulating the retrieval process with a simple example. The colrev.crossref package provides an example for unit tests.
• The packages docs page explains the steps of developing CoLRev packages.
• The search-feed offers functionality for storing records.
• CEP003 describes principles for SearchSources.
• CEP002 describes the standard data schema for records.
• The colrev.crossref package implements similar unit tests.

User Story

1. The user initializes a CoLRev project using colrev init.
2. He/she conducts a search operation with colrev search -a colrev.prospero, specifying search parameters relevant to their topic.
3. The package retrieves records based on the parameters and saves them in the data/search directory of the project.
4. When users run colrev load, the records from the search directory are added to the data/records.bib file, which integrates all search results in the project.
5. When users run colrev search again, records are retrieved from the API, updating the existing records.

Useful Links

• PROSPERO
• BeautifulSoup Documentation
• BeautifulSoup with JavaScript Tutorial

Expected Effort

2 months, 3-4 people.

[trathienphuc-tran]

I would like to work on this issue 🙋

[ammar-uni]

I would like to hopefully work on this

[OlhaKomashevska]

I would like to work on this issue

[geritwagner]

Following up on our session, I double-checked libraries suitable for PROSPERO and I have a working version on my machine. Let's discuss the options you have (Python libraries) as a next step.