1076 scrape project impact

[dmijatovic]

Scrape project impact

Closes #1076

Changes proposed in this pull request:

• Enable mentions scraper to scrape project citation based on items provided in the output
• Updated info text in output and impact section
• Moved output section before the impact section on the project edit page. On the project view page the order is reversed.
• Added citation section to show scraped citations. It shows scraped citations separately.
• Update rpc to include scraped impact items into the total impact item count
• Changed reference_papers_to_scrape to filter only on items with DOI

How to test:

• make start to build the solution and generate test data. In this example it is better to start without test data so remove data using docker compose down --volumes and then re-start docker compose up
• login to RSD as rsd-admin. If you have latest .env settings, first user will be promoted to rsd-admin in dev mode
• crate new project
• add following items in the output section:
  • doi:10.1186/s13321-017-0220-4
  • 10.1186/1758-2946-6-3
• navigate to impact section. Add one item manually.
• wait for 10 minutes or more for scrapers to run
• confirm that scraper section is populated (280 items).
• preview project and confirm all impact items are shown. It includes all items from impact and citation sections.

Remarks

• On the software page we use reference papers and mentions as "key terms". The scraped citations are shown in the second tab of the reference paper in the edit section while on the public software page scraped citations are integrated into mentions. I find confusing that "public" page might show more mention items than the edit mention section.
• On the project page we use output and impact as "key terms". We use output as base for scraper and show all scraped items in the impact section. Currently there is no any visual indication which items are created manually and which items are scraped (added automatically). The user can delete both manual and scraped item but the scraped items will be added by scraper again. We should indicate scraped items and disable delete button.
• We use different approaches for showing scraped mentions/citations/publications on software and project page. Although the users might be different I think that using one approach could be beneficial

After talking to Jason we think that showing scraped items in the separate citations section is most appropriate approach. If we agree on this approach we will need to update edit software page to use similar approach.

Edit project - output

Edit project - impact (manually added)

Edit project - citation (scraped)

Project page - impact & output

Scraped citations and manual added impact items are shown as impact.

PR Checklist:

• [ ] Increase version numbers in docker-compose.yml
• [ ] Link to a GitHub issue
• [ ] Update documentation
• [ ] Tests

[cmeessen]

Works as described. I noted that on the software page, "Output" is sorted after "Impact". Would it make more sense to first have the output and then the impact which refer to the output?

[sonarcloud[bot]]

Quality Gate passed for 'rsd-database'

Issues
0 New issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[sonarcloud[bot]]

Quality Gate passed for 'rsd-frontend'

Issues
2 New issues

Measures
0 Security Hotspots
13.8% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[jmaassen]

Looks great. Some small improvements of the info texts:

On the output tab:

"Here you can add output that was produced by the project itself, such as papers, books, articles, software, datasets, videos, blogs, etc. The RSD will periodically look for citations of this output using OpenAlex and add them to the citations list on this page."

On the impact tab:

"Here you can add mentions of your output that cannot be found automatically by the RSD. These can be papers by others re-using your software, articles or videos in the press describing the results, policy documents based on your results, etc."

On the citations tab:

"Here we list all the citations of your output that the RSD was able to find automatically by using the DOIs of your output and OpenAlex. On the project page these citations are showed in the impact section together with the items you added manually."

[dmijatovic]

Closed in favor of #1115