Eva Weekly Notes -- 2024-03-04

[evamaxfield]

RA Projects

AwardFindR / award-pynder

• 5/16 data sources ported to Python
• handed off to Nic + Isaac

Research Software

EAGER / Software Sustainability

• [ ] Create dataset and basic EDA for "follow up sample" -- specifically:
  • Select only CISE and BIO from 2013 - 2023 (ask Nic if he wants to 2023 because need grants that have concluded)
  • Remove no code and survey responses from the set
  • Remove EAGER, Workshop, REU, and coop grants
• [ ] Build on top of this dataset by:
  • Retrieve project reports
  • Identify Publications and Software from those reports
  • Put these in a dataset linked by award (award-software and award-publication)
  • Find multiple publications under the same award using WoS
• [ ] Come up with clear questions to answer using the existing survey data
• [x] Timeline plot / think through other ways to visualize "typical development patterns as they relate to grant funding timelines"

RS-Graph / Database for RS Research

• [x] Review and talk through planning doc with Nic
• [x] Put together a plan for Akhil and Terra to work on the CRediT work
• [x] Ask Isaac for help writing a Llama/Mistral/etc. fine-tuning script for the Dev - Author annotation data
• [x] ICSSI: minor plan for abstract proposal
• [ ] Write WoS script for getting papers who mention software, tool, pipeline, or package in title from set of journals

Civic

Public Comment Segmentation

• [x] Write evaluation script for application of model for determining the segmentation quality (i.e. are we missing parts before, after, etc.)
• [x] Train and eval models for Seattle and Oakland
• [ ] Choose next city council for data annotation and prep data

ML for PIT

• No progress.

PhD Activities

Writing

• [x] Review and contribute to the Gig Driver Deactivation paper
• [ ] Review and contribute to the CZI Software Dependency Graph paper

Reading

• Read this week:

  • https://doi.org/10.1007/s11192-023-04806-2

• Plan to read this week:

  • https://www.semanticscholar.org/paper/Ecosystem-level-determinants-of-sustained-activity-Valiev-Vasilescu/e8eb4dc83c7f74848c331005aa7c124dc13cf6a9?utm_source=direct_link
  • https://www.semanticscholar.org/paper/Beyond-the-Repository-Casari-Ferraioli/21dd9f046fc807a498840e19f6c81e69ffad8651?utm_source=direct_link
  • https://www.semanticscholar.org/paper/Fairness-and-Abstraction-in-Sociotechnical-Systems-Selbst-Boyd/9dae5041ed97ab21f637fb282d758fd9cb771476?utm_source=direct_link
  • https://doi.org/10.1162/qss_a_00275
  • https://doi.org/10.1162/qss_a_00277
  • https://doi.org/10.1162/qss_a_00264
  • https://doi.org/10.1162/qss_a_00266

[evamaxfield]

Specific tasks 2024-03-07:

• ~finish constructing dataset for ankil and terra~
• put together script for evaluating whole public comment segmentation model -- partially finished, the best model is fine-tune-mpnet-three-sentences so need to train and keep that one for use
• choose next city council and prep data for annotation
• ~keep working on icssi~

[evamaxfield]

Specific tasks 2024-03-08

spent the morning working on icssi PLoS dataset. Have a basic pipeline working for extracting key details and filtering out data from the whole corpus. Also did some minor EDA from a sample of ~20,000 JATS XMLs (iirc): https://github.com/evamaxfield/aligning-credit/blob/main/eda.ipynb

data which meets our requirements tends (so far with this trial sample) to:

• be relatively recent (which makes sense)
• predominately come from PLoS Comp Bio and PLoS ONE
• the number of authors reporting software as a contribution is roughly half of the largest contribution category reported (which i think also makes sense -- if we are filtering out papers which do not have a link to github.com / .io then we would expect 1 or 2 contributors with software as a contribution per paper imo.

Stuff to do to wrap up the day:

• put together script for evaluating whole public comment segmentation model (train fine-tune-mpnet-three-sentences for real and then apply)
• read two papers from above list
• start JOSS review
• choose next city council and prep data
• check in on nic for new email dump
• check in on akhil and terra for practice annotations