Evaluate ChatGPT performance on measurementTechnique Extraction
Issue Description
It would be good to be able to evaluate how well ChatGPT extracts measurementTechniques based on Dataset names and descriptions. While the performance may vary based on data type and repository, it would be good to at least evaluate some datasets that already have curated values for measurementTechniques to see how well the results overlap.
Approach:
[x] Identify records which have measurementTechnique values from the following repositories (@DylanWelzel since you've already done this, can you send @ZubairQazi the list of record ids?
[x] NCBI GEO
[x] LINCS
[x] REFRAMEDB
[ ] Randomly select 25 records from the measurementTechnique-containing subset of each of the above repositories
[ ] Run the ChatGPT measurementTechnique prompt (providing only the name and description) for each of the 75 records (25 per repo)
[ ] Confirm presence/absence of the measurementTechnique values for each record in the predictions by ChatGPT
Issue Discussion
No response
Please select the type of metadata improvement
[ ] Standardization (normalizing free text to an ontology)
[X] Augmentation (adding values for metadata fields missing values)
[ ] Clean up (addressing redundancy or messy metadata)
[ ] Structure (changing the structuring of the metadata to support front end UI features)
Issue Name
Evaluate ChatGPT performance on measurementTechnique Extraction
Issue Description
It would be good to be able to evaluate how well ChatGPT extracts measurementTechniques based on Dataset names and descriptions. While the performance may vary based on data type and repository, it would be good to at least evaluate some datasets that already have curated values for measurementTechniques to see how well the results overlap.
Approach:
Issue Discussion
No response
Please select the type of metadata improvement
Meta URL
No response
Related WBS task
https://github.com/NIAID-Data-Ecosystem/nde-roadmap/issues/13
For internal use only. Assignee, please select the status of this issue
Status Description
No response
Request status check list