Gold Standard

[abalai-ash]

We want to be able to understand the Gold Standard dataset and how it can be applied to what we are doing. Some questions to consider for this research are:

• How do we create a Gold Standard dataset?
• What are the best practices that researchers or people are doing when creating this dataset? We also want to be able to compare this to noisy data and see what steps we can take to implement this.

Below are a few sources on the Gold Standard dataset:

1. Petkova: https://www.ontotext.com/blog/gold-standard-key-to-information-extration-data-quality-control/2.
2. Vazquez: https://ieeexplore.ieee.org/document/9709852/authors#authors
3. Lima: http://ceur-ws.org/Vol-2788/om2020_STpaper1.pdf
4. Tekumalla: https://link.springer.com/article/10.1007/s00521-021-06614-2
5. https://www.researchgate.net/post/What_are_the_steps_one_should_follow_to_prepare_a_gold_standard_dataset
6. https://www.sciencedirect.com/topics/computer-science/gold-standard-data
7. (Talks briefly about experimentation on gold standard datasets) https://link.springer.com/article/10.1007/s00521-021-06614-2

[abalai-ash]

Here is a paper I wrote compiling various sources that answer the key questions above. This is just a draft and is not meant to be distributed or published anywhere. It is also a work-in-progress, so I am open to expanding on different topics and clarifying terms in the paper. UPDATED: An_Informal_Guide_To_Implementing_The_Gold_Standard.pdf