vanderbilt-data-science / ancient-artifacts

Dynamic image analysis to identify ancient artifacts in soil samples. We will work on microdebitage (the debris of ancient stone knapping first) and later expand to other materials (e.g., mortars).
MIT License
6 stars 2 forks source link

Ensure uniqueness (or limit duplication) of rows #82

Closed csbell-vu closed 3 years ago

csbell-vu commented 3 years ago

In our current dataset, some of our rows seem to be replicated more than 3,000 times. We should remove all of these duplicates, or at least either inspect or limit the number of duplicates so that the analysis is not biased by odd outputs of the scanner.