bigcode-project / bigcode-analysis

Repository for analysis and experiments in the BigCode project.
Apache License 2.0
113 stars 20 forks source link

add PII detection pipeline and analysis notebooks #25

Closed loubnabnl closed 1 year ago

loubnabnl commented 1 year ago

Analysis of detect-secrets tool for detecting secret keys in code. follow-up of the analysis started here https://github.com/bigcode-project/bigcode-analysis/pull/24

EDIT: This PR also adds the PII detection pipeline with regexes and detect-secrets, and the notebooks for analysis and lightTag pre-filtering/postprocessing for annotations