Open minhhn2910 opened 1 year ago
Hi, great repo. Could you share the Normalizer and Java Parser code for function_level to regenerate token list similar to your function_normalized_tokens and function_tokens in the original processed dataset ?
function_level
function_normalized_tokens
function_tokens
Hi, great repo. Could you share the Normalizer and Java Parser code for
function_levelto regenerate token list similar to yourfunction_normalized_tokensandfunction_tokensin the original processed dataset ?