Idea for testing in a non-synthetic setting: Evaluate full documents (perplexity should go way up from initial), replace all names with names from your fine-tuning set (perplexity should go back down) #32

Open toddnief opened 1 month ago