So I found out that if you put 1 fresh milk container is a regular crafting table you will get 4 fresh milk back.
This might actually be an unintended cross-mod bug caused by The Vegan Option, which converts recipes that use Milk Buckets into OreDictionary versions of the recipe--thereby allowing Fresh Milk to be used for this recipe when Pam's HarvestCraft intends for it only to work with a Milk Bucket.
See https://www.reddit.com/r/LifeInTheWoods/comments/49mq59/infinite_milk_bug/