Updating Github actions to only run tests necessary, and adding pip caching.

[mattkwiecien]

Will fill in description later, just triggering a build.