Implement CrossQ

[noahfarr]

Problem Description

I would like to implement the CrossQ algorithm as proposed in https://openreview.net/pdf?id=PczQtTsTIX. I already did some testing and it looks like its not that much effort to implement it based on SAC. I will implement it for myself anyway, so I might as well contribute to the repo. Please let me know if this contribution is something you are interested in.

Checklist

• [x] I have installed dependencies via poetry install (see CleanRL's installation guideline.
• [x] I have checked that there is no similar issue in the repo.
• [x] I have checked the documentation site and found not relevant information in GitHub issues.

[noahfarr]

Here is the current state of my implementation: https://github.com/noahfarr/cleanrl/blob/master/cleanrl/crossq.py