[RFC, META-ISSUE] Complete continuous integration (CI) for unit-testing, documentation, and test coverage

[adam2392]

Hi, this is a meta issue to track the items required to bring causal-learn into a more PRable repository:

This list is to track the high-level issues that need to be resolved in a series of PRs. Each may have some description as well associated regarding what needs to be done in detail, what the end result should look like and some motivation. My recommendation is using a combination of GH actions and circleCI as this has worked well in the past for me, but we can change based on discussion.

• [ ] Implement easy-to-use build, test, docs, and formatting commands, such as https://github.com/py-why/dodiscover/blob/b0cb5c48317cf1c6859b3dc9646925fdac3ecf1f/pyproject.toml#L112-L122. These will be used to ease developer workflows and also make CIs easier to run.
• [ ] Implement different requirement dependency groups e.g. for building, testing, doc-building and actually running of causal-learn
• [ ] Enable CI that runs all unit-tests in each PR and commit to main. Should also upload test coverage via GH actions
• [ ] Enable CI that builds (installs) causal-learn in each PR and commit to main. Testing installation for Windows, Mac OSX and Linux via GH actions.
• [ ] Enable CI that builds the documentation in each PR and commit to main via circleCI
• [ ] Add basic templates to GH issues and PRs: Can copy and modify files from https://github.com/py-why/dodiscover/tree/main/.github

Some possible issues that may need to be resolved along the way:

• [ ] Ensure unit-tests run in a reasonable amount of time. I'm not sure how long the unit-tests takes to run currently, but if it requires some intensive computations, then it might be beneficial to refactor to make the unit-tests faster
• [ ] Ensure docs are built in a reasonable amount of time. Examples should be self-contained and short to illustrate a point. If large dataset is needed, one can always trim the dataset and then add a note saying why we did it.

Some implementation details related to CI:

• Copy GH action workflow from https://github.com/py-why/dodiscover/blob/main/.github/workflows/main.yml.
• Enable CircleCI (need an admin) and copy over circleCI artifact redirector for building docs https://github.com/py-why/dodiscover/blob/main/.github/workflows/circle_artifacts.yml
• Copy CircleCI workflow from https://github.com/py-why/dodiscover/blob/main/.circleci/config.yml

Still a WIP to get this list fully flushed out, but here's a first go at it.

[kunwuz]

Thanks so much for listing these To-Dos and instructions! Let's make the pipeline more robust together :)

[jaron-lee]

This all looks good. I might add a few points:

• Pre-commit hooks would help with getting used to CI pipelines
• Consider adding a changelog functionality?

[MarkDana]

Thanks Yujia, Adam and Jaron! Very much looking forward to working together on professionalizing causal-learn :)

[tofuwen]

Great, thanks all! :)