search

hylang / hy-mode

Hy mode for Emacs
GNU General Public License v3.0
191 stars 47 forks source link

Setup CI for testing #95

Closed brandonwillard closed 4 years ago

brandonwillard commented 4 years ago

This project really needs CI. We can probably set up a GitHub Action that runs the tests without too much effort.

ekaschalk commented 4 years ago

What do you want to get out of it?

Typically it's jedhy that's the issue, because it's coupled with Hy's internals. Have the tests ever not been passing now that I think about it, it's been a bit for me.

brandonwillard commented 4 years ago

Are you asking what I want to get out of CI/automated testing?

ekaschalk commented 4 years ago

Yes, for this project. Active development hasn't been happening for awhile so this would be forward looking. It would cover syntax, font locks, and indentation. These things pretty rarely other users contribute to.

brandonwillard commented 4 years ago

At the very least, it would be nice if we could reasonably merge PRs—no matter what the changes entail—without having to check out each PR branch and run the tests locally (if us maintainers are being responsible, that is).

Furthermore, there's also the fact that each developer's local setup can easily differ in ways that Cask package management cannot account for (e.g. I run the feature/native-comp branch of Emacs), so the part of the CI process that constructs and tests the supported platforms is also very important—especially since most people definitely won't want to install multiple Emacs versions.

This is just scratching the surface of why CI is good for even small open-source projects, and, given that it's also free and relatively easy to set up, there would need to be some pretty strong disadvantages to CI for it to not be worthwhile.

Plus, it's not difficult to argue that having CI promotes both active development and general use of the package, since at least minimal functionality can be verified by a casual observer who might decide to use or develop it based on this information.

ekaschalk commented 4 years ago

Yea CI is almost always good as long as you have actual coverage.

This project has enough coverage I suppose to be useful in some PRs that have historically been received here, like font lock updates for language changes. Also I'm glad people are actively using that branch.

CI promotes both active development and general use of the package

This is true for general use projects like evil and ivy, but I have no reason to believe this for major modes. The popularity of the mode is pretty dependent upstream on Hy. I would love a reason to invest time into this like bbatsov does with cider, but that is not how Hy's development has played out.

I'm fine with adding CI, but also know I consider this repo more-or-less complete pending any new motivation from Hy.

brandonwillard commented 4 years ago

Yea CI is almost always good as long as you have actual coverage.

CI can be good even if it simply verifies that the package installs properly on a given system! And what about coverage across different versions of Emacs? The coverage you're talking about doesn't cover that.

Regardless, I'm not aware of any relationships between coverage measures and CI applicability—aside from the trivial case of zero coverage—that are actively used in the development world. The most relevant consideration is almost always simply the cost/benefit trade-off of deploying and maintaining CI, and—again—in this case that's not a real deterrent.

Here are a few non-evil/ivy-sized *-mode projects that have CI (and not all appear to have large test suites/coverage):

Referencing only the last case, ox-latex-subfigure, we can get a pretty good idea of how simple it can be to set up CI: e.g. we only need to make some minor modifications to a workflow file like this.

CI promotes both active development and general use of the package

This is true for general use projects like evil and ivy, but I have no reason to believe this for major modes. The popularity of the mode is pretty dependent upstream on Hy. I would love a reason to invest time into this like bbatsov does with cider, but that is not how Hy's development has played out.

In my experience, the use and popularity of a language can also be quite correlated with how well it's supported by editors/IDEs, so you're only referring to one side of that relationship. In other words, there might be a good chance that the quality of hy-mode affects the use and popularity of Hy itself, and that alone is a good reason to automate testing.

ekaschalk commented 4 years ago

Like I said, I'm fine with and will approve any added CI but I don't consider it a priority, with your remarks still welcome and considered.

I chose the more generous interpretation of your respectable maintainers comment. I'll again choose to interpret your last remark as not an intentional slight against my work and myself. If I'm incorrect, file your grievance over email as this is not proper for the issue tracker. I will not allow passive aggression directed at anyone involved in this project.

On Thu, Jul 30, 2020, 2:15 PM Brandon T. Willard notifications@github.com wrote:

Yea CI is almost always good as long as you have actual coverage.

CI can be good even if it simply verifies that the package installs properly on a given system! And what about coverage across different versions of Emacs? The coverage you're talking about doesn't cover that.

Regardless, I'm not aware of any relationships between coverage measures and CI applicability—aside from the trivial case of zero coverage—that are actively used in the development world. The most relevant consideration is almost always simply the cost/benefit trade-off of deploying and maintaining CI, and—again—in this case that's not a real deterrent.

Here are a few non-evil/ivy-sized *-mode projects that have CI (and not all appear to have large test suites/coverage):

Referencing only the last case, ox-latex-subfigure, we can get a pretty good idea of how simple it can be to set up CI: e.g. we only need to make some minor modifications to a workflow file like this https://github.com/linktohack/ox-latex-subfigure/blob/master/.github/workflows/test.yml .

CI promotes both active development and general use of the package

This is true for general use projects like evil and ivy, but I have no reason to believe this for major modes. The popularity of the mode is pretty dependent upstream on Hy. I would love a reason to invest time into this like bbatsov does with cider, but that is not how Hy's development has played out.

In my experience, the use and popularity of a language can also be quite correlated with how well it's supported by editors/IDEs, so you're only referring to one side of that relationship. In other words, there might be a good chance that the quality of hy-mode affects the use and popularity of Hy itself, and that alone is a good reason to automate testing.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hylang/hy-mode/issues/95#issuecomment-666705128, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA62ZLRYC2BDVXSMXRMNEODR6HPHZANCNFSM4PKB24DQ .