Add CI with "nightly" Build Artifact Compatible to Latest IntelliJ Version

[banagale]

I'm submitting a ...

• [X] feature request
• [X] potential offer of PR for request

What is the current behavior?

When Intellij increments versions (i.e. 2020.2->3), it breaks gitflow4idea and other plugins.

There is a build-it-yourself and manually install method, but the friction likely causes many to choose between the latest IDE and gitflow4idea. It has driven me back to the terminal which I've largely managed to automate away from.

This friction can cause pressure to build for releases that validate to new versions, even if the project is not ready.

What is the proposed behavior?

An automated CI workflow that:

• Performs IntelliJ Platform Plugin Verification
• Automatically detects and sets the most recent version of intellij['version'] in build.gradle
• Builds an update of the plugin that can be installed using the Install Plugin from Disk method.
• Uploads the update to the gitflow4idea repo's packages directory
• Updates a wiki page with a direct link to download the package and the above linked instructions.

What is the motivation / use case for changing the behavior?

• Reduce pressure on maintainer(s) to constantly stay on top of new intellij releases
• Create insurance against delays from Intellij in approval and listing plugin version updates
• Provide "easy" use of plugin between releases to those using the newest IDE
• Stub out a CI that can perform more complex operations, including issue management, automated code update and PR creation (such as detect for new intellj version, and create PR with gradle updated and a PR for it)

Please tell me about your environment:

I use this plugin on pycharm every day. I have experience building useful CI workflows using Github Actions.

Other information

I like cats 🐈 but I think that may be okay. Puppies are cute too.

[OpherV]

First let me state that I realize the current state is not ideal where a new major version of IDEA forces people to choose between the latest IDE and gitflow4idea. The reason why I don't specify compatibility in the plugin for future versions other than the current ones is that sometimes things in the IDEA platform change in new version and break gitflow4idea in unexpected ways. These types of failures could lead to bad things happening to your project's git tree, and since I consider git to be critical infrastructure I REALLY don't want people to run something that touches their source control and hasn't been properly tested. This was brought up in the past (e.g #200)

I get that people are upset that they don't have a version compatible with the new IDE for a couple of weeks, but that's not as bad as people being upset that using this plugin broke something in their git tree (even if the underlying cause is not the plugin itself, as it serves just as a UI glue between IDEA and the git flow CLI tool)

Generally, I would LOVE to have a CI/CD solution and automatic tests for this plugin. The dream scenario is a CI process that run tests against every IDEA platform and every new version, to make sure there are no regressions due to either changes in JetBrains code or git flow code, then create and publish a new plugin version if all tests pass.

The crux of the problem is in the tests themselves. This plugin is 99.9% UI. Jebrains provide a mechanism to do testing in IDEA, but not UI testing. e.g. there's no framework to simulate a scenario of specific UI clicks, and read from the IDEA UI to validate its state.

I reached out to Jetbrains about this in the past, but they had no solution and suggested using an external tool. Unfortunately I didn't find any free utility or platform that could help me do that, but I don't write Java daily so it might be just my lack of experience there. #205 has been open for a couple of years about this very subject.

I feel that any CI/CD solution should start with a POC of automatic testing. I'll be thrilled if you have a go at it and achieve some success :)

[banagale]

Thank you for the detailed response.

I was not aware of the previous ticket and agree with the caution around offering builds of an untested plug-in.

I have had a git tree corrupted before and it can be scary. If this plugin caused data corruption it could undermine credibility in IntelliJ and that would not be fun to be a part of.

I do not do Java application work typically but I have done a fair amount of automated testing and regularly geek out on the Test & Code podcast.

The CI portion will actually be the “easy” part for me. GitHub Actions is powerful and the integration with repo event triggers offers plenty of possibilities.

I took an initial look at some Java UI testing info but will need to look more to understand what options are available and perhaps best suited to the task.

I think my next step would be to unfold research in #205 and use that as a basis for more engagement toward realizing an automated CI process that reduces the overhead of mitigating the risks of a critical regression.

All that said I wasn’t counting on learning a new UI test automation suite as part of this proposal, and have a few projects now. So without promises I’ll signal my interest in taking next steps and see where that leads.

[OpherV]

@banagale Did some more research on this.

After speaking with @hsz, turns out such IntelliJ actually starting providing some sort of tooling for this since I last checked: https://github.com/JetBrains/intellij-ui-test-robot

If you can get some sort of POC running where we have a Github Action that spools up Idea and runs an example test I could take it from there and write all the necessary tests, so we could push for automatic building.

[banagale]

Great. I should have some time this week, squished around the holiday. Thanks for this update.

[hsz]

Instead of using the IntelliJ Platform Plugin Verifier GitHub Action, it is highly advised to use the official and built-in integration provided by the IntelliJ Gradle Plugin via the runPluginVerifier task.

[hsz]

Have you ever considered introducing the EAP channel? This and a couple of other solutions features related to the testing and releasing using the GitHub Actions as CI/CD were already introduced by the IntelliJ Platform Plugin Template. This project may help you settling your own configuration within the existing repository too - take a look at the Gradle and GitHub workflows configuration files.

[banagale]

Just to update this, I still want to work towards this goal. I'll look at @hsz's recommendations. I have a lot of work right now, and am happy to accept help here too.

[enderteszla]

@banagale , I would pretty much rather help you if you specified what exactly I could do: what you've already done or tried to do and what yet remains.

[banagale]

@banagale , I would pretty much rather help you if you specified what exactly I could do: what you've already done or tried to do and what yet remains.

I have not made progress on this ticket.

I did have a look at what @hsz referenced and that is the correct starting spot I think.

Even stubbing out the correct template in a committed branch referenced from this issue would probably move this ticket forward.

I would be happy to review / provide feedback, but can not lead at this time.

Regardless, if you make any progress on your own, please update this issue so others can see.

gitflow is not the only plug-in facing these update breakage issues.