2.0 - resolve LATEST-EAP-SNAPSHOT

jonathanlermitage commented 4 months ago

Describe the need of your request

With gradle plugin v1, I was using intellij { version.set("IC-LATEST-EAP-SNAPSHOT") } With gradle plugin v2, I am using dependencies { intellijPlatform { intellijIdeaCommunity("...") } } but, what should I use as intellijIdeaCommunity's parameter? I tried LATEST-EAP-SNAPSHOT but it doesn't work. I am looking at https://plugins.jetbrains.com/docs/intellij/tools-intellij-platform-gradle-plugin-faq.html#how-to-check-the-latest-available-eap-release -> should I really compute the latest EAP snapshot version string by myself? Is there a helper for that? Thx.

Proposed solution

Allow { intellijPlatform { intellijIdeaCommunity("LATEST-EAP-SNAPSHOT") } } or something similar, as in plugin v1.

Alternatives you've considered

No response

Additional context

No response

YannCebron commented 4 months ago

Note, in general building against LATEST-EAP-SNAPSHOT is not recommended, as it can have unpredictable results (and you'll usually want to target a specific minimum EAP build).

jonathanlermitage commented 4 months ago

I don't want to build plugins for EAP IDEs, I want to test them with EAP builds, by running gradlew runIde.
Some developers are using EAP IDEs, and I want to fix issues asap. Also, testing against EAP IDEs helps me to fix issues before the availability of the next IDE stable build.
Finally, I don't want to find specific EAP version strings by myself (this is time-consuming and boring), I simply want the latest version, automatically. I know they can be unpredictable, but I am doing this since many years, and it helps me a lot. I have a very good experience with that. I used to test my plugins with a minimum stable IDE version, the latest stable IDE version, and the latest EAP.

YannCebron commented 4 months ago

There's no need to build against some EAP version, it's sufficient to configure it to run against that EAP build.

jonathanlermitage commented 4 months ago

To give more context, I used to test my plugin (by testing, I mean running gradlew runIde) with:

a minimal stable IDE version. This version never changes, so it's hard-coded. This is what I am using in my since-build property.
the latest stable IDE version. I have a script that finds this version string. I'm doing this because I don't want a until-build property. I don't want to publish new releases just for updating this tag. But, this is another story.
the latest EAP. I explained why.

Also, the LATEST-EAP-SNAPSHOT property was available in plugin v1. I don't understand why it has been removed. It was useful for some plugin developers.

hsz commented 4 months ago

This is already possible. To build your plugin with a specific IntelliJ Platform but run with another, for testing purposes, you have to define a custom task with:

val runLatestEapIde by tasks.registering(CustomRunIdeTask::class) {
    version = "LATEST-EAP-SNAPSHOT"
}

Note that, by default, we use now JetBrains CDN where only released IDEs are hosted. To switch to IJ Maven Repository where snapshots are present, add this line to the gradle.properties.

org.jetbrains.intellij.platform.buildFeature.useBinaryReleases=false

This property name as well as the CDN/Maven resolution may soon change.

bric3 commented 2 weeks ago

Since 2.0.0 is out the property mentioned above is gone, one can use something like this.

intellijPlatformTesting {
    runIde {
        create("runIntelliJLatest") {
            type = IntelliJPlatformType.IntellijIdeaCommunity
            useInstaller = false
            val versionProvider = providers.gradleProperty("runPlatformLatestVersion").orElse("LATEST-EAP-SNAPSHOT")
            version = versionProvider
            sandboxDirectory = providers.zip(layout.buildDirectory.dir("idea-sandbox/runIntelliJ"), versionProvider) { dir, version ->
                dir.dir(version)
            }
        }
    }
}

The only issue I found is that the JBR is not used (if other dependencies are not using it), e.g. here the main version don't use the installer and as such dont need the JBR, but adding the JBR runtime dependency to help the runIntelliJLatest don't seem to work for that task:

dependencies
    intellijPlatform {
        intellijIdeaCommunity(project.providers.gradleProperty("buildPlatformVersion"))

        pluginVerifier()
        instrumentationTools()
        zipSigner()

        jetbrainsRuntime()
    }
}

jonathanlermitage commented 2 weeks ago

@bric3 I find the latest EAP snapshot from https://www.jetbrains.com/updates/updates.xml by using an xpath expression: /products/product[@name='IntelliJ IDEA']/channel[@id='IC-IU-EAP-licensing-EAP']/build[1]/@fullNumber. It returns something like 243.12818.47.
I'm using it since a while, and it always refers to IDE builds with a JBR. It may skip very few EAP builds (I did not check), but at least it works.
I do not set useInstaller, I don't know if it helps.

My (ugly and absolutely not optimized yet) script looks like:

val ideaVersion = findIdeaVersion(pluginIdeaVersion) // pluginIdeaVersion comes from properties and it set to LATEST-STABLE, LATEST-EAP-SNAPSHOT, or a specific version number
...
intellijPlatform { // https://plugins.jetbrains.com/docs/intellij/tools-intellij-platform-gradle-plugin-dependencies-extension.html
            create(pluginIdeaPlatformType, ideaVersion)
            instrumentationTools()
            pluginVerifier()
            testFramework(TestFrameworkType.Platform)
        }

/**
 * Find the latest stable or EAP IDE version from the JetBrains website, otherwise simply returns the given IDE version string.
 * @param ideaVersion can be LATEST-STABLE, LATEST-EAP-SNAPSHOT or a specific IDE version string (year.maj.min).
 */
fun findIdeaVersion(ideaVersion: String): String {

    /** Find the latest IntelliJ EAP version from the JetBrains website. Result is cached locally for 24h. */
    fun findLatestIdeaVersion(isStable: Boolean): String {

        /** Read a remote file as String. */
        fun readRemoteContent(url: URL): String {
            val t1 = System.currentTimeMillis()
            val content = StringBuilder()
            val conn = url.openConnection() as HttpURLConnection
            conn.requestMethod = "GET"
            BufferedReader(InputStreamReader(conn.inputStream)).use { rd ->
                var line: String? = rd.readLine()
                while (line != null) {
                    content.append(line)
                    line = rd.readLine()
                }
            }
            val t2 = System.currentTimeMillis()
            logger.quiet("Download $url, took ${t2 - t1} ms (${content.length} B)")
            return content.toString()
        }

        /** Find the latest IntelliJ version from the given url and xpath expression that picks the desired IDE version and channel.
         * The result is cached for 24hr.*/
        fun getOnlineLatestIdeVersion(definitionsUrl: URL, xpathExpression: String): String {
            val definitionsStr = readRemoteContent(definitionsUrl)
            val builderFactory = DocumentBuilderFactory.newInstance()
            val builder = builderFactory.newDocumentBuilder()
            val xmlDocument: Document = builder.parse(ByteArrayInputStream(definitionsStr.toByteArray()))
            val xPath = XPathFactory.newInstance().newXPath()
            return xPath.compile(xpathExpression).evaluate(xmlDocument, XPathConstants.STRING) as String
        }

        val t1 = System.currentTimeMillis()
        // IMPORTANT if not available, migrate to https://data.services.jetbrains.com/products?code=IC
        val definitionsUrl = URL("https://www.jetbrains.com/updates/updates.xml")
        val xpathExpression =
            if (isStable) "/products/product[@name='IntelliJ IDEA']/channel[@id='IC-IU-RELEASE-licensing-RELEASE']/build[1]/@version"
            else "/products/product[@name='IntelliJ IDEA']/channel[@id='IC-IU-EAP-licensing-EAP']/build[1]/@fullNumber"
        val cachedLatestVersionFile =
            File(System.getProperty("java.io.tmpdir") + (if (isStable) "/jle-ij-latest-stable-version.txt" else "/jle-ij-latest-eap-version.txt"))
        var latestVersion: String
        try {
            if (cachedLatestVersionFile.exists()) {

                val cacheDurationMs = 24 * 60 * 60_000 // 24hr
                if (cachedLatestVersionFile.exists() && cachedLatestVersionFile.lastModified() < (System.currentTimeMillis() - cacheDurationMs)) {
                    logger.quiet("Cache expired, find latest EAP IDE version from $definitionsUrl then update cached file $cachedLatestVersionFile")
                    latestVersion = getOnlineLatestIdeVersion(definitionsUrl, xpathExpression)
                    cachedLatestVersionFile.delete()
                    Files.writeString(cachedLatestVersionFile.toPath(), latestVersion, Charsets.UTF_8)

                } else {
                    logger.quiet("Find latest EAP IDE version from cached file $cachedLatestVersionFile")
                    latestVersion = Files.readString(cachedLatestVersionFile.toPath())!!
                }

            } else {
                logger.quiet("Find latest EAP IDE version from $definitionsUrl")
                latestVersion = getOnlineLatestIdeVersion(definitionsUrl, xpathExpression)
                Files.writeString(cachedLatestVersionFile.toPath(), latestVersion, Charsets.UTF_8)
            }

        } catch (e: Exception) {
            if (cachedLatestVersionFile.exists()) {
                logger.warn("Error: ${e.message}. Will find latest EAP IDE version from cached file $cachedLatestVersionFile")
                latestVersion = Files.readString(cachedLatestVersionFile.toPath())!!
            } else {
                throw RuntimeException(e)
            }
        }
        if (logger.isDebugEnabled) {
            val t2 = System.currentTimeMillis()
            logger.debug("Operation took ${t2 - t1} ms")
        }
        return latestVersion
    }

    if (ideaVersion == "LATEST-STABLE") {
        val version = findLatestIdeaVersion(true)
        logger.quiet("Found latest stable IDE version: $version")
        return version
    }
    if (ideaVersion == "LATEST-EAP-SNAPSHOT") {
        val version =  findLatestIdeaVersion(false)
        logger.quiet("Found latest EAP IDE version: $version")
        return version
    }
    logger.warn("Will use user-defined IDE version: $version")
    return ideaVersion
}

bric3 commented 2 weeks ago

@jonathanlermitage Thank you ! Looks interesting.

JetBrains / intellij-platform-gradle-plugin