search

skrapeit / skrape.it

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
https://docs.skrape.it
MIT License
808 stars 59 forks source link

[IMPROVEMENT] unify maven central release process #123

Closed christian-draeger closed 3 years ago

christian-draeger commented 3 years ago

Since we have submodules like core, mockmvc, ktor, and more in the future it would be nice to have a unified way of releasing these artifacts to mVen central. Currently we are using a gradle plugin to this with the core module. But all modules should support the release process. If that would be possible via github actions somehow it would be awesome.

christian-draeger commented 3 years ago

1.0.0 release waits for #127

christian-draeger commented 3 years ago

multi module spilt (#69) done ✅

christian-draeger commented 3 years ago

after multi module split we have following modules that should be released separat:

and an aggregator release that is including all the modules and is build from preject root--> it.skrape:skrapeit

ruffCode commented 3 years ago

I have a fix for this I think.

The issue with the fetchers modules is in how they are referenced throughout the project. Gradle is treating the root fetchers folder as a module.

In settings.gradle.kts, include the modules this way, then in all the imports change ":fetchers:submodule" to ":submodule" 

include(":http-fetcher")
include(":ktor-fetcher")
include(":browser-fetcher")
project(":basis-fetcher").projectDir = file("fetcher/basis-fetcher")
project(":http-fetcher").projectDir = file("fetcher/http-fetcher")
project(":ktor-fetcher").projectDir = file("fetcher/ktor-fetcher")
project(":browser-fetcher").projectDir = file("fetcher/browser-fetcher")

Place a "gradle.properties" in every module you want to publish with 

POM_ARTIFACT_ID=
POM_NAME=
POM_DESCRIPTION=
VERSION_NAME=

In the root build.gradle, we want to make sure that the publishing plugin is only being applied to the artifacts you want to publish. So we can declare, for instance, val excludeFromPublishing = listOf( "dsl", "examples", "test-utils", "scraping")

Then, remove apply(plugin = "com.vanniktech.maven.publish") from the allprojects block and in the subprojects block 

subprojects {
    if (this.name !in excludeFromPublishing) {
        apply(plugin = "com.vanniktech.maven.publish")
    }
//.....
}

I tested this by publishing to mavenLocal and it seemed to work.

christian-draeger commented 3 years ago

Ok will try as soon as I am back 2 keyboard :) big thx for figuring out 🙂