[IMPROVEMENT] Upgrade to Ktor 2

[rushiiMachine]

Ktor 2.0.0 dropped about a month ago and it's incompatible with Ktor 1.5 so an upgrade would be useful

[cjguttormsson]

I've run into an issue due to this. I was getting java.lang.ClassNotFoundException: io.ktor.client.features.HttpTimeout, and it turns out that that's due to a conflict between the 1.x.x version of ktor (from skrape.it) and the 2.x.x version of ktor (which I was using directly). Updating to ktor 2 would be appreciated, so that I and others can use the latest versions of both libraries simultaneously.

[christian-draeger]

I will have a look this week

[aaulia]

Just want to chime in, that using Ktor 2.x.x with current version of skrape will broke AsyncFetcher and HttpFetcher (due to incompatibility mentioned by @cjguttormsson)

[christian-draeger]

Support will be released within the next few days. I just merged a PR we already got a while ago. Means current Snapshot version will support ktor2 already

[christian-draeger]

released version 1.3.0-alpha.1 to maven central. will play around with it a bit more and afterwards release 1.3.0

if anyone else wants to try out version 1.3.0-alpha.1 in the meantime, it would be greatly appreciated. :)

[iMemon]

I have experimented with 1.3.0-alpha.1 because my project was already using Ktor 2.0. Seems like it is working fine. But I was getting the error mentioned in the following image

So I needed to add resources.excludes.add("mozilla/public-suffix-list.txt") in the packagingOptions

So now packagingOptions will be like this:

packagingOptions {
    resources.excludes.add("META-INF/DEPENDENCIES")
    resources.excludes.add("mozilla/public-suffix-list.txt")
  }

Then I checked if this error is occuring in previous version or not. Found that:

• 1.1.6 needs only "META-INF/DEPENDENCIES"
• After that we need to add both "META-INF/DEPENDENCIES" and "mozilla/public-suffix-list.txt"

[christian-draeger]

Thx for verifying

[cjguttormsson]

Just tried out 1.3.0-alpha.1, worked great for me!

[christian-draeger]

Great I will release as final 1.3.0 version soon

[UKMIITB]

@christian-draeger When is the final version 1.3.0 coming out ? I can still see only 1.3.0-alpha.1 in maven central

[slott]

Any news here? I just discovered scrape.it and 1.3.0-alpha.1 seems to work like a charm. Wondering whats holding up a proper release ?

[RoyalSWiSH]

1.3.0-alpha.1 seems to work, but with 1.3.0-alpha.2 i get this duplicate class error (I use ktor 2.3.7 in the rest of my project, which might cause the conflict in skrapeit 1.2.2)

Duplicate class it.skrape.fetcher.ExtensionsKt found in modules skrapeit-async-fetcher-1.3.0-alpha.2 (it.skrape:skrapeit-async-fetcher:1.3.0-alpha.2) and skrapeit-http-fetcher-1.3.0-alpha.2 (it.skrape:skrapeit-http-fetcher:1.3.0-alpha.2)
Duplicate class it.skrape.fetcher.ExtensionsKt$WhenMappings found in modules skrapeit-async-fetcher-1.3.0-alpha.2 (it.skrape:skrapeit-async-fetcher:1.3.0-alpha.2) and skrapeit-http-fetcher-1.3.0-alpha.2 (it.skrape:skrapeit-http-fetcher:1.3.0-alpha.2)
Duplicate class it.skrape.fetcher.ExtensionsKt$installBasicAuth$1 found in modules skrapeit-async-fetcher-1.3.0-alpha.2 (it.skrape:skrapeit-async-fetcher:1.3.0-alpha.2) and skrapeit-http-fetcher-1.3.0-alpha.2 (it.skrape:skrapeit-http-fetcher:1.3.0-al
...

packagingOptions {
        resources {
            excludes += '/META-INF/{AL2.0,LGPL2.1}'
            // Error when importing skrapeit
            excludes += '/META-INF/DEPENDENCIES'
            excludes += 'mozilla/public-suffix-list.txt'
        }
    }

[christian-draeger]

Thanks for reporting. I will have a look when I find time. If someone is open to send PR would be welcome.

[kqvanity]

@iMemon packagingOptions is now deprecated at the latest Android Gradle plugin.

android {
    packaging {
        resources {
            excludes.addAll(listOf(
                "/META-INF/{AL2.0,LGPL2.1}",
                "/META-INF/DEPENDENCIES",
                "mozilla/public-suffix-list.txt"
            ))
        }
    }
}