skrapeit / skrape.it

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
https://docs.skrape.it
MIT License
805 stars 59 forks source link

[QUESTION] How to open http logging? #152

Closed chachako closed 3 years ago

chachako commented 3 years ago

In jsoup I can do this (https://stackoverflow.com/a/58524885), skrape.it has a simple way?

christian-draeger commented 3 years ago

hey, with kotlin you can do even better. no need to writing a decorator like in java. if understood correct you want to do something with the instantiated request object. to do so you can just use kotlin build-in scope function called also to execute an additional effect on somerthing. (have a look here for more info).

that means you could just do something like this while using skrape{it} library:

fun main() {
    val responseBody = skrape(HttpFetcher) {
        request {
            url = "https://mvnrepository.com/artifact/kotlin"
            also {
                // the instantiated request object will be available as an implicit receiver / variable called `it` inside of the  also lambda function
                println("try to ${it.method} url: ${it.url}")
                println("request-headers: ${it.headers}")
                // do what ever you want here - maybe use a logger intead of println or trigger some other side effects
            }
        }
        extract {
            responseBody
        }
    }

    println(responseBody)
}

When you see also in the code, you can read it as “and also do the following with the object.”

i hope this helps :) if you like the project don't hesitate to give it a star to support it 🌟

christian-draeger commented 3 years ago

is this question answered and issue can be closed?