search

skrapeit / skrape.it

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
https://docs.skrape.it
MIT License
805 stars 59 forks source link

[BUG] Skrape.It causes a stack dump when trying to run it from an Android application #144

Closed Git-Jiro closed 3 years ago

Git-Jiro commented 3 years ago

Describe the bug I get a stack trace when trying to use skrape.it from within an Android app.

Minimal example, gradle.build and a stack trace are in this gist: https://gist.github.com/Git-Jiro/34e7f49d6abddfe825f53cc6df4d4a4d

Expected behavior Scraper should not cause a stack trace. (My code works fine in normal java / kotlin application)

christian-draeger commented 3 years ago

i will try to have a look asap. sorry for inconvenience

Git-Jiro commented 3 years ago

Hi! No rush. I found out version 1.0.0 works fine for me. So I use that version for now.

christian-draeger commented 3 years ago

Ok got it. It's because the http client has been refactored to use ktor Client with apache. We need to choose some http client implementation that will work properly on jvm as well as on Android like ktor client with okhttp or simular

christian-draeger commented 3 years ago

ok could test version 1.1.1 on android now and it works for me. i added an example here --> https://github.com/skrapeit/skrape.it/blob/master/examples/android/README.md

Git-Jiro commented 3 years ago

Hi! Thanks for providing a sample application. However I have a small issue with the sample: When I try to open it I get an error message like this here: https://issuetracker.google.com/issues/187470273?pli=1

The project file seems to have been created with the latest beta build of Android Studio Arctic Fox (2020.3.1) and does not work with Android Studio 4.2.1.

Improvement Idea to make the sample more useful for more people: Recreate it with a "non beta build" of Android studio.

christian-draeger commented 3 years ago

Hey, Thats true. I used the beta of android studio because I needed support for jetpack-compose (which I used in the example and which will be the defacto standard from this summer on). But I agree that there obviously will be a lot of projects out there that are not able or don't want to migrate instantly. Thereby it would definitely still be a benefit to have a working Android example that builds on none beta releases of libraries like jetpack-compose as well as on Android studio final releases.

Migration should be possible straight away, since the part of the example app where skrapeit is involved will not change really. I will try to add some example soon.