search

rflechner / ScrapySharp

reborn of https://bitbucket.org/rflechner/scrapysharp
MIT License
346 stars 75 forks source link

Error while dockerize ScrapySharp #41

Open gtrq24 opened 1 year ago

gtrq24 commented 1 year ago

Hello, I have a problem when I try to dockerize an application that uses ScrapySharp, because in docker it throws me the following error "Unable to cast object of type 'System.Net.FileWebRequest' to type 'System.Net.HttpWebRequest'" in the line where "Click" function on hyperlink obraz_2023-02-28_092022505

duane-w commented 1 year ago

This is related to this issue: https://github.com/dotnet/runtime/issues/22718

On linux Uri treats "/user/signin" as a valid UriKind.Absolute. I fixed this issue by using if (Action.StartsWith("/") == false && Uri.TryCreate(Action, UriKind.Absolute, out url)).

SeppPenner commented 11 months ago

In my case, the issue was the same: A link in a form was not fully qualified and I adjusted it with something like:

loginForm.Action = (baseUrl.EndsWith("/") || loginForm.Action.StartsWith("/"))
    ? $"{baseUrl}{loginForm.Action}"
    : $"{baseUrl}/{loginForm.Action}";
SeppPenner commented 11 months ago

For me, this is a dotnet bug on Linux / Mac... Might still be useful to state this as "Known issues" in the docs somewhere.