search

sjdirect / abot

Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.
Apache License 2.0
2.24k stars 557 forks source link

Abot.Demo.exe no longer runs on mono #123

Closed sjdirect closed 8 years ago

sjdirect commented 8 years ago

The e.CrawledPage.HttpWebResponse object from the PageCrawlCompleted event is null!!!! Tried to run this with mono 3.2-4.2.3 with the same effect.

To reproduce...

1) Build abot solutions locally 2) Move /Abot.Demo/bin/debug folder to linux machine 3) Install mono-complete using these instructions 3) run it 

mono Abot.Demo.exe

Status:[NA] Url:[http://abotx.org/] Parent:[http://abotx.org/] Retry:[0] - [AbotLogger]

sjdirect commented 8 years ago

Adding MoMA analysis report Reports.zip

lytico commented 8 years ago

reason: in abot/Abot/Poco/HttpWebResponseWrapper.cs:

the line: this.IsMutuallyAuthenticated = response.IsMutuallyAuthenticated; throws a NotImplementedException under mono

cause

HttpWebResponse.IsMutuallyAuthenticated is not implemented in mono

tested with Mono JIT compiler version 3.2.8 (Debian 3.2.8+dfsg-4ubuntu4)

if i comment the line, everything works ok. nicer would be to test if running under mono.

do you want a pull request?

sjdirect commented 8 years ago

I'll have to take a closer look at what functionality could possibly be altered by that. If the pull request is just commenting that line out then don't worry about a pull request. Thanks for the input.

lytico commented 8 years ago

i modified my code like this, so its supported on windows, and in future versions of mono, if they implement the feature:

    protected static bool? IsMutuallyAuthenticatedImplemented = null;
        public HttpWebResponseWrapper(HttpWebResponse response)
        {
         ....
            if (!IsMutuallyAuthenticatedImplemented.HasValue) {
                try {
                    this.IsMutuallyAuthenticated = response.IsMutuallyAuthenticated;
                    IsMutuallyAuthenticatedImplemented = true;
                }
                catch (NotImplementedException e) {
                    IsMutuallyAuthenticatedImplemented = false;
                }
            } 
            this.IsMutuallyAuthenticated = IsMutuallyAuthenticatedImplemented.Value ? response.IsMutuallyAuthenticated : false;

              ....
sjdirect commented 8 years ago

Thanks lytico!