sjdirect / abotx

Cross Platform C# Web crawler framework, headless browser, parallel crawler. Please star this project! +1.
https://abotx.org
131 stars 23 forks source link

Javascript rendering does not work on Azure Web App or Api managed PAAS #7

Closed sjdirect closed 7 years ago

sjdirect commented 8 years ago

Email contents....

Steven,

Just wanted you to know that even though I’ve managed to get Abot / AbotX running on an Azure WebAPI instance, there is a huge problem with javascript rendering. My WebAPI code was exhibiting horrible performance issues that only seemed to happen when I published the code to an Azure WebAPI instance. When running on my development machine (a Windows 10 VM) it worked just fine. So I opened a support case with Azure support, and they got back to me with why my code was running so slow. It seems that phantomjs.exe is wanting to execute some code that Microsoft disallows in the WebAPI ‘sandboxed’ instance. They identified the code as: NTUserSystemParametersInfo(). They said that the phantomjs executable would attempt this call hundreds of times, each time a failure, before giving up.

So, my fix looks like it’s going to require that I change my code from a WebAPI project to something else that can run on a ‘pure’ Windows VM world (Microsoft support said that there is no problem running phantomjs.exe on a real Windows VM). But before going down that path, I thought you’d like to know this information, because I believe that at one time you had told me that I am the first of your customers to try to run Abot / AbotX on an Azure WebAPI instance. Also, I thought you might have an idea of a workaround (short of disabling javascript rendering!) that would save me the effort of re-writing my code.

Rob Jones

sjdirect commented 8 years ago

Hi Rob,

I can understand why this would be a problem on Azure's PAAS offerings. At the moment the only way for me to support this would be to create another implementation of the javascript renderer using something other than phantomjs. However, I don't think I would create another implementation unless I was able to create one that was better than phantomjs in many other ways as this is very time consuming. Lets leave this open and I'll ponder if there is anything else I can think of.

sjdirect commented 7 years ago

Closing this issue since I have no plans on creating another AbotX.Core.IJavascriptRenderer