Can I use Lara to build DOM from C# HttpClient?

[zydjohnHotmail]

Hello: In one of my projects, I need to visit one web site and click on some DOM elements to submit some forms to the web server.
The web site contains one main page, and about 100 children pages. From my testing, I can visit one specific child page, and run the following JavaScript to click on certain DOM element, and submit the form.

document.getElementsByClassName('class_name1')[0].click(); document.getElementsByClassName('class_name2')[0].value = 1; var forms = dom.querySelectorAll('form'); if (forms.length == 2) { document.querySelectorAll('form')[1].submit();
}

However, as there are almost 100 children pages, it is not easy to visit each page using browser. I want to know if I can use C# HttpClient to load all children pages, and use Lara to convert the HTML string to DOM, so I can use JavaScript to click on some buttons and submit the form? By the way, I am on the client side, not the server side, as all the mina page and children pages are not under my control, but I can visit them after I login. Thanks,

[pablocar80]

For your app it seems you need something to parse html such as Html Agility Pack and then to click on pages you could use a headless browser like Selenium

[zydjohnHotmail]

Hello: I have tried to parse html by HtmlAgilityPack, but the issue is: the HTTPClient returned only string, without running JavaScript, HtmlAgilityPack parses the HTML, just return nothing for any JS select like: document.GetElementsByClassName, therefore, this way seems not working. I think in Node.js, maybe I can use Fetch to get the web page, and use jsdom to render the DOM, then use cheerio to click on the button, then submit the form, this should work. However, I can't find any solution can bring both Node.js and .Net together. I need to use .Net, as there are quite a lot of complicated math and linq operations, after I got the math results, I know which web page, and which button on the web page I should click and submit the button. Any other suggestions on this issue? Thanks, John

---

From: pablocar80 notifications@github.com Sent: February 12, 2021 2:47 PM To: integrativesoft/lara lara@noreply.github.com Cc: zydjohnHotmail zydjohn@hotmail.com; Author author@noreply.github.com Subject: Re: [integrativesoft/lara] Can I use Lara to build DOM from C# HttpClient? (#103)

For your app it seems you need something to parse html such as Html Agility Pack and then to click on pages you could use a headless browser like Selenium

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/integrativesoft/lara/issues/103#issuecomment-778204781, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKZKVHT644LYZEV7J4ITOBLS6UWPJANCNFSM4XQPIBLA.

[pablocar80]

Would using that in combination with Selenium work?

[zydjohnHotmail]

Hello: I have not tried with Selenium yet. But I have done quite a lot with PuppeteerSharp, for a few web pages, PuppeteerSharp works well, but I may have to do the same job on many web pages, I didn't think it is a suitable solution. Besides, sometimes, if I can load multiple pages using the strings returned by HttpClient, then the project will do a much better job, however using PuppeteerSharp, then I can do only one page at a time. For Selenium I have no experience, but I think it could be the same as PuppeteerSharp. In short, I prefer a solution that I can render a string to make a DOM, then use JavaScript to click on some buttons to submit the form. By the way, do you have experience for other DOM, like: Javascript.Net DOM or ReactJS.NET? Thanks, John

---

From: pablocar80 notifications@github.com Sent: February 12, 2021 3:48 PM To: integrativesoft/lara lara@noreply.github.com Cc: zydjohnHotmail zydjohn@hotmail.com; Author author@noreply.github.com Subject: Re: [integrativesoft/lara] Can I use Lara to build DOM from C# HttpClient? (#103)

Would using that in combination with Selenium work?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/integrativesoft/lara/issues/103#issuecomment-778239032, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKZKVHQZQOKHH6M6VY2TZ6DS6U5UFANCNFSM4XQPIBLA.

[pablocar80]

I don’t have experience in those. Lara won’t fit your need I think, it doesn’t parse HTML and can only execute on pages that have loaded Lara’s client-side controller.

[pablocar80]

@zydjohnHotmail have you tried using the headless version of PuppeteerSharp? That should let you do multiple pages at once.