metanorma / obp-access

Data access on ISO OBP
1 stars 2 forks source link

Use direct HTTP request to load the page #8

Closed ocvit closed 5 months ago

ocvit commented 5 months ago

Metanorma PR checklist


Guys, you don't need a freaking browser to load a single page.

This is a simplified example. Normally, you would introduce a proper HTTP client (external gem or wrapper around net/http), request retries, response validations, user agent injection (this website doesn't care really), cookie injection (technically, they use JSESSIONID but it does nothing and can be skipped) and other standard stuff.

ronaldtse commented 5 months ago

@ocvit this is absolutely better than using a browser. The initial consideration of using a browser stemmed from not know how Vaadim pages work, but since you have gotten it to work with net/http it is of course so much better. Thank you for your contribution!