Investigating the ability to navigate through POST

[johantell]

Hi!

I've ran into a case where I would like to navigate with a POST requests rather than a GET to a url that I want to print a PDF for. Essentially it's a workaround for a 414 URI Too Long error.

For that I figured I would be able to evaluate an expression where I create a form an post it to the URL I want to print from.

However that of course messes up the wait_for option since navigating to a new page removes the loop that checks for the property.

Do you see a place where chromic_pdf could be extended to send posts and do the evaluation the wait_for expression after the new page has been shown? Or do you have any good ideas on how I could make use of the current API's to achieve the same thing?

I assume I could write my own protocol macro for it but then I'd be losing out on updates/improvements on main

[maltoe]

Hi @johantell

Do you see a place where chromic_pdf could be extended to send posts...

Short answer: No.

Long answer: I've never tried, it could be possible, but I'm sceptical about the complexity tradeoff.

To begin with, I don't know of an API in the DevTools protocol that is capable of sending POSTs like form submits (Page.navigate does not cut it). If there is, and it could be implemented in a way that it replaces Page.navigate, I'd be open for it.

If there is no such API, we would need to implement something with scripts, i.e. Runtime.evaluate. For instance, one could initially navigate to about:blank, then do the actual desired navigation (with GET or POST) in a script, wait for the script to terminate (Runtime.evaluate will return an error because the execution context of the script is gone once you navigate away), then continue with another wait for frameStoppedLoading and the rest of the Navigate protocol. All in all definitely possible, but quite complex and finicky, IMO.

Could you elaborate a bit on your use-case and why you have such long URLs? I'm assuming the long URL contains data you want printed, is that right? Is there no other way to transmit the data on a side-channel? Since you're thinking about extending ChromicPDF, I'm assuming the PDF printing service is also under your control, right? Could you give it some state where you could first store your data (uploaded via some API) and then make ChromicPDF navigate to a shorter URL which just renders it?

[maltoe]

Side-note: The Runtime.evaluate solution outlined above somewhat points in the direction of "arbitrary remote control over Chrome", much like wallaby / selenium / other browser automation libs (e.g. used for E2E tests) can offer. I was asked about features like this repeatedly in the past ("is it possible to click a button / fill a form / do xyz with ChromicPDF?") and the answer always is: Yes, theoretically, but it's a lot of work, gets complicated quickly and we lose the little sweet spot we've found for ChromicPDF. I don't want to sacrifice the friendliness of the API or the simplicity of the implementation for (at best mediocre) support for all imaginable remote control use-cases / all of DevTools. Hope you understand this POV.

[johantell]

Thanks for the quick reply @maltoe!

I totally understand the concern of opening a can of worms with going down the route of remote control and agree that it is important to keep the API simple and clear.

In short, what we're trying to achieve is to pass a set of encrypted options to a separate server (we want to separate the node running a chrome from our production one) which will unwrap it and as our main server to render a page. In cases where we have a lot of settings passed to it the query param we're passing the information with becomes too long and the server refuses to take care of it.

We could of course store the store the settings and identify it by a token on our server but we wanted to prevent having to store it in the database. So we have a way of solving it, just wanted to see if we could sort it out without database requirements (in memory would have been nice but that's of course another can of worms to open :D).

Anyway, I totally understand not wanting to go down that route so we'll revert to the other strategy instead :)

[maltoe]

Just one more idea: You could make the template a LiveView and push the data via evaluate option (AFAIK no length limit on the expression string) and a LV event to the server (push_event). This way your script wouldn't terminate as no navigation happens :shrug: And afterwards wait on some selector again. In theory, this should work, so not sure how reliable it would be.

[edit] or if you don't want a LiveView, you could replicate its behaviour: send your encrypted data to the server via xhr or fetch and modify your DOM with the response