Feature request: one POST request per line of the -p file

[Mikolaj]

Right now, it just bundles all the lines in one POST request, so it's hard to benchmark with recorded request traces.

Or, in case people really need content with line breaks, treat blank lines as separators and issue one POST per each block of lines? An alternative is to quote everything, but it's a pain for casual users.

Wrap at the end of the file. Send as many requests as -n specifies. This is different (and better in some scenarios and simpler) to what httperf does, because httperf waits for all responses to arrive before it processes the file again. Also, I think, httperf parses the file each time (sensible if the file is exceptionally large) and we could keep it in memory after the first parse and even keep the whole ready requests to send in memory, to maintain the uhttpc strengths: speed and simplicity.

[hvr]

Sounds like a good idea, under two conditions:

1. Everything must be read into memory (and possibly pre-formatted into a HTTP request) before the benchmark starts, otherwise the benchmark is flawed (as we want to measure the pure network I/O, and as little as possible of HTTP-parsing/formatting/file-I/O)
2. Using a separator in -p would make it quite annoying to use -p for binary data (which I do often); I'd rather use multiple -ps to denote different POST requests to be submitted in round-robin fashion.

[Mikolaj]

Full agreement on the first point (httperf suffers in that way). The second proposal doesn't scale nicely (plenty of files to create in you have a large trace;), but it would still be very useful and adding -l afterwards (request per line) should be rather trivial. Now let's find somebody to pay for the feature or some dup^H^H^Hnoble volunteer to do it for free. ;D

[Mikolaj]

@hvr: Do you know how to use http://hackage.haskell.org/package/cmdargs to allow multiple -p (and -l) options? Right now it only take the last appearance of -p into account. I couldn't find anything even in the Explicit module.

But perhaps we are trying to abuse the convention for command-line programs or at least the simplified convention encoded in CmdArgs? Perhaps overwriting the arguments of the same option is the common sensible behaviour and merging the arguments is an exotic exception? I can't remember, I usually just copy-paste or bring back from shell history any complex command-line tool invocations. ;) But if so we may consider just adding another option --post-binary-files which takes a Haskell-like (or comma-separated) list of file names. Perhaps it's less confusing and easier to edit anyway?

[Mikolaj]

@hvr: ping? any opinions?

[hvr]

@Mikolaj I was gonna merge this... it's just that I was planning to port the code over to optparse-applicative... that's why I'm hesitating

[Mikolaj]

Oh, great, please take your time and let me know how I can help. BTW, not sure if I mentioned, with the -l switch your program works absolutely great feeding long json files to a server. It bombs the server 20 and more times faster than httperf, processes the responses just as quickly and doesn't break connections due to overload (httperf is one of a very few of such tools that have an equivalent of -l).

[hvr]

@Mikolaj fyi, I finally got to convert the code to using optparse-applicative... so if you'd mind rebasing your patch...

[Mikolaj]

Rebased. Note however, that the rebased patch was not tested, unlike the original.

[Mikolaj]

Bump. :)

[hvr]

Landed via uhttpc-0.1.1.0 release!

[Mikolaj]

Thanks a lot. :)