Create a remote dataset benchmark

[westonpace]

Still probably some testing to do here. The sample datasets work ok. I tried collecting the entire dataset (and not specifying any filter/query) but ran into some issues so I want to debug that a bit further.

[jonkeane]

/document

[jonkeane]

Thanks for looking at this + putting this all together. I made a few changes to the structure (and then a bunch of smaller changes to the arrowbench package that I uncovered as I worked on this, which should have been done already, but no time like the present!)

The major things I did was:

• Integrated these new datasets into the known_datasets object. They are just like the other dataset that we have (other than the fact that we don't plan on downloading/cacheing it) so I thought they fit better there. The ensure_dataset() almost worked without needing to make it remote-specific (and I made the one change we needed to support these list-all-the-file datasets which AFAIK weren't actually supported in R when we wrote {arrowbench})
• There were a few things that I hardcoded in (like the schema) instead of pulling it from the file. This is definitely not good standard practice, but for these benchmarks IMO it's good to have this stuff hard coded so that if things change suddenly we know about it. This is similar to having the dimensions hard coded in. (note: I did this in a function so that it was created in the subprocess that the benchmarks are run in)
• I had to adjust the result output parsing to add results_sentinel_end due to a silly R warning about needing to auto close a file that's not closed — I need to look into that more and file a Jira about it if it persists, but I was seeing it on my system. In practice it's not a big deal, but it makes arrow slightly noisier than it should.
• I took out the test versions of the datasets — we only have a handful of small test datasets that ship with the package to facilitate the unit tests in the benchmarks package. We could add small versions of these files in there + test them, but considering the point of these is to do it over S3 I don't think that we should add them. If they work here, they should work on conbench and if they don't there we'll see it and can fix it.

[westonpace]

Thanks for cleaning things up @jonkeane . I looked over your changes and it all looks good to me. Simpler and cleaner and it still tests the same thing :)