Upstream 2023 03 01 1

[illusional]

Based on https://hail.zulipchat.com/#narrow/stream/300487-Hail-Batch-Dev/topic/resource.20table.20query.20woes/near/338629272, merge to 9e0081c and future commits (none of which are available now) need to be merged separately.

@lgruen, this the main merge about the memory service that we could potentially reverse the hack, but I don't know much about this, I just merged to the most relevant branch:

 val (open, write) = if (n <= 50) {
  (fs.openCachedNoCompression _, fs.writeCached _)
} else {
  ((x: String) => fs.openNoCompression(x), fs.writePDOS _)
}

Out version was merged to:

val (open, write) = ((x: String) => fs.openNoCompression(x), fs.writePDOS _)

[lgruen]

That merge looks right if we still want to disable the use of the memory service (which I think we do). There have been some improvements to it recently though. I'll add it to the agenda for our catch-up on Friday.

For now, I think we should just go ahead with the version we had so far.