Closed thomcc closed 1 year ago
Considering the size and scope of the PR, this doesn't seem overly complex. It boils down to having the experience and knowledge to know if the right parts of the stdlib have received the right treatment.
To that end, I want @workingjubilee to review and approve this one.
This fixes many, many things that were unintentionally exposed, broken, needing reconsidering, or that should have been fine but weren't[^1]. This is mostly a consequence of the fact that we gained a much better idea of what we need from
postgrestd
between when it was written and now.Sadly, it's an enormous single PR, which is mostly due to time constraints.
[^1]: I mean, really, why is is there so much low level networking code in wierd places?
libc
usage instd::sys_common::net
is one thing, butstd::os::unix::net
should not be so large, complex, and full of syscalls!Removed APIs and bugfixes
Alright there's a lot here. Almost everything should not impact public API, only the runtime functionality. A single exception to this exists:
std::os::unix::{ucred,net}
are currently removed (rather than just having their functionality stubbed out). These should be re-added once we're confident they're fully disabled, but in the short term (see below for more details).Where possible, the changes were either done in a way that should minimize likelihood of merge conflicts, although in cases where we'd fail to notice a problematic addition, I went with the merge conflict route.
std::net
now is fully blocked from performing networking.std::os::{unix, linux}
no longer grant access to networking (or other system functionality...)std::os::unix::{ucred, net}
are compiled out entirely for now.net::unix
return errors has been done, but thelibc::
access is so extensive compared to how rarely used these APIs are... that I think it's worth disabling until we have some way of checking we got everything.Stdio functionality is now also fully disabled (rather than just being inconvenient).
Basically all other system I/O code (e.g. the underlying
FileDesc
type has no avaliable IO anymore) has been disabled too. This mostly impacts what external libraries do, but should help us avoid anything creeping back in in the future.Backtrace capture is now disabled (via panics/hooks,
std::backtrace
, ...). These contained tons of system information if you manually parse their output (way more than we're probably comfortable with). In the future we may attempt to find a way to expose this more safely (writing one to the log, perhaps).Panic hook functionality has been reduced / adjusted.
stderr
.Threading functionality is more completely disabled.
pthread_t
is now blocked.sys::locks::*
, for example.futex
es are longer used by various synchronization code, we just use the generic parker and Once now.Internal error handling is now slightly more considered. In particular, our old
errno()
function would get us into trouble as it previously always claimed success.assume_init()
ed a potentially-uninitialized value.Other pieces of the internals have been tuned up a bit.
all the code that used to did
cfg(target_os = "postgres")
now has the correctcfg(target_family = "postgres")
abort_internal()
now aborts properly again, since several of the places where it's used are inside the panic impl, which leads to stack overflow.std::process::abort()
is still apanic!
as before.abort()
by panicking inside aDrop
while another panic is running. (There's probably no way around that).A bunch of spooky system code that was completely wrong for our use case is removed. This almost certainly was not actually being used, but it's definitly gone now.
Stuff needing futher work/thought
None of this is a blocker, but here are some things I don't love and would like to get back to.
Right now it compiles with unused import warnings, which I've silenced. I'll get to it.
(As mentioned) I'd like to go back to finish up
std::os::unix::{net, ucred}
, since I don't like having a changed public API. These are pretty obscure so there's not a lot of urgency there.I'd like to think more about the overridden panic hook, but I have almost no concrete thoughts there.
We should have a better impl for
std::sync::Once
and the thread parker, I think the current still probably still broken in some cases (deadlocks). Asys_common::once
impl for single would be good.Loss of backtraces will probably make debugging harder, so we should have a way to compile them back in during development.
I don't trust
std::is_{x86,aarch64}_feature_detected!
not to be considered system information exposure. We should consider replacing it with a version that always reports baseline features.This changes a lot more than the old version, so we need a way to stay on top of the changes going forward.
I kind of punted on the error handling/errno stuff after I realized how annoying it would be to do right, and just focused on the important parts. Would like to come back.
Find some way of running tests. The stdlib's own tests are obviously unsuitable (because we disable most of it), so it's not clear what we do want?
pgx
's tests, and maybe the tests of it's dependencies?Definitely other stuff.
Bonus: Includes finished macOS support
Added bonus: macOS is sort of supported now in a mostly experimental configuration. That is, this replaces #40, since while working on it I noticed these issues (and the
cfg
fixes here make macOS support nearly automatic. Not to mention, my only fast machine is a macbook, so I'd still be working if I hadn't added it)One notable wonkiness about the support is that we can't use the same target vendor convention we've been using on apple for two reasons:
target_vendor = "apple"
is used a fair bit in the wild to detectany(macos, ios, tvos, watchos)
, and the stdlib (and folks in the wild) has build-time code which assumesenv::var("TARGET").unwrap().contains("apple-darwin")
is the way to check for macOS. Honestly I don't blame either, so I've stuck it at the end for now, as<arch>-apple-darwin-postgres
, which seems to work.