Investigate the potential mechanisms for project-wide opt-out from unsafe API

[qwwdfsad]

In the face of #354, it's worth considering a project-wide opt-out (one that affects all the dependencies) that effectively acts like a kill switch for the unsafe memory access and replaces it (either by means of static initialization, service loading or java agent) with the "safe" (copying) implementation that does not impose dangers of unsafe API, while pretending to be the one.

[joffrey-bion]

project-wide opt-out (one that affects all the dependencies)

I'm puzzled by this proposal. Are you proposing that such kill switch in my project would affect all external dependencies too (including the stdlib/kotlinx-io itself)?

Some optimized low-level code can use APIs that are unsafe in general but that are in fact locally safe, and encapsulate that in higher-level APIs that cannot make it unsafe. Why would we want to kill the performance of such code with no safety benefit?

[qwwdfsad]

Yes, it's going to be dependency-wide (as we have no control over the notion of dependencies, nest access and things like that).

Why would we want to kill the performance of such code with no safety benefit?

Unsafe API imposes a very non-trivial integrity issue -- if you have an application that uses kotlinx-io, that encapsulates it properly, written in a locally-safe and correct manner, but in your dependencies (whether it's plain simple external dependency, or a dynamically loaded plugin that shares the same classloader hierarchy) there exists one incorrect or malicious usage -- your application immediately becomes compromised because unsafe API works basically as vulnerability multiplier. It's both the source of non-trivial vulnerabilities and supply-chain attacks (in case of dependencies), and an explicit attack vector for things like IDEA plugin system.

For kotlinx-io to be the IO layer, we want to let authors of security-sensitive applications to be able to opt-out from such a capabilities if there is any need to.

[joffrey-bion]

Thanks for the details, that's very helpful.

I still fail to see how exactly this case differs from any other dependency vulnerability. Yes, depending on something malicious is a security issue, but that's true for everything other than kotlinx-io too.

I'm mostly looking at obvious unsafe APIs that allow accessing the internal bytearray behind a bytestring, or things like that. Things using Java APIs directly like byte arrays would be just as unsafe and people have been doing that for years. So I'm assuming you're talking about other unsafe APIs, here?

[fzhinkin]

Regular API has integrity checks preventing accidental or/and malicious access to a data not accessible by a particular Buffer instance and the Unsafe ABI bypasses these checks. I don't think we can make an application safe from malicious libraries in the world where reflection exists, but we can at least provide a barrier against malfunctioning component that accidentely corrupt or read something they shouldn't.

[joffrey-bion]

Thanks. I also agree that there is little we can do about malicious code if people run it as part of their dependencies. It doesn't have to use any kotlinx-io unsafe APIs to be dangerous, so I don't think it would be worth killing the performance of all IO-related code.

So if this about reducing bugs in incorrect dependencies, then what's the difference between that, and suddenly tuning all mutable collections into self-copying persistent collections? It would also reduce bugs, but also at a great performance cost. We just don't do that because it's not worth it: it's the responsibility of the maintainer of a library to fix bugs or decide on the tradeoff between performance and correctness.

Do we have a compelling case where killing the performance in such manner would be worth it? At first glance, the cost seem to far outweigh any benefit I can think of (and I'm usually all for favoring correctness). That's why I'm guessing my "first glance" is just too shallow and I'm probably missing obvious things, hence my questions.

[fzhinkin]

At first glance, the cost seem to far outweigh any benefit I can think of (and I'm usually all for favoring correctness). That's why I'm guessing my "first glance" is just too shallow and I'm probably missing obvious things, hence my questions.

The issue is about investigating potential mechanisms, so various options should be considered. And if the initially proposed solution or any other solutions will have only net-negative effects, we should probably abstain from implementing them, for sure.