search

linux-nfs / nfsd

Linux kernel source tree
Other
0 stars 0 forks source link

Memory safety of server XDR encoding and decoding functions #41

Open chucklever opened 10 months ago

chucklever commented 10 months ago

In the 90s, machine-generated XDR was de rigueur for user space RPC, but kernel developers thought that the generated code was not efficient enough to scale up for NFS. Thus currently all in-kernel RPC code uses hand-written XDR encoding and decoding.

Over time this has lead to the inclusion of special processing in the XDR layer that actually belongs elsewhere.

These days, the efficiency of machine-generated XDR code is comparable to hand-written code. Where hand-written falls short is the risk of introducing memory safety issues: buffer overruns and other problems.

I would like to see how feasible it would be to convert (at least) the server XDR encoding and decoding functions to provably correct machine-generated code based directly on XDR language descriptions of the NFS procedures. This would require scrubbing out the non-XDR activities that this code currently does, so it could be quite a heavy lift.

If there are other ways of approaching good memory safety in this area, we should look into that too.

chucklever commented 9 months ago

I could start by converting the XDR definition of NFSv3 to EverParse3D - or maybe start with just READDIRPLUS.

chucklever commented 9 months ago

Another option would be to rewrite the RPC layer's XDR helpers in Rust, which is memory-safe. It's not clear whether those safety benefits would be conferred on C callers (eg, NFS encoders and decoders).