Support NFS

[mmattioli]

Proposal

Implement an NFS module so that images can be booted over the network rather than loaded into local memory

Reasoning

As previously mentioned in #33, there are inherent advantages in being able to mount the filesystem over the network and not have to rely on the local memory in order to do so. Furthermore, it allows for greater flexibility with the PyPXE package as a whole.

[psychomario]

Even though I can't get to it for a week, I would like to request doing the initial implementation of this.

[mmattioli]

I don't care when we get to it, I just wanted to open the issue so that it was documented and we know it's something to work on (whenever that may be)

[mmattioli]

@psychomario just curious if you ever got around to drawing out a rough implementation for this?

[psychomario]

I looked into it and it turns out this is a huge project. I plan on starting it in two weeks when I move back to uni. I'll probably do it as a separate project we can submodule.

[mmattioli]

Submodule in reference to a Python submodule, correct? I hope you're not referring to git submodules, they're a pain to work with lol. We could probably name the module nfs.py and then make the main class nfsd() following suit with the others.

[psychomario]

I had meant git submodules. I don't think it would be wise to cram it all in one file, there are lots of things we would need to implement, if we don't split it up it would become unwieldy very quickly.

[mmattioli]

I'm not too thrilled about git submodules, they're such a pain to work with and make it difficult for others to use the code. The whole purpose of adding NFS support is so that it's a part of PyPXE so why wouldn't we just make it part of the PyPXE module like the rest of the services?

[psychomario]

Because its going to be so big. I suppose I could add it in as a subdirectory under the modules, so we can split it down but still import it as one.

[mmattioli]

That's more in line of what I was thinking. Add a submodule to the PyPXE module so that it still works as such

from pypxe import nfs

where nfs would be a folder within pypxe that had another __init__.py so that it imports cleanly and we don't need to bother with git submodules.

[psychomario]

I've started writing up a document on implementing this, and will start the programming soon. It's under my nfs branch, /pypxe/nfs/

[mmattioli]

bump

[psychomario]

Apologies, I never got past writing up some docs for this. I should have some time at the end of this week to start fleshing out some of the functions in pypxe/nfs/operations.py

[psychomario]

Just a quick update: I've just now got a successful mount. I should have a directory listing shortly.

I expect to have a pull request ready over the next week or so. This will hopefully include rw access and authentication.

[mmattioli]

:+1: Nice!

[psychomario]

I've managed to get read only access working fine.

I just removed all the socket closing code, and pushed each connection to it's own thread and there's been a huge speed increase. I also significantly improved network reading so I'm no longer having crashing issues from clients batching requests.

[mmattioli]

Nice. So closing the sockets takes up a noticeable amount of resources? Or is it just faster across the board to separate them into their own threads regardless of whether or not you close the sockets?

[mmattioli]

Apologies, clicked the wrong button facepalm

[psychomario]

The client was creating hundreds of TCP connections per second, which the server couldn't handle in a single thread. If the connections aren't closed, the client uses the same one, so we get a free single buffer, of sorts, rather than having to quickly serve all the unique connections.

Thinking about it, for a single client (as all my testing so far), threading won't make a difference, as it just parses until the connection is closed.

[psychomario]

Another update: I've managed to get a client to mostly boot over nfs. Systemd's Login Service seems to be failing. I'm going to start implementing write properly tomorrow (most relevant functions have skeletons so far), to see if I can accidentally fix the problem.

There are instructions under pypxe/nfs/README.md in the nfs branch if anyone is interested in trying it out.

[mmattioli]

I'll take a peek at your instructions and see if I can think of something

[jcea]

Bump!

[icb-]

Unfortunately, I don't have time to work on NFS support at the moment. If someone does have the time, I would gladly accept PRs.