[WiP] LDP functionality

[michielbdejong]

Comparison:

• Gold: 56 passing tests (Total tests run: 90, Failures: 6, Skips: 28)
• Trellis: 52 passing tests (Total tests run: 90, Failures: 9, Skips: 29)
• ldp-resource-store: 40 passing tests (Total tests run: 90, Failures: 18, Skips: 32)
• NSS: 22 passing tests (Total tests run: 90, Failures: 21, Skips: 47)

[RubenVerborgh]

That’s just awesome 👏

[michielbdejong]

comment to test CLA setup

[michielbdejong]

Drastically simplified the code architecture, so that there is still a concept of workers, but they don't pass messages on to the next worker; instead, they expose a handler that returns a result or throws an error.

39 tests passing, and counting.

will keep working on this for the rest of the week and then submit as a pull request to a repo on the inrupt org. Thinking of calling it wac-ldp-kit.

[RubenVerborgh]

Why the workers though? Is there a discussion on necessity/advantages/disadvantages?

[michielbdejong]

The workers came out of two discussions, one about how we can achieve the highest level of modularity, so that for instance @kjetilk's Auth* Proxy idea is also implementable by simply connecting a shuffled set of workers together. Each worker could in theory be an npm module, although they would only have value in combination with the other ones, so that's why I thought I might as well keep them in one repo.

The other discussion was about the idea of microservices, but simplified because these workers live in the same process and there's no message queueing. So since each of my workers comes with a Task type and a Result type, and the worker itself is stateless and only exposes one function that converts a Task into a Result, it should be easy to write unit test for such a worker and reason about how it works.

I went through two experimental variations and I hope you agree that the third one, here, is a lot better than the previous versions, and that it's now a nice balance between straight-forward but still enjoying the benefits of high modularity. At least I'm starting to be pretty happy with it now. :)

[kjetilk]

This sounds really great!

I think I would still prefer to have those that are relevant to the proxy in a separate npm package, so that you would only need to install the ones that you'd actually use though.

[RubenVerborgh]

@michielbdejong Thanks for explaining.

However, if I may be critical (and I take that to be my job), then I must say that none of the listed benefits are a consequence of the worker/task-based design.

• Unless I am missing something, modularity is not affected by worker/task at all. Workers will only accept a specific type of task, so it would be the same as wiring together things with non-worker interfaces. (e.g., no difference in wiring Worker X for task A and Worker Y for task B together, as opposed to simply wiring components with interfaces A' and B').

• Unit-testability is a consequence of a good modular design, but not specific to worker/task.

• Microservices similarly do not demand a worker/task interface.

So I currently only see the extra complexity created by worker/task (= two classes for everything, and things need to be worker-shaped instead of taking the most natural form for the thing to implement), but no additional benefits. Having ResourceDeleter, ResourceUpdater, etc. imposes a very specific architecture, but I don't think we have sufficient arguments to justify it.

Perhaps in a concrete question: the task/worker paradigm complexifies the code considerably, but for what gains that cannot (easily) be achieved without it?

[michielbdejong]

To do:

• [x] debug instead of console.log
• [x] lint
• [x] jsdoc
• [x] unit test setup
• [x] server should instantiate storage and then use that to instantiate app
• [x] use LdpParserResult instead of LdpTask
• [x] If-Match in ResourceWriter
• [x] create a class (not string) for AtomicTree paths
• [x] Rename Worker#handle to Processor#process
• [x] rename LdpParserResult to LdpTask, or at least something that's independent of the LdpParser processor.
• [x] If-None-Match in ResourceReader
• [x] implement ResourceDeleter
• [x] implement ContainerDeleter
• [x] better LdpParser documentation
• [x] pick good CLA text and set it up on inrupt repo

Follow-up:

• get more LDP testsuite tests to pass
• more unit tests
• implement GlobReader (since it's not part of LDP as such)
• implement WAC-related workers, see if i can do something like Ruben's Representation interface at the level that handles Turtle and Json-LD.

[CLAassistant]

All committers have signed the CLA.

[michielbdejong]

first draft ready and mostly working, will read through the whole PR myself once more now and then submit this for review.

[michielbdejong]

Split into https://github.com/inrupt/wac-ldp/pull/6 https://github.com/inrupt/wac-ldp/pull/7 and https://github.com/inrupt/wac-ldp/pull/8