GSoC 2017: Middlewares for common web application chores in Mux.jl

[sarvghotra]

I am opening this issue just to discuss the matter related to the above mentioned project. I am an aspirant to participate in GSoC 2017 with this project.

I have made couple of pull requests, can someone please take a look at them.

@MikeInnes

[sarvghotra]

Following are the functionalities that I think are required. But it would be really nice if others can review it and give some inputs.

1. Routing (based on)

• [ ] Scheme
• [ ] Subrouter - Tree like routing, children nodes are tested only when parent matches
• [ ] Default - When nothing matches
• [ ] Header-value
• [ ] Custom matcher - Accepts function to perform routing
• [ ] Regular expression support
• [ ] URL host
• [ ] Query
• [ ] Variables in the path support eg. - route('/myapp/{user}/tags/', myapp)

For session and secure session (3rd point), I am taking inspiration from WSGI's Beaker

2. Session

• Session middleware
• Flash session - session remains valid until read
• Multiple backend option - File based, DBM files, memory, memory cached
• Extensible backend - Support for other backends
• Lazy loading session - Only when a session object is actually accessed will the session be loaded
• Cookie based session - client side storage (limited size)
• Multiple sessions per request

Functionalities

• [ ] Datadir - an absolute path to the directory that stores the files
• [ ] Timeout - seconds/hard deadline until the session is considered invalid
• [ ] Config - dict based function to set the configuration of a session
• [ ] Type - the name of the back-end to use for storing the sessions
• [ ] Save - save the session
• [ ] Auto - the session will save itself anytime it is accessed during a request
• [ ] Delete - delete a session (no session to be used further)
• [ ] Invalidate - If a session should be invalidated, and a new session created and used during the request
• [ ] Cookieexpires - when the cookie used to track the client-side of the session will expire
• [ ] Maxlength - max length of the cookie value
• [ ] Cookiedomain - what domain the cookie should be set to

3. Secure session (Encodes and decodes authenticated and optionally encrypted cookie values)

• Generate random key
• decode/encode - AES like encryption
• Hash function - HMAC like method to validate

Functionalities

• [ ] Secret - ensure session integrity using HMAC or similar method (takes a random key)
• [ ] Secure - to instruct the browser not to send the cookie other than SSL
• [ ] Encode / decode - to encode and decode
• [ ] Encryptkey - key to use for the AES cipher.
• [ ] Validatekey - key used to sign the AES encrypted data

Next two points, I am still figuring out. Any pointers for inspiration would be great.

3. Load balancing

4. Authentication

[MikeInnes]

Hey Sarvjeet, sorry for not getting back earlier on this. This looks like a good list of functionality. I'll try to review your PRs over the next couple of days.

[MikeInnes]

I like your work so far. Can you submit a proposal? Would like to see some thoughts on HTTP.jl, as well as what you'd use as a test case. Perhaps a reimplementation of juliaobserver.com in julia?

[sarvghotra]

@MikeInnes I have submitted a proposal, just now. Please take a look at it. My display name is Sarvjeet Ghotra.

Will get back to you soon, after some investigation on it.

what you'd use as a test case. Perhaps a reimplementation of juliaobserver.com in julia?

Could you please elaborate it a little bit ?

Would like to see some thoughts on HTTP.jl

[MikeInnes]

Will take a look. I think it'd be good to build a small web app as a test case, (a) to prioritise features from the list you have above, and (b) to show off and document what Mux can do. So that's one idea. Up to you if you have alternative ideas on how to do those things, of course.

[jpsamaroo]

So, I've spent a (small) bit of time trying to get at least a few of these addressed. Since this issues dates back a few years, I'm not sure how much of this still needs implementing. Hopefully someone can let me know if my understanding of the state of this issue is correct, and let me know if I got something wrong:

1. Routing (based on)

• [ ] Scheme

No idea what this is This is HTTP/HTTPS/etc. I'll be working on updating #38 when I find the time.

• [ ] Subrouter - Tree like routing, children nodes are tested only when parent matches

Handled by nested branches

• [ ] Default - When nothing matches

Is this necessary? Just adding another middleware "below" a stack of not-matching middleware should accomplish this, right? Like Mux.notfound() in the examples in the README.

• [ ] Header-value

To be implemented

• [ ] Custom matcher - Accepts function to perform routing

Handled by branch?

• [ ] Regular expression support
• [ ] URL host

See MuxMiddleware

• [ ] Query

WIP from #39

• [ ] Variables in the path support eg. - route('/myapp/{user}/tags/', myapp)

Handled by page?

For session and secure session (3rd point), I am taking inspiration from WSGI's Beaker

1. Session

• Session middleware
• Flash session - session remains valid until read
• Multiple backend option - File based, DBM files, memory, memory cached
• Extensible backend - Support for other backends
• Lazy loading session - Only when a session object is actually accessed will the session be loaded
• Cookie based session - client side storage (limited size)
• Multiple sessions per request

Functionalities

• [ ] Datadir - an absolute path to the directory that stores the files
• [ ] Timeout - seconds/hard deadline until the session is considered invalid
• [ ] Config - dict based function to set the configuration of a session
• [ ] Type - the name of the back-end to use for storing the sessions
• [ ] Save - save the session
• [ ] Auto - the session will save itself anytime it is accessed during a request
• [ ] Delete - delete a session (no session to be used further)
• [ ] Invalidate - If a session should be invalidated, and a new session created and used during the request
• [ ] Cookieexpires - when the cookie used to track the client-side of the session will expire
• [ ] Maxlength - max length of the cookie value
• [ ] Cookiedomain - what domain the cookie should be set to

1. Secure session (Encodes and decodes authenticated and optionally encrypted cookie values)

• Generate random key
• decode/encode - AES like encryption
• Hash function - HMAC like method to validate

Functionalities

• [ ] Secret - ensure session integrity using HMAC or similar method (takes a random key)
• [ ] Secure - to instruct the browser not to send the cookie other than SSL
• [ ] Encode / decode - to encode and decode
• [ ] Encryptkey - key to use for the AES cipher.
• [ ] Validatekey - key used to sign the AES encrypted data

To be implemented; I think this should go in its own package, like MuxSessions.jl or something like that, since this would be quite a bit of work.

Next two points, I am still figuring out. Any pointers for inspiration would be great.

1. Load balancing

This might also be its own package (along with other "meta" functionality); my recommendation for this specific one is to just make a simple function that can be given a Vector of host:port Pairs, and will redirect/proxy to them, with scheduling being user-defined (built-in for Round-Robin, with an example or two of somthing more complicated).

2. Authentication

Definitely should be in its own package(s); authentication mechanisms are a broad landscape.

And here are some of my own ideas that I might implement eventually (not to be tracked here, just mentioning them):

• Stacked caching - so you can layers caches for efficiency, e.g. In-memory LRU -> Redis -> PostgreSQL
• Sticky-sessions - when load balancing, can attach a cookie that will allow future connections to be automatically redirected to the previously-selected backend webserver
• Traffic statistics - using Cassette (because I can), track the paths that requests take through a webapp tree, and allow it to be used for routing purposes (like in a load balancer) or just logged somewhere for later analysis
• Many, many more...