CWA Golang MVP

[wg102]

This ticket is to create a Minimal Viable Prototype (MVP) of CWA Jupyter.

The goal is simply to have the table that displays the notebooks per namespace. And visually the buttons/columns but not with any of the functionality working yet. This way, we will have a quick introduction to GoLang, be able to estimate better any tasks related to Golang refactoring, and have the basic structure to run the code locally.

This only includes the notebooks table. Not kubecost and not the volume table (the later still need to have decisions about UX on how we include it in CWA). It does not include deletion or creation of notebooks. It does not included i18n.

We plan to do peer programming for it, in order to agree on how the other items should be tackled. any findings that will affect CWA and it's work will be reported under the parent ticket.

[Jose-Matsuda]

Things to figure out:

• [ ] How to build so it can be tested
• [ ] How to test / verify / the MVP which would include using it as the backend when we are running the frontend (probably already some knowledge for this, but not in the upstream specific folder format)

[Jose-Matsuda]

Observations (this comment will be updated as more become clear, or if something is unclear or unhelpful will delete areas): From Home, bring up the dev tools and hit "Notebooks" the resulting calls are seen in the picture below

Notebooks

Request URL: https://kubeflow.aaw-dev.cloud.statcan.ca/jupyter/api/namespaces/jose-matsuda/notebooks Request Method: GET :path: /jupyter/api/namespaces/jose-matsuda/notebooks

pvcs

Request URL: https://kubeflow.aaw-dev.cloud.statcan.ca/jupyter/api/namespaces/jose-matsuda/pvcs Request Method: GET :path: /jupyter/api/namespaces/jose-matsuda/pvcs

[wg102]

Idea for the structure of the files. Initially we wanted to keep the file structure close to upstream to help managing differences and upstreaming in the future. Following our investigation, we find that the structure of the upstream file is more complicated then needs be, especially in the common folder. To avoid said complexity, we are looking into a leaner version of folders that would be a hybrid between the clean code in our current jupyter-apis repo, and the upstream division. The idea would be to have 3 or 4 folders in the common folder (under /common/backend-go of course)

Common folder

• Route: Similar to their route idea. There would be a file per type of route, like upstream currently but less divided.
  • Get.go
  • Delete.go
  • Post.go
  • Patch.go
• Api : All the methods, divided per type
  • notebook.go
  • pvc.go
  • namespace.go
  • etc
• Error : All error handling since it will be common to the entire Crud-Web-App
• Everything else : csrf, structures etc.

Jupyter folder

All other folder would keep the current upstream structure, since they seem to have a lot less subfolders. At worse, they will addopt a similar folder as the ones explained above

[Jose-Matsuda]

A good link to keep around is https://github.com/kubernetes-client/python/blob/master/kubernetes/README.md as this should illuminate what types of objects the code expects, especially if we choose to not implement the api calls right away and spoof it.

Example) In the route for listnotebooks It returns an object as defined here in the notebook.py. Now to find that we have from the apis.py file it says

v1_core = client.CoreV1Api()
custom_api = client.CustomObjectsApi()
storage_api = client.StorageV1Api()

And following that custom_api trail in the readme from above we can get to the definition in what it expects to be returned.

The other docs are here

[skyeturriff]

Choosing the web framework

There are a bunch of frameworks out there that could meet our needs, some more lightweight libraries and other full-fledged frameworks. Depending on our use cases, and whether we'd prefer to work with something minimal with basic routing capabilities, or are looking for more functionality in terms of middleware functions like logging, error handling, etc.

gin

• Most popular Go web framework
• Full fledged support for routing; request handlers can be attached to routes
• Built-in support for request parsing, formatting responses, json marshalling/demarshalling into structs, logging, auth, error management, and extensible to custom middleware functions
• As a "rule of thumb", you will have to write less code when using Gin vs other lower level frameworks
• May be much more than we need

mux

• Fast and simple routing with support for path-matching using URL host, path, path prefix, schemes, header and query vaues, HTTP methods. Also supports custom matchers. (The matching based on HTTP verb and path/path prefix I think would be very useful in our case).
• Support for subrouting to "nest" routes, which can be use to create routing groups (i.e. routes that share the same prefix)
• Includes support for basci helpers like extracting route paramsa
• Uses standard request handlers so any extra functionality that we'd need we would have to implement
• There may be separate libraries that we can pull in form the Gorilla toolkit that can handle more features

chi

• Stil lightweight a good mix of built-in routing functions but also has some of the added functionalities present in gin. A good middle point between gin and mux
• Fast
• Still conforms to the stdlib signature for its middleware functions (unlike gin), so may be more flexible in this sense

Bottom line

Gin has a lot of built-in support for a wider range of functionalities, meaning we get more out of the box and fewer lines of code are needed, which in certain cases may lead to a bit lower performance. However this framework is still highly rated for performance. Using something lighter weight like mux or chi, which would be mostly using them as routing helpers, we can capitalize on the performance benefits but may have to do more work to implement things like parsing requests, handling errors and response, logging. etc. If there is not much middleware-type functionalities needed for our endpoints, or we don't mind the extra lines of code, we are probably better going with a more lightweight library. Many sources suggested starting with something like mux or chi, and if it doesn't meet needs, then adopting gin.

Refs: https://news.ycombinator.com/item?id=25012115 https://www.alexedwards.net/blog/which-go-router-should-i-use https://dzone.com/articles/comparison-of-popular-web-frameworks-in-go https://brunoscheufler.com/blog/2019-04-26-choosing-the-right-go-web-framework https://github.com/go-chi/chi

[wg102]

Removing this from the sprint. See issue above. any work for 1.3 will be done in this epic

[chuckbelisle]

@wg102 please review and close if no longer required

[Jose-Matsuda]

closing as no longer required at this time