search

ChifiSource / Toolips.jl

a manic web-development framework
https://toolips.app
MIT License
60 stars 1 forks source link
development julia web

readme

[![deps](https://juliahub.com/docs/Toolips/deps.svg)](https://juliahub.com/ui/Packages/Toolips/TrAr4?t=2) [![version](https://juliahub.com/docs/Toolips/version.svg)](https://juliahub.com/ui/Packages/Toolips/TrAr4) [![pkgeval](https://juliahub.com/docs/General/Toolips/stable/pkgeval.svg)](https://juliahub.com/ui/Packages/General/Toolips)
[documentation](https://documentation.c/toolips) **|** [extensions](https://github.com/ChifiSource#toolips-extensions) **|** [examples](https://github.com/ChifiSource/OliveNotebooks.jl/tree/main/toolips)

Toolips is an extensible web and server-development framework for the Julia programming language.

Toolips is able to create ...

Here is a simple " hello world" project. 

module HelloWorld
using Toolips
# hello world in toolips
home = route("/") do c::Connection
    write!(c, "hello world")
end

export start!, home
end

Here we use route to create a Route{Connection}, home. home is then exported, along with start! -- which is used to start our server.

# starts our server:
using HelloWorld; start!(HelloWorld)
# providing IP
using HelloWorld
start!(HelloWorld, "127.0.0.1":8000)

routing

home = route("/") do c::Connection
    write!(c, "hello world!")
end

To create a Route, we provide the route Function with a target, a String path starting at / to mount the website's base URL and a Function passed through do. The general Toolips process on a route is creating data and then writing it to the Connection with write!. The Function we provide will take a <: of an AbstractConnection. We are able to annotate this argument in our route call to change our route's functionality based on the dispatch. This creates what is effectively multiple dispatch routing, consider the example below:

module HelloWorld
using Toolips

desktop_home = route("/") do c::Connection
    write!(c, "hello world")
end

mobile_home = route("/") do c::MobileConnection
    write!(c, "hello world")
end

# multi-routing our home
home = route(mobile_home, desktop_home)
export start!, home
end

In the case above, mobile clients will be redirected to the latter Function, as their Connection will convert into a MobileConnection.

Routes are stored in the Connection under Connection.routes. We can dynamically change our routes by mutating this Vector{<:AbstractRoute}.

module ToolipsServer
using Toolips
using Toolips.Components

home = route("/") do c::Connection
    new_route = route("/newpage") do c::Connection
        write!(c, "second page")
    end
    push!(c.routes, new_route)
    # creating a quick page to link to our route
    lnk = a("othersite", text = "visit new route", align = "center", href = "/newpage")
    style!(lnk, "margin-top" => 10percent)
    write!(c, lnk)
end

export default_404, home
end

There are several "getter" methods associated with the Connection, here is a comprehensive list:

get_args
get_heading
get_ip
get_post
get_method
get_post
get_parent
get_client_system

All of these take a Connection and are pretty self explanatory with the exception of get_client_system. This will provide the system of the client, but also whether or not the client is on a mobile system. Note that the operating system is given as the request header gives it, of course.

client_operating_system_name, ismobile = get_client_system(c)

There's also

proxy_pass!(c::Connection, url::String)
startread!(c::AbstractConnection)
download!(c::AbstractConnection, uri::String)
respond!(c::AbstractConnection, args ...)

Routes can be exported as any Vector{<:AbstractRoute} or AbstractRoute. Only routes which are exported will be loaded, exporting names which do not actually exist in the project will break the server. The following functions/methods may be used to create new routes with base Toolips:

# creates a regular route
route(::Function, ::String) -> ::Route{<:AbstractConnection}
# creates a `multi-route`
route(::Route ...) -> ::MultiRoute
# mounts the file or directory in the value to the path in the key.
mount(::Pair{String, String}) -> ::Route{AbstractConnection}
module ServerSample
  route()
end

extensions

Extensions appear in Toolips in four main forms:

Connection extensions allow us to utilize MultiRoute with new multiple dispatch Connection configurations. Routing extensions allow us to change the functionality of the Toolips router in different instances. Server extensions allow us to add autoloaded data, or perform actions alongside before our routes whenever a Connection is served. Finally, Component extensions give us more composable Component types to work with, and more high-level web-development capabilities.

Connection extensions are typically used through MultiRoute. This is done by providing multiple routes to route, which will call different routes depending on the incoming client Connection. For example, the MobileConnection is the quintessential Connection extension provided by Toolips.

module Sample
using Toolips

desktop = route("/") do c::Connection
    write!(c, "this page is only served to mobile users")
end

mob = route("/") do c::MobileConnection
    write!(c, "this page is only served to mobile users")
end

# make multiroute
mult_rt = route(desktop, mob)

export mult_rt, start!
end

responses

Like most web-development frameworks, creating websites or APIs with Toolips primarily revolves around creating a response. In the case of an API, this is actually pretty simple. write! will convert any provided type to a String and then write it to the incoming Connection stream. 

module Multiply
using Toolips

home = route("/") do c::Connection
    args = get_args(c)
    arg_keys = keys(args)
    if ~(:y in arg_keys) || ~(:x in arg_keys)
        write!(c, "you have not provided `x` or `y`.")
    end
    write!(c, string(x * y))
end

mutable struct APIClient ip::String requests::Int64 max::Int64 name::String end

getindex(apc::Vector{APIClient}, ip::String) = begin found_client = findfirst(c::APIClient -> c.ip == ip, apc) if isnothing(found_client) throw(KeyError(ip)) end apc[found_client] end end

in(ip::String, apc::Vector{APIClient}) = begin found_client = findfirst(c::APIClient -> c.ip == ip, apc) ~(isnothing(found_client)) end

clients = Vector{APIClient}()

verify = route("/") do c::AbstractConnection nm = get_post(c) allnames = [client.name for client in clients] if length(nm) > 3 && replace(nm, " " => "") != "" && ~(nm in allnames) write!(c, "you are verified, $nm ! Have fun with the crystal API!" push!(clients, APIClient(get_ip(c), 1, 50, nm)) end end

crystals_api = route("/crystals") do c::AbstractConnection args = get_args(c) arg_keys = keys(args) if ~(get_ip(c) in clients) write!(c, "You are not verified! Please POST your name to our home-page to identify yourself.") end end

export crystals_api end

For more detailed websites, we might be building a more complicated response. `Toolips` provides the `Components` `Module`, [ToolipsServables](https://github.com/ChifiSource/ToolipsServables.jl). This `Module` includes the `File` type for easily serving parametrically files by path and `AbstractComponent` types for high-level parametric HTML and CSS templating.
#### files
Files in `Toolips` can either be built manually with the `File` constructor or can be directly mounted to a route with `mount`. `mount` takes a `Vector{Pair{String, String}}`, and will return a `Route` or a `Vector{<:AbstractRoute}` -- depending on whether or not the provided path is a file or a directory. A directory will be recursively routed, creating a route for each file in each sub-directory below it...
```julia

When created manually, a File is able to be written with write!, like normal. This also gives us the ability to use interpolate!, which will interpolate Components by name or interpolate values by using interpolate! in place of write!.

function interpolate!(c::AbstractConnection, f::File{<:Any}, components::AbstractComponent ...; args ...)

For example, using this Method to interpolate HTML with components and values...

<body>
<div>
<h2>hello client</h2>
<a>your ip address is $ip</a>
<h4>would you like to name yourself?</h4>
$namebutton
</div>

This example interpolates HTML -- but is the catchall, or top-level function (binded to File{<:Any} -- meaning you could also write different methods to change behavior depending on file type.

function interpolate!(c::AbstractConnection, f::File{:md}, components::AbstractComponent ...; args ...)
    raw::String = string(f)
    interp_positions = findall("```", raw)
    ...
end

components

This package also allows us to create callbacks for these components...

And ToolipsSession expands on this by providing server-side callbacks and some pretty extreme fullstack capabilities.

templating

As demonstrated in this README thus far, Toolips has a diverse set of a capabilities when it comes to templating. Templating in Toolips is done by constructing and composing components into a body and then writing it to the Connection, or interpolating a file via the interpolate! function.

creating extensions

connection extensions

A Connection extension creates a new Connection which can be used with multi-route, or otherwise with a new router. The running example of this inside Toolips is the MobileConnection.

mutable struct MobileConnection{T} <: AbstractConnection
    stream::Any
    data::Dict{Symbol, Any}
    routes::Vector{AbstractRoute}
    MobileConnection(stream::Any, data::Dict{Symbol, <:Any}, routes::Vector{<:AbstractRoute}) = begin
        new{typeof(stream)}(stream, data, routes)
    end
end

The MobileConnection is created whenever an incoming client is on mobile. This is determined by get_client_system. Two functions are used for this; convert and convert!. convert is called on the Connection to see if the Connection should convert. If it should convert, then convert! is called.

function convert(c::AbstractConnection, routes::Routes, into::Type{MobileConnection})
    get_client_system(c)[2]::Bool
end

function convert!(c::AbstractConnection, routes::Routes, into::Type{MobileConnection})
    MobileConnection(c.stream, c.data, routes)::MobileConnection{typeof(c.stream)}
end

# for IO Connection specifically...
function convert!(c::IOConnection, routes::Routes, into::Type{MobileConnection})
    stream = Dict{Symbol, String}(:stream => c.stream, :args => get_args(c), :post => get_post(c), 
    :ip => get_ip(c), :method => get_method(c), :target => get_target(c), :host => get_host(c))
    MobileConnection(stream, c.data, routes)::MobileConnection{Dict{Symbol, String}}
end

Note that the MobileConnection is actually a MobileConnection{<:Any}. We build a data dictionary in order to turn the IOConnection into a MobileConnection, whereas in the case of the Connection we are provided the standard HTTP.Stream directly. This simple system facilitates both types. Beyond this, you are free to extend other Connection functions to enhance your interface if they are not compatible with your current Connection. Not implementing this will mean that the Connection will not work with multi-threading.

get_ip(c::MobileConnection{Dict{Symbol, String}}) = c.stream[:ip]
get_method(c::MobileConnection{Dict{Symbol, String}}) = c.stream[:method]
get_args(c::MobileConnection{Dict{Symbol, String}}) = c.stream[:args]
get_target(c::MobileConnection{Dict{Symbol, String}}) = c.stream[:target]
get_host(c::MobileConnection{Dict{Symbol, String}}) = c.stream[:host]
write!(c::MobileConnection{Dict{Symbol, String}}, a::Any ...) = c.stream[:stream] = c.stream[:stream] * join(string(obj) for obj in a)

Let's implement a PostConnection in order to demonstrate this:

module PostConnections
using Toolips
import Toolips: AbstractConnection, convert, convert!
mutable struct PostConnection{T} <: AbstractConnection
    stream::Any
    data::Dict{Symbol, Any}
    routes::Vector{AbstractRoute}
    PostConnection(stream::Any, data::Dict{Symbol, <:Any}, routes::Vector{<:AbstractRoute}) = begin
        new{typeof(stream)}(stream, data, routes)
    end
end

function convert(c::AbstractConnection, routes::Routes, into::Type{PostConnection})
    get_method(c) == "POST"
end

function convert!(c::AbstractConnection, routes::Routes, into::Type{PostConnection})
    PostConnection(c.stream, c.data, routes)::PostConnection{typeof(c.stream)}
end

function convert!(c::IOConnection, routes::Routes, into::Type{PostConnection})
    stream = Dict{Symbol, String}(:stream => c.stream, :args => get_args(c), :post => get_post(c), 
    :ip => get_ip(c), :method => get_method(c), :target => get_target(c), :host => get_host(c))
    PostConnection(stream, c.data, routes)::PostConnection{Dict{Symbol, String}}
end

Now let's use it:

module PostSample
using Toolips
using Main.PostConnections
using Toolips.Components

# regular `GET`
home_main = route("/") do c::Connection
    write!(c, h2("main", text = "you landed!", align = "center"))
end

home_post = route("/") do c::PostConnection
    write!(c, "welcome to the API :)")
end

home = route(home_main, home_post)

export home_main, home_post, home
end
routing extensions

Another type of extension that can be created for toolips is the routing extension. Routing extensions are created by extending the route! function. This function may be extended by adding new methods for Route types (<:AbstractRoute), Connection types (<:AbstractConnection), a Vector with <:AbstractRoute as its type parameter, or extension types (<:AbstractExtension).

- `route!` is also called **again** on a `MultiRoute` if a `MultiRoute` is being used. In the binding for the quintessential `MultiRoute` type, for example, the incoming `Connection` checks for conversion into any of the dispatched functions.

All of these considered, there are a lot of ways to extend the routing of `Toolips`.
###### server extensions
###### component extensions

## multi-threading
`Toolips` includes a distributed computing implementation built atop [ParametricProcesses](https://github.com/ChifiSource/ParametricProcesses.jl). This implementation of multi-threading allows us to serve each incoming connection on a different thread simply by providing the number of threads to utilize.
```julia

For the most part, this is straightforward -- but there are some things to be aware of...

session = Session(["/"]) # <- active route "/"

main = route("/") do c::Connection mainbody = body("mainbod") clickable = h3("sample", text = "hello") style!(h3, "transition" => 2seconds) push!(mainbody, clickable) on(c, clickable, "click") do cm::ComponentModifier alert!(cm, "goodbye!") style!(cm, "sample", "opacity" => 0percent) end write!(c, mainbody) end

function load_alert(cm::ComponentModifier)

end

export session, main

end

This is not multi-threading compatible for two different reasons; our `c` is annotated to `Connection`, and our `Session` callback has a `Function` inside of it we will need to serialize. To  avoid this with `Function` callbacks, we simply need to define the `Function` in our `Module`, as it is already loaded to our threads.
```julia
module ThreadedSampleServer
using Toolips
using Toolips.Components
using ToolipsSession

session = Session(["/"]) # <- active route "/"

main = route("/") do c::Toolips.AbstractConnection
    mainbody = body("mainbod")
    clickable = h3("sample", text = "hello")
    style!(h3, "transition" => 2seconds)
    push!(mainbody, clickable)
    on(load_alert, c, clickable, "click")
    write!(c, mainbody)
end

function load_alert(cm::ComponentModifier)
  alert!(cm, "goodbye!")
  style!(cm, "sample", "opacity" => 0percent)
end

export session, main

end

From here, we simply provide the threads key-word argument to start!

julia> using ThreadedSampleServer; ThreadedSampleServer.start!(ThreadedSampleServer, "192.168.1.15":8000, threads = 4)
[ Info: Precompiling ThreadedSampleServer [307046d4-7f21-496b-9a80-f3bfb096e574]
🌷 toolips> loaded router type: Vector{Toolips.Route{Toolips.AbstractConnection}}
🌷 toolips> server listening at http://192.168.1.15:8000
      Active manifest files: 9 found
      Active artifact files: 3 found
      Active scratchspaces: 0 found
     Deleted no artifacts, repos, packages or scratchspaces
🌷 toolips> adding 4 threaded workers ...
🌷 toolips> spawned threaded workers: 2|3|4|5
[ Info: Listening on: 192.168.1.15:8000, thread id: 4
   pid                 process type                        name active
  –––– –––––––––––––––––––––––––––– ––––––––––––––––––––––––––– ––––––
  2080    ParametricProcesses.Async ThreadedSampleServer router   true
     2 ParametricProcesses.Threaded                           1  false
     3 ParametricProcesses.Threaded                           2  false
     4 ParametricProcesses.Threaded                           3  false
     5 ParametricProcesses.Threaded                           4  false
built with toolips

Because Tooips was built primarily to drive other chifi software, ChifiSource has created a number of projects with Toolips. Here is a list of large projects we have created based on Toolips, along with their repository links.

I thank you for all of your help with our project, or just for considering contributing! I want to stress further that we are not picky -- allowing us all to express ourselves in different ways is part of the key methodology behind the entire chifi ecosystem. Feel free to contribute, we would love to see your art! Issues marked with good first issue might be a great place to start!

guidelines

When submitting issues or pull-requests for Olive, it is important to make sure of a few things. We are not super strict, but making sure of these few things will be helpful for maintainers!

  1. You have replicated the issue on Unstable
  2. The issue does not currently exist... or does not have a planned implementation different to your own. In these cases, please collaborate on the issue, express your idea and we will select the best choice.
  3. Pull Request TO UNSTABLE
  4. Be specific about your issue -- if you are experiencing multiple issues, open multiple issues. It is better to have a high quantity of issues that specifically describe things than a low quantity of issues that describe multiple things.
  5. If you have a new issue, open a new issue. It is not best to comment your issue under an unrelated issue; even a case where you are experiencing that issue, if you want to mention another issue, open a new issue.
  6. Questions are fine, but not questions answered inside of this README.