[RFC] Ribbit Core Architecture

[mrbut]

What?

This RFC is an outline for an approach to the core Ribbit application's plugin architecture.

The problem?

Ribbit in its base MVP state is coupled with a few technologies: Webpack for bundling and code splitting, React, React-Router, React-Redux.

This causes a few problems:

1. The Ribbit application is set-up so at some point in the future require a major overhaul if the team wishes to switch to another server.
2. The application is not agnostic. It depends too heavily on React to build static assets. It assumes that react-router is the routing mechanism, and it assumes that Redux is the state management library of choice (or that any state management library is used at all).
3. The application does not provide entry points for plugging additional functionality into the process.
4. Lack of modularity introduces ambiguity for ownership of functionality. A reader is required to read through the entire application to begin to create an understanding of how to break down the steps of setting up Universal components.

The solution?

1. Modularizing the application in the server

To modularize the application into its building blocks. (Note: each building block is a separate PR to ensure there is space to go into the details.)

We can map our application into six phases. The six phases to set up Universal JS are:

1. Routing - mapping components to routes
2. Serializing - exporting state/data required for render from the client side to send as an object in the server's response
3. Execution - Bundling the root of the application and stringifying
4. Rendering - Constructing a template literal of an HTML file with injected stringified app bundle, scripts, and interpolating any additional template data. Then writing the HTML template literal strings to file aka the HTML static assets.
5. Response - in the case of ribbit, the response is generating a JSON manifest file that maps routes to the static assets created in the rendering phase.
6. Deserializing & Attaching - Ensuring that when the SPA adds event handlers to the page, its data/store get hydrated with the data/state sent with the server response.

These building blocks— which are referenced as a "phase" or "phases" later in this document—can be used as a means of modularizing out the application from file structure to the actual process of executing the Universal Javascript set-up.

Folder Structure Refactor Proposal

The current server folder structure:

/server
    /controllers
        htmlTemplate.js
        writeFile.js
    /helpers
        buildRoutesCliCommand.js
        getWebpackConfig.js
        sendFetches.js
    server.js

The proposed updated folder structure

server/ 
    consts/
        ...
    controllers/
        htmlTemplate.js
        writeFile.js
    modules/
    attaching/
        index.js
    deserializing/
        index.js
    execution/
        index.js
        bundleApp.js
        generateBundleConfig.js
        generateWebpackConfig.js
    serializing/
        index.js
        generateClientData.js
    rendering/
        index.js
        writeStaticFiles.js // previous send fetches
    routing/
        index.js
        generateRoutes.js
    presets/
        index.js
        React.js
        Vue.js
    utils/
        importPlugins.js
    server.js

Breaking up the functionality in the current helper files. Additionally removing the "helpers" folder and placing all js in it into the correct "phase" folder.

Structuring inputs and outputs for phases

Each phase acts as a precursor to its corresponding phase. It takes in input, and its output is consumed by a later phase.

ribbit init > routing > serializing > executing > rendering > response -> deserializing + attaching

Phase input and output approach

1. Routing - mapping components to routes

   Input (from ribbit init)

   1. ribbit.config.js - required to locate the user's routes file. Which exists in the config's appRoot path.
   2. ribbit.routes.json - details: mapping of components to route paths, mapping for components to templates, mapping for templates to static view files, and output folder for a set of mapped routes.
   3. Plugins with the routing key

      Output

   4. ribbit.routes.json - generated with ribbit init and additional input from a user for dependencies (bundle related dependencies), plugins (functionality that needs to plugin into the ribbit application), and webpackSettings (additional Webpack functionality required to render the application). Provided via the file system.
   5. routing - {routes:[], routeAssetNameMap:[]} an object contains the mappings for routes and names for static

2. Serializing

   Input

   1. client-side data - users will use client-side function in their react app to export state from the server side. Data from from exported const {store, window} from 'ribbit' functions.
   2. Plugins with the serializing key

      Output

   3. clientData - an object that contains a store to be passed to a state management library or any additional data the user may want to add to the window

      {
          store: {
              store: //,
              stateManager: //
          },
          window: {// data to be stored on the window},
      }

3. Execution

   Input

   1. ribbit.routes.json
   2. Ribbit.config.js Plugins with the execution key
   3. Ribbit.config.js Dependencies for bundling

      Output

   4. Bundled Application - SPA bundled in bundlered and ready to receive any wrappers from plugins i.e. <App /> becomes <StaticRouter context={} location={}><App /></StaticRouter>
   5. RenderToString(Stream) javascript - rendered SPA ready to pass to HTML template string

4. Rendering

   Input

   1. RenderToString(Stream) javascript
   2. Extracted CSS
   3. Ribbit.routes.json - component to route mappings, component to template mappings, assetName updates, and component to output folder mappings.

      Output

   4. HTML Template string
   5. Static files generated

5. Response

   Input

   1. Output path for ribbit.manifest.json
   2. Ribbit.routes.json

      Output:

      1. ribbit.manifest.json -

6. Deserializing + Attaching

   Input

   1. Function for pre-loading data on the client side

   2. Client is about to hydrate with static store state when event listeners take over

The methods produced for generating the outputs above can still be passed to our current middleware set-up. Organizing the app into methods makes it easier to pinpoint what each part of our application is doing and to locate the pieces of functionality that users are most likely to want to plugin to.

2. Decoupling opinionated frameworks from ribbit and injecting them as plugins

How does one plugin into the Ribbit architecture? This question breaks into two parts. The first, what functionality is running in each phase/how do we map functionality to a phase? The second, how do we plugin into that functionality from external plugins?

An approach to predictable functionality

Using library/framework presets as a means for creating predictable funcationality. Examples of preset would be: React or Vue. These presets would be saved as arrays with indexes containing default methods for a particular preset that will be run in chronological order within each phase of the build process. An example would be that in React generating JSX is required to generate the HTML string in the rendering phase. Thus for the JSX preset, we create a jsxCompose function, which will always run in the execution phase. All preset functions are required to take an array of plugins as their first argument. Additional inputs will be dictated by the data required to complete the function.

Example of a present: modules/presets/react.js

module.exports = {
    attaching: [],
    deserializing: [],
    execution: [
        jsxComponent(plugins, App) {//..}
    ],
    rendering: [],
    routing: [],
    serializing: []
}

An approach to plugging external functionality into Ribbit

In each phase at the beginning, a function getPhasePlugins will be called. This function will look into the ribbit configs plugins property, which is an array of plugins.

ribbit.config.js

//...ommitted
preset: "react",
  plugins: ['@aribbit/react-redux', '@ribbit/react-router'],
  dependencies: [
    "babel-loader",
    "@babel/preset-env",
    "@babel/preset-react",
    "purify-css",
    "react-redux",
    "redux",
    "react"
  ],
//...ommitted

getPhasePlugins receives a single argument which is a the current phase as a string: getPhasePlugins('execution'). The function will map through the plugins array in the ribbit.config.js file. Each (plugin)[https://github.com/team-bimmer/ribbit/issues/44] exports an object with keys that match each phase. getPhasePlugins will loop through each project in the array and grab any method that is in an object with the same name as the current phase.

The output of getPhasePlugins will be an object. Each key in the object maps to a function from the phase in the preset object. Each key's value is an array of all the methods that exist in the plugins under the current phrase name.

The array will then be passed into the preset function as the first argument, and the function in the array will be composed together.

Example: If we use the configuration in ribbit.config.js above, getPhasePlugins will check in the exported objects from the packages. The packages would be in the user's node_modules folder.

/Users/someProject/node_modules/@ribbit/plugins/react-router

 import React from 'react';
 import {StaticRouter} from 'react-router-dom';
 module.exports = {
     serialize: {
         jsxCompose(App, {location, context}) {return (<StaticRouter content={content} location={location}><App /></StaticRouter>)}
     },
     routing: {},
     executing: {},
     responding: {},
     rendering: {},
     deserializing: {}
 }

/Users/someProject/node_modules/@ribbit/plugins/react-redux

import React from 'react';
import {Provider} from 'react-redux';
 module.exports = {
     serialize: {
         jsxCompose(App, {store}) {return <Provider store={store}><App /></Provider>}
     },
     routing: {},
     executing: {},
     responding: {},
     rendering: {},
     deserializing: {}
 }

During the serializing phase (like every other phase before it) the getPhasePlugins will loop through the plugins array looking for a serialize key on their exported objects. It will then output the following object:

const serializingPluginFns = getPhasePlugins('serializing');
// returns:
const serializingPluginFns = {
    jsxCompose: [
        jsxCompose, // from react-redux
        jsxCompose, // from react-router
    ]
}

When the jsxCompose function is run in serialize after the application bundling is complete the plugin methods array will be passed to is.

jsxCompose(serializingPluginFns, <CompiledApp />)

The output will be a wrapped bundled App component produced by composing functions together.

That is the approach to injecting external plugins into the ribbit build process! 😃

3. Abstract the server (creating a wrapper around Express)

As this is not MVP critical, this will be placed in a separate RFC written shortly.

[mrbut]

Would additionally like to propose we add something like webpack's resolve modules: https://webpack.js.org/configuration/resolve/#resolve-modules.

ribbit.config.js

resolve_modules: ['node_modules', 'my_local_plugins', 'my_local_presets]

^__^ This way users can map to local paths instead of needing to worry about the need to cut a release for every single tiny change (aka we can test all the things or just use our own local private thing!)

[brianwhon]

The problem-1 What do we mean by switching to another server?

routing-output-ii how is this generated?

react-router are for this plugin, are we assuming they are also using redux?

[mrbut]

Per my comments earlier, I'm requesting to revamp the plugin portion of this to match closer follow the two principles:

1. Every plugin on does a single thing
2. Smart plugins combine plugins together so users do not need to continuously rewrite the same composition for plugins

The plugins key in ribbit.config.js will control the order of composing the plugins and smart plugins will export an object that will be consumed.

Adding my board scribbles. Will write extended details:

[mrbut]

Plugin config rewrite:

ribbit.config.js has a new plugins: property. It can accept either an array or an object.

The array is a series of plugins that will be executed from right to left. The array can hold smart plugins and regular plugins.

Regular plugins export a function and they take care of a single task. The receive a single input as an argument and return a single output.

Smart plugins export an object that has preset set compositions of existing functions.

This approaches allows for users to have granular control over their plugins and also ensure that if there is a universal need for a set of functionality a smart plugin can remove the need to repeatedly write out the same plugin compositions.

Example with array and no smart plugins:

plugins: ['@ribbit/react-redux`, '@ribbit/react-router']

Plugins like dependencies will be symlinked. The genPhasePlugins function will dynamically require all the modules in the plugins and check the exported object. It will generate an object that maps the plugin (#51) phase to the function it's trying to plug into.

Example with array and smart plugins:

plugins: ['@ribbit/react-redux-router`]

Plugin is exported as an object and treated as the pointer for the ribbit.config.js file's plugins property.

Example with object

plugins: {
    execution: ['@ribbit/react-redux`, '@ribbit/react-router']
}

Plugin object can completely compose all phases plugins. User has complete control.