search

hyoo-ru / HabHub

Peering social blog
The Unlicense
0 stars 0 forks source link

[mol_learn] MAM - Mam owns language-Agnostic Modules #1

Closed PavelZubkov closed 2 years ago

PavelZubkov commented 2 years ago

MAM - Mam owns language-Agnostic Modules

What is MAM?

MAM is a modular system that uses a new way of working with code and organizing it. Agnostic means that the module is not locked into any concrete language. A module is a directory that contains files in different languages.

MAM is designed to automate the process of working with modules as much as possible and to remove the routine. Agreements are used instead of configurations. The application code is separated from the development environment code. Only used modules will be added to the bundle. A module is created in one action. The versioning system allows you to maintain only one version of a module - the latest.

Installation and setup VSCode

  1. Update NodeJS to LTS

  2. Clone the repository

    git clone https://github.com/hyoo-ru/mam.git ./mam && cd mam

  3. Install NPM dependencies

    npm install

  4. Install VSCode plugins

  5. You can also use Gitpod Gitpod Online Dev Workspace (install plugins)

Separate the application/library code from the dev environment

No need to clone the repository for each project. This is done once. MAM separates the developer environment from the application code into a separate repository. In a single environment, you can work on multiple projects in a uniform way. You can update the developer environment centrally for all projects.

Where is the MAM code located?

This repository depends on the NPM package mam. The source of this package is the $mol_build module, located here. All of the MAM logic is implemented in this module. It was one of the first to be created and has long needed refactoring. A prototype of a new version of the builder has been created, but there are no resources to finalize it. Maybe someone would like to help?

How to create a module?

  1. Think of a name and create a folder with this name.
  2. Create the implementation files for this module.

Conventionally, the modules can be divided into three types:

Namespace

This is the root folder for your modules. You can use the my namespace for your projects and experiments. $mol is used mol. If you will be using MAM in your company, you will have a namespace with the company name. That is, you just create a folder and give it the name you want.

The complete path will be as follows: mam/mol, mam/my.

There is a limitation, at this level you cannot create a file index.html. You need to create a module inside the namespace, and put it there.

Module

Just a folder with the source code

Submodule

You can create modules to an unlimited depth. In practice, there are rarely more than 3-4 levels, if you have more than that, you may have a wrong way of decomposing your code into modules or a very large project.

Creating your first module

Clone the MAM repository, install the NPM dependencies and the plugins for VSCode - instructions above. We will now create a module to be developed during this manual.

  1. Go to mam directory

    cd mam

  2. Create namespace

    mkdir my && cd my

  3. Create module directory

    mkdir counter && cd counter

I usually give my for namespace for my small projects. You can name it whatever you like. In the next step, in the counter module, we will create the files that implement it. Now see what files can make up the module.

What languages/formats can the module consist of?

How do I name the module files?

Usually the file names, repeat the module names, for example: counter.view.tree, counter.view.ts, counter.view.css - for files in module my/counter. But the builder will scan all the files in the module's directory and find the necessary entities, even if the files have different names. The main thing is to name the entities correctly in the code.

Creating Counter

We will create a Counter app and use it to learn how to use MAM. In the next chapters we will upgrade it.

Create a file mam/my/counter.index.html with the following content:

<!doctype html>
<html style="height: 100% ">

    <head>
        <meta charset="utf-8" />
        <meta name="viewport" content="width=device-width, height=device-height, initial-scale=1">
    </head>

    <body style=" width: 100%; height: 100%; margin: 0 ">
        <div id="root"></div>
        <script src="web.js"></script>
    </body>

</html>

Create a file mam/my/counter/counter.ts and add there all the code below:

class View {
    dom_name() {
        return 'div'
    }

    // object with attributes
    attr(): { [key: string]: string|number|boolean|null } {
        return {}
    }

    // object with event handlers
    event(): { [key: string]: (e: Event) => any } {
        return {}
    }

    // object with fields
    field(): { [key: string]: any } {
        return {}
    }

    // children elements
    sub(): Array<View | Node | string | number | boolean> {
        return []
    }

    _dom_node = null as unknown as Element // for caching dom node
    dom_node() {
        if ( this._dom_node ) return this._dom_node

        const node = document.createElement( this.dom_name() )
        Object.entries(this.event()).forEach( ([name , fn])=> node.addEventListener(name, fn) )

        return this._dom_node = node
    }

    // actualization attributes and fields
    dom_node_actual() {
        const node = this.dom_node()

        Object.entries(this.attr()).forEach( ([name, val])=> node.setAttribute(name, String(val)) )
        Object.entries(this.field()).forEach( ([name, val])=> node[name] = val )

        node.setAttribute('view', this.constructor.name)

        return node
    }

    // preparing for children rendering
    dom_tree() {
        const node = this.dom_node_actual()

        const node_list = this.sub().map( node => {
            if ( node === null ) return null
            return node instanceof View ? node.dom_tree() : String(node)
        } )

        this.render( node , node_list )

        return node
    }

    // children rendering with reconsilation
    render( node: Element, children: Array<Node | string | null> ) {
        const node_set = new Set< Node | string | null >( children )
        let next = node.firstChild

        for (const view of children) {
            if (view === null) continue

            if (view instanceof Node) {
                while(true) {
                    if (!next) {
                        node.append(view)
                        break
                    }
                    if (next === view) {
                        next = next.nextSibling
                        break;
                    } else {
                        if (node_set.has(next)) {
                            next.before(view)
                            break;
                        } else {
                            const nn = next.nextSibling
                            next.remove()
                            next = nn
                        }
                    }
                }
            } else {
                if( next && next.nodeName === '#text' ) {
                    const str = String( view )
                    if( next.nodeValue !== str ) next.nodeValue = str
                    next = next.nextSibling
                } else {
                    const text = document.createTextNode( String( view ) )
                    node.insertBefore( text, next )
                }
            }
        }

        while( next ) {
            const curr = next 
            next = curr.nextSibling
            curr.remove()
        }
    }
}

The View class is a wrapper for the DOM node and provides an interface to make working with DOM nodes easier.

Now let's create several components based on the View class

class Button extends View {

    dom_name() { return 'button' }

    title() { return '' }

    click( e: Event ) {}

    sub() {
        return [ this.title() ]
    }

    event() {
        return {
            click: (e: Event) => this.click(e)
        }
    }
}

class Input extends View {
    dom_name() { return 'input' }

    type() { return 'text' }

    _value = ''
    value( next = this._value ) {
        return this._value = next
    }

    event_change( e: Event ) {
        this.value( (e.target as HTMLInputElement).value )
    }

    field() {
        return {
            value: this.value(),
        }
    }

    attr() {
        return {
            type: this.type(),
        }
    }

    event() {
        return {
            input: (e: Event)=> this.event_change(e),
        }
    }
}

Based on the components, let's create an application class

class Counter extends View {
    // Synchronization with local storage, all application tabs will be synchronized.
    storage<Value>( key: string, next?: Value ) {
        if ( next === undefined ) return JSON.parse( localStorage.getItem( key ) ?? 'null' )

        if ( next === null ) localStorage.removeItem( key )
        else localStorage.setItem( key, JSON.stringify( next ) )

        return next
    }

    // This is where we will store the counter value. This method has two-way binding with localStorage
    // use `this.count()` for get value
    // use `this.count(10)` for set value
    // This is called a channel and will be discussed in the chapter on reactivity.
    count( next?: number ) {
        return this.storage( 'count' , next ) ?? 0
    }

    // Helper for converting to string and number
    count_str( next?: string ) {
        return String( this.count(next ? Number(next) : undefined) )
    }

    // Increment action
    inc() {
        this.count( this.count() + 1 )
    }

    // Decrement action
    dec() {
        this.count( this.count() - 1 )
    }

    // Create increment button, bind with increment action and cache
    _Inc = null as unknown as View
    Inc() {
        if (this._Inc) return this._Inc

        const obj = new Button
        obj.title = ()=> '+'
        obj.click = ()=> this.inc()

        return this._Inc = obj
    }

    _Dec = null as unknown as View
    Dec() {
        if (this._Dec) return this._Dec

        const obj = new Button
        obj.title = ()=> '-'
        obj.click = ()=> this.dec()

        return this._Dec = obj
    }

    // Count input binded to count store
    _Count = null as unknown as View
    Count() {
        if (this._Count) return this._Count

        const obj = new Input
        obj.value = (next?: string)=> this.count_str( next )

        return this._Count = obj
    }

    // children
    sub() {
        return [
            this.Dec(),
            this.Count(),
            this.Inc(),
        ]
    }

    static mount() {
        const node = document.querySelector( '#root' )
        const obj = new Counter()

        node?.replaceWith( obj.dom_tree() )

        // Reactivity will be in the next chapter, for now we will use the hack.
        setInterval( ()=> obj.dom_tree() , 100 )
    }
}

// Don't forget to call the `mount`
Counter.mount()

How to build a module manually?

Just run the command npm start path/to/module. It is possible to run a dev server, described here. When you run the build manually, the builder collects all the files, when the dev server is running, only the few necessary files.

Where do I look for the bundle?

Build artifacts are created in the - folder, which is located in the folder of the module to be built. When creating a repository in .gitignore it is only necessary to add a single line -*.

You can also see three more folders -css, -view.tree, -node - these are intermediate results, they are also created for all modules as needed, which depends on the module being built. In the git settings, you have to ignore everything whose name starts with a minus sign.

What files does the bundle consist of?

You will see the following files:

- web.js - javascript bundle for the browser, css compiles into js bundle
- web.js.map - source maps for js bundle
- web.esm.js - as an esm module
- web.esm.js.map - source maps for esm
- web.test.js - bundle with tests
- web.test.js.map - source maps for test
- web.view.tree - the declarations of all view.tree components included in the bundle
- web.locale=*.json - bundles with localized texts, each language has its own bundle
- web.deps.json - information about the graph of module dependencies
- web.audit.js - information about static typing errors
- web.d.ts - a bundle with types of everything in the js bundle
- node.js - all of the above, but for nodejs
- node.js.map
- node.esm.js
- node.esm.js.map
- node.test.js
- node.test.js.map
- node.view.tree
- node.locale=*.json
- node.deps.json
- node.audit.js
- node.d.ts
- index.html - entry point for launching the module in the browser
- test.html - entry point for running tests in the browser
- README.md - module documentation
- package.json - allows you to publish the assembled module in the NPM

Build Counter application

In MAM you can build any module without prior preparation. Let's build our application.

cd mam
npm start my/counter

After launching, the builder will return an error ReferenceError: document is not defined for the line const node = document.querySelector('#root'). Note the file name node.test.js.

The builder creates all types of bundles at once. The node.test.js bundle runs automatically after the build. The bundle contains all of our code, tests of our code and tests of all the modules our code depends on. It will start even if there are no tests yet at all.

In the application code, there is a run of the `Counter.mount' method. It runs under NodeJS, so it throws an error. Now we will install a hack in this place and fix it later.

Add a line to the beginning of the Counter.mount method.

static mount() {
    if ( typeof document === 'undefined' ) return // +

    const node = document.querySelector( '#root' )
    const obj = new Counter()

    node?.replaceWith( obj.dom_tree() )

    setInterval( ()=> obj.dom_tree() , 100 )
}

Run the build again. When finished, look in the mam/my/counter/-/ directory.

You will find index.html copied without changes. File test.html with additional code:

<script src="/mol/build/client/client.js" charset="utf-8"></script>
<script>
    addEventListener( 'load', ()=> {

        const test = document.createElement( 'script' )
        test.src = 'web.test.js'

        const audit =  document.createElement( 'script' )
        audit.src = 'web.audit.js'

        test.onload = ()=> document.head.appendChild( audit )
        document.head.appendChild( test )

    } )
</script>

This additionally loads the file client.js, which establishes a connection to the server via web sockets and reloads the page at the command of the dev server.

And js code that adds the download of the two files via the script tag. The file web.test.js contains code for tests that will run in the browser after each file change.The file web.audit.js contains one line console.info("Audit passed"), in case Typescript finds errors in our code, instead of the line Audit passed the text of errors will be displayed in the browser console.

The readme.md file contains the content of the root file mam/readme.md. If the module does not contain its own readme.md file, it is searched in the parent module, and so on to the root.

In the web.js file you will find our code.

A complete list of files and a short description is given above.

How to run the dev server?

You can run the dev server, and it will automatically rebuild the project when files change, as well as reload the tab in the browser.

cd mam
npm start

A link http://127.0.0.1:9080 will appear in the terminal. Open it, you will see the contents of the mam directory in the file manager. Open the developer tools and on the network tab disable caching. In the file manager, open the project module. At the moment, only modules containing the index.html file are supported.

By opening the folder with the index.html file, the url will contain http://127.0.0.1:9081/my/counter/-/test.html. The path to the module /my/counter, the folder where the build artifacts are put /-/ and the bundle type test.html.

When the browser loads the html file, it will load the js file referenced in the script tag: <script src="web.js"></script>. The browser will make a request at the following url http://127.0.0.1:9081/my/counter/-/web.js.

The dev server analyzes the path to the module and the bundle type, build only this bundle type and returns it in the response. If you start the build manually, the builder will build all types of bundles at once.

Right now the dev server only supports frontend projects. If you create a backend project, you can start the dev server and manually send requests for it to build the required type of bundle - node.js or node.test.js.

Next, use the dev server in practice.

How to import/export modules?

Just write the name of the entity in the code and nothing more. But the name of the entity must be special. It starts with $ and contains the path to the module.

$mol_data_string - means that the file is in mam/mol/data/string. The name of the entity contains the path to the module in the file system.

Such names are called `Fully qualified names - FQN'. You should use FQN names instead of imports and exports.

// mam/my/csv/cvs.ts
function $my_csv_decode( text = 'a;b;c\n1;2;3' ) {
    return $mol_csv_parse( text )
}

function $my_csv_encode( list = [['a','b','c'], [1,2,3]] ) {
    return list.map(
        line => line.map( cell => `"${cell.replace( /"/g, '""' )}"` ).join(';')
    ).join('\n')
}

Note that we do not have to put decode and encode into separate submodules - mam/my/csv/decode and mam/my/csv/encode. You can declare them in mam/my/csv, as long as the prefix is the same. For example, the name $my_json cannot be used in mam/my/csv, but you can use $my_csv_to_json because the prefix matches. The MAM builder will find the necessary declarations in the code itself.

$using_fully_qualified_names

Initially we just put all the entities in one file, so now let's put them in order and put them in their places.

The new module structure will look like this:

$my_counter - mam/my/counter
$my_counter_view - mam/my/counter/view
$my_counter_button - mam/my/counter/button
$my_counter_input - mam/my/counter/input

Create mam/my/counter/view/view.ts and move the contents of the View class into it. And replace the class name with $my_counter_view.

class $my_counter_view {
    dom_name() {
        return 'div'
    }

    attr(): { [key: string]: string|number|boolean|null } {
        return {}
    }

    event(): { [key: string]: (e: Event) => any } {
        return {}
    }

    field(): { [key: string]: any } {
        return {}
    }

    sub(): Array<$my_counter_view | Node | string | number | boolean> {
        return []
    }

    _dom_node = null as unknown as Element
    dom_node() {
        if ( this._dom_node ) return this._dom_node

        const node = document.createElement( this.dom_name() )
        Object.entries(this.event()).forEach( ([name , fn])=> node.addEventListener(name, fn) )

        return this._dom_node = node
    }

    dom_node_actual() {
        const node = this.dom_node()

        Object.entries(this.attr()).forEach( ([name, val])=> node.setAttribute(name, String(val)) )
        Object.entries(this.field()).forEach( ([name, val])=> node[name] = val )

        node.setAttribute('view', this.constructor.name)

        return node
    }

    dom_tree() {
        const node = this.dom_node_actual()

        const node_list = this.sub().map( node => {
            if ( node === null ) return null
            return node instanceof $my_counter_view ? node.dom_tree() : String(node)
        } )

        this.render( node , node_list )

        return node
    }

    render( node: Element, children: Array<Node | string | null> ) {
        const node_set = new Set< Node | string | null >( children )
        let next = node.firstChild

        for (const view of children) {
            if (view === null) continue

            if (view instanceof Node) {

                while(true) {
                    if (!next) {
                        node.append(view)
                        break
                    }
                    if (next === view) {
                        next = next.nextSibling
                        break;
                    } else {
                        if (node_set.has(next)) {
                            next.before(view)
                            break;
                        } else {
                            const nn = next.nextSibling
                            next.remove()
                            next = nn
                        }
                    }
                }

            } else {

                if( next && next.nodeName === '#text' ) {
                    const str = String( view )
                    if( next.nodeValue !== str ) next.nodeValue = str
                    next = next.nextSibling
                } else {
                    const text = document.createTextNode( String( view ) )
                    node.insertBefore( text, next )
                }

            }
        }

        while( next ) {
            const curr = next 
            next = curr.nextSibling
            curr.remove()
        }
    }
}

Do the same with the Button class:

class $my_counter_button extends $my_counter_view {
    dom_name() { return 'button' }

    title() { return '' }

    click( e: Event ) {}

    sub() {
        return [ this.title() ]
    }

    event() {
        return {
            click: (e: Event) => this.click(e)
        }
    }
}

And with the Input class:

class $my_counter_input extends $my_counter_view {
    dom_name() { return 'input' }

    type() { return 'text' }

    _value = ''
    value( next = this._value ) {
        return this._value = next
    }

    event_change( e: Event ) {
        this.value( (e.target as HTMLInputElement).value )
    }

    field() {
        return {
            value: this.value(),
        }
    }

    attr() {
        return {
            type: this.type(),
        }
    }

    event() {
        return {
            input: (e: Event)=> this.event_change(e),
        }
    }
}

In the file mam/my/counter/counter.ts we have only the class Counter. Now change its name to FQN, and the references to the other classes in it.

class $my_counter extends $my_counter_view {
    storage<Value>( key: string, next?: Value ) {
        if ( next === undefined ) return JSON.parse( localStorage.getItem( key ) ?? 'null' )

        if ( next === null ) localStorage.removeItem( key )
        else localStorage.setItem( key, JSON.stringify( next ) )

        return next
    }

    count( next?: number ) {
        return this.storage( 'count' , next ) ?? 0
    }

    count_str( next?: string ) {
        return String( this.count(next ? Number(next) : undefined) )
    }

    inc() {
        this.count( this.count() + 1 )
    }

    dec() {
        this.count( this.count() - 1 )
    }

    _Inc = null as unknown as $my_counter_view
    Inc() {
        if (this._Inc) return this._Inc

        const obj = new $my_counter_button
        obj.title = ()=> '+'
        obj.click = ()=> this.inc()

        return this._Inc = obj
    }

    _Dec = null as unknown as $my_counter_view
    Dec() {
        if (this._Dec) return this._Dec

        const obj = new $my_counter_button
        obj.title = ()=> '-'
        obj.click = ()=> this.dec()

        return this._Dec = obj
    }

    _Count = null as unknown as $my_counter_view
    Count() {
        if (this._Count) return this._Count

        const obj = new $my_counter_input
        obj.value = (next?: string)=> this.count_str( next )

        return this._Count = obj
    }

    sub() {
        return [
            this.Dec(),
            this.Count(),
            this.Inc(),
        ]
    }

    static mount() {
        if ( typeof document === 'undefined' ) return

        const node = document.querySelector( '#root' )
        const obj = new $my_counter()

        node?.replaceWith( obj.dom_tree() )

        setInterval( ()=> obj.dom_tree() , 100 )
    }
}

$my_counter.mount()

Just like in the previous practice section, in the web.js file you will find our code.

What if a module is used many times and its name is too long

Just create an alias for it.

const Response = $mol_data_record({
    status: $mol_data_number,
    data: $mol_data_record({
        name: $mol_data_string,
        surname: $mol_data_string,
        age: $mol_data_number,
        birth_date: $mol_data_pipe( $mol_data_string, $mol_time_moment ),
    }),
})

Creating aliases:

const Rec = $mol_data_record
const Str = $mol_data_string
const Num = $mol_data_number

const Response = Rec({
    status: Num,
    data: Rec({
        name: Str,
        surname: Str,
        age: Num,
        birth_data: $mol_data_pipe( Str, $mol_time_moment ),
    }),
})

Which modules will be included in the bundle?

Only used modules will be included in the bundle. The builder creates a list of dependencies of the module to be built, so recursively for each dependency. The final bundle contains only the modules you actually use. All files and all entities in module files are included in the bundle. Submodules, if not used, are not included.

Let's create a module to work with localStorage. Create a file mam/my/counter/storage/storage.ts with the following contents:

class $my_counter_storage {

    static value<Value>( key: string, next?: Value ) {
        if ( next === undefined ) return JSON.parse( localStorage.getItem( key ) ?? 'null' )

        if ( next === null ) localStorage.removeItem( key )
        else localStorage.setItem( key, JSON.stringify( next ) )

        return next
    }

}

At this point it does not need to be used in $my_counter.

And in the file mam/my/counter/button/button.ts let's add one more class for the minor button. We'll use it later, but for now we'll just add a class. The contents of the file will be as follows:

class $my_counter_button extends $my_counter_view {

    dom_name() { return 'button' }

    title() { return '' }

    click( e: Event ) {}

    sub() {
        return [ this.title() ]
    }

    event() {
        return {
            click: (e: Event) => this.click(e)
        }
    }

}

class $my_counter_button_minor extends $my_counter_button {

    attr() {
        return {
            '$my_counter_button_minor': true,
        }
    }

}

Build the $my_counter module manually, If you haven't already started the dev server.

npm start my/counter

Check the web.js' bundle. The$my_counter_button_minoris included in the bundle, because this class is declared in the same file as$my_counter_buttonwhich is used in the project.To prevent$my_counter_button_minorfrom being included in the bundle, you need to create a directoryminor` in the module folder, and put the source code of the class there.

Make sure that the class $my_counter_storage are not added to the web.js bundle.

Now let's add the use of $my_counter_storage to the code.

// mam/my/counter/counter.ts
class $my_counter extends $my_counter_view {

// delete
// -    storage<Value>( key: string, next?: Value ) {
// -        if ( next === undefined ) return JSON.parse( localStorage.getItem( key ) ?? 'null' )
// -
// -        if ( next === null ) localStorage.removeItem( key )
// -        else localStorage.setItem( key, JSON.stringify( next ) )
// -
// -        return next
// -    }

    count( next?: number ) {
// -        return this.storage( 'count' , next ) ?? 0
        return $my_counter_storage.value( 'count' , next ) ?? 0 // +
    }

After building you will find $my_counter_storage in the bundle.

Cross-language dependencies

All module files are included in the bundle. The modular system is separate from languages, dependencies can be cross-language. The modular system is separate from languages, dependencies can be cross-language.

For example, we have a module with styles for tables. We want them to have one color in a light theme, and another in a dark theme:

/my/table/table.css

/* Dependent on `/my/theme */
[my_theme="dark"] table {
    background: black;
}
[my_theme="light"] table {
    background: white;
}

Note that in the code above my_theme is FQN, in css you cannot use the $ sign, so an exception applies here and it is not used

/my/theme/theme.js

document.documentElement.setAttribute(
    'my_theme' ,
    ( new Date().getHours() + 15 ) % 24 < 12 ? 'light' : 'dark' ,
)

The $my_table depends on $my_theme. $my_theme will include in the bundle, and will change color depending on the time of day.

Let's add the example above to our application. Create a module $my_theme with the file theme.ts and add the following code to it:

// mam/my/theme/theme.ts
setInterval( ()=> {
    document?.documentElement.setAttribute(
        'my_theme' ,
        new Date().getSeconds() < 30 ? 'light' : 'dark' ,
    )
} , 1_000 )

The code has been changed a little bit. setInterval is needed to see the changes without reloading the page. And the attribute will change every thirty seconds.

Create a css file for the $my_counter module.

/* mam/my/counter/counter.css */
[my_theme="light"] {
    background-color: white;
}

[my_theme="dark"] {
    background-color: black;
}

You will now see on the page that the background color changes every thirty seconds. The module $my_theme changes the attribute for the element html - document.documentElement every thirty seconds and the appropriate style applies.

The builder found the module name $my_theme in the counter.css file and added the module $my_theme to the bundle. Find its code in web.js.

After adding the css file, several new modules from the $mol namespace were added to the bundle. Styles are now added to the js bundle using the function $mol_style_attach, find it in web.js. New modules in the bundle are its dependencies.

How do I change the location of a module?

During refactoring, you may need to move the module to another location. Since the names are location-dependent, they need to be changed as well. In all module files FQN-names are written the same way, except *.css - here there is no $ sign in the beginning. You need to run the find and replace command. Example: find my_module_name and replace to my_module_new_name after moved module.

Let's move common modules like $my_counter_view, $my_counter_button from the Counter application to the $my_counter module. And let's leave only the application code in the application.

Create a directory for the $my_lom module

cd my
mkdir lom

Move folder $my_counter_view to $my_lom_view and use find and replace for string "my_counter_view" to "my_lob_view" for mam directory.

Repeat the previous step for $my_counter_button, $my_counter_input, $my_counter_storage, $my_theme.

FQN names everywhere are written in the same style, which allows in one operation of search to find all places where a module is used, in one operation of replacement to change the names when moving a module.

Now all of the library code is in the $my_lom module, and the application code is in the $my_counter module. Make sure that after refactoring the application works!

Monorepo and polyrepo

MAM supports the use of both monorepo and polyrepo at the same time. You can create as many modules as you want in one repository. Or you can tell MAM where to look for the repository, and then when you build the module, it will clone the repositories you want. MAM automatically clones missing repositories when the builder starts.

In order to the builder to clone repository, it needs to know where to clone it from. You need to create a repository for the namespace and create there a file *.meta.tree - example

pack apps git \https://github.com/hyoo-ru/apps.hyoo.ru.git - pack is an instruction indicating that it is a remote repository. The apps is the name of the module. git - version control system (only git is supported now). After \ a link to the repository.

For example, if after installing MAM, you run yarn start hyoo/draw. The builder will look in the root .meta.tree and find the line pack hyoo git \https://github.com/hyoo-ru/mam_hyoo.git there without finding the hyoo folder. It will clone the repository and if it doesn't find the draw module, it will look in the /hyoo/hyoo.meta.tree and find a link to draw there.

For VSCode to work correctly with multiple repositories, create a file mam/.gitmodules with similar content (specify paths for your repositories):

[submodule "mam_mol"]
    path = mol
    url = git@github.com:hyoo-ru/mam_mol.git

[submodule "hyoo"]
    path = hyoo
    url = git@github.com:hyoo-ru/hyoo.git

Tip: VSCode does not always immediately show the repository added to .gitmodules on the Source Control tab, to fix this try restart VSCode. Also try changing the order of the modules in .gitmodules.

Let's now put the $my_lom and $my_counter modules into separate git repositories.

  1. Create the mam_my repository on github(or another system).
  2. Create a git repository in the $my module and link with github repository 
    cd mam/my
echo "# mam_my - namespace for MAM-based projects" >> readme.md
git init
git add readme.md
git commit -m "Init"
git branch -M main
git remote add origin https://github.com/**YOUR_NAME**/mam_my.git
git push -u origin main
  3. Do steps 1 and 2 for the modules $my_lom and $my_counter. 
    
cd mam/my/lom
echo "# mam_lom" >> readme.md
echo "-*" >> .gitignore
git init
git add --all
git commit -m "Init"
git branch -M main
git remote add origin https://github.com/**YOUR_NAME**/my_lom.git
git push -u origin main

cd mam/my/counter echo "# mam_counter" >> readme.md echo "-*" >> .gitignore git init git add --all git commit -m "Init" git branch -M main git remote add origin https://github.com/**YOUR_NAME**/my_counter.git git push -u origin main

4. Add `mam/my/my.meta.tree` with following content:
```tree
pack lom git \https://github.com/**YOUR_NAME**/my_lom.git
pack counter git \https://github.com/**YOUR_NAME**/my_counter.git
  1. Push my.meta.tree 
    cd mam/my
git add my.meta.tree
git commit -m "Add meta.tree"
git push

Now delete the module directories $my_lom and $my_counter. And run the builder npm start my/counter, the builder will download the necessary repositories and build the project. You will see a similar log in the terminal:

come
    time \2022-03-25T19:42:27.723Z
    place \$mol_exec
    dir \my/lom
    message \Run
    command \git init

come
    time \2022-03-25T19:42:27.738Z
    place \$mol_exec
    dir \my/lom
    message \Run
    command \git remote show https://github.com/**YOUR_NAME**/my_lom.git

come
    time \2022-03-25T19:42:28.475Z
    place \$mol_exec
    dir \my/lom
    message \Run
    command \git remote add --track main origin https://github.com/**YOUR_NAME**/my_lom.git

come
    time \2022-03-25T19:42:28.488Z
    place \$mol_exec
    dir \my/lom
    message \Run
    command \git pull

What versioning model does MAM use?

It is called verless or versionless. It works on the open-close principle.

What it gives:

There is currently no means to auto-update modules when multiple repositories are in use. Therefore, it is now necessary to manually perform a `git pull'.

In case the update breaks something, the standard tricks work: fixing the revision, etc.

Splitting code by platform

Two platforms are supported now, the browser and nodejs. The code in the *.web.ts files will only be included in the web bundle. The code in the *.node.ts files will be included only in the node-bandle. Code without platform tag *.ts, will be included in both bundles.

In the previous practice we added a hack to make the project run on NodeJS: if (typeof document === undefined) return. Let's fix that.

First, let's update the $my_lom_view module by adding two static properties to the class.

// mam/my/lom/view/view.ts
static root: ()=> typeof $my_lom_view
static mount() {
    const node = document.querySelector( '#root' )
    if ( !node ) return

    const View = this.root()
    const obj = new View

    node.replaceWith( obj.dom_tree() )
    setInterval( ()=> obj.dom_tree() , 100 )
}

We moved mount to its place and root is needed to know which component is root. Remove the mount method and its call from $my_counter and add a value setting for $my_lom_view.root there.

// mam/my/counter/counter.ts

// ...
// -    static mount() {
// -        if ( typeof document === 'undefined' ) return
// -
// -        const node = document.querySelector( '#root' )
// -        const obj = new $my_counter()
// -
// -        node?.replaceWith( obj.dom_tree() )
// -
// -        setInterval( ()=> obj.dom_tree() , 100 )
// -    }
}

// - $my_counter.mount()
$my_lom_view.root = ()=> $my_counter // +

Create a file view.web.ts in the module $mol_lom_view with this content:

// mam/my/lom/view/view.web.ts

if ( typeof document !== 'undefined' ) {
    setTimeout( ()=> $my_lom_view.mount() )
}

This code will only be added to the web.js bundle. The $my_lom_view.mount method call is asynchronous, because in the bundle $my_lom_view is above $my_counter and setting the value for $my_lom_view.root is done after running this code. Make sure this code is in the web.js' bundle and not in thenode.js' bundle.

How to use NPM packages in the node?

On the backend, there is no need to add NPM packages to the bundle. To use NPM packages, there is a special $node module. You simply write $node['is-odd'] or $node.ramda in your code. The MAM builder will automatically install the NPM packages used and add them to the package.json file in the bundle.

Let's add some isomorphism to Counter and refactor the DOM code as well. Create a directory for the module $my_lom_dom_render and in it render.ts file with following content:

// mam/my/lom/dom/render/render.ts
function $my_lom_dom_render(
    node : Element | DocumentFragment ,
    children: NodeList | Array< Node | string | null >
) {
    const node_set = new Set< Node | string | null >( children )
    let next = node.firstChild

    for (const view of children) {
        if (view === null) continue

        if (view instanceof Node) {

            while(true) {
                if (!next) {
                    node.append(view)
                    break
                }
                if (next === view) {
                    next = next.nextSibling
                    break;
                } else {
                    if (node_set.has(next)) {
                        next.before(view)
                        break;
                    } else {
                        const nn = next.nextSibling
                        next.remove()
                        next = nn
                    }
                }
            }

        } else {

            if( next && next.nodeName === '#text' ) {
                const str = String( view )
                if( next.nodeValue !== str ) next.nodeValue = str
                next = next.nextSibling
            } else {
                const text = document.createTextNode( String( view ) )
                node.insertBefore( text, next )
            }

        }
    }

    while( next ) {
        const curr = next 
        next = curr.nextSibling
        curr.remove()
    }
}

Remove the $my_lom_view.render method. In the $my_lom_view.dom_tree method, use the $my_lom_dom_render function instead.

// mam/my/lom/view/view.ts
// ...
    dom_tree() {
        const node = this.dom_node_actual()

        const node_list = this.sub().map( node => {
            if ( node === null ) return null
            return node instanceof $my_lom_view ? node.dom_tree() : String(node)
        } )

// -    this.render(node, node_list)
        $my_lom_dom_render( node, node_list ) // +

        return node
    }
// ...

The $my_lom_view class is now a decent size, and $my_lom_dom_render can be reused.

Create a directory for the $my_lom_dom_context module and a context.ts file with this content:

// mam/my/lom/dom/context/context.ts
let $my_lom_dom_context: typeof globalThis

In the general code for web and node we simply declare the variable and its type. Create a file context.web.ts:

// mam/my/lom/dom/context/context.ts
$my_lom_dom_context = self

In the code for web, this variable will be assigned a window object. Create a file context.node.ts:

// mam/my/lom/context/context.node.ts
$my_lom_dom_context = new $node.jsdom.JSDOM( '' , { url : 'https://localhost/' } ).window as any

any is necessary because jsdom does not implement the full browser api, and its type does not converge to typeof globalThis

In the code for node, this variable is assigned to the instance of the class JSDOM so that under node our application can run. This can be used in tests.

Now wherever the document object is used, you must use $my_lom_dom_context.document.

// $my_lom_view.mount
const node = $my_lom_dom_context.document.querySelector( '#root' )

// $my_lom_view.dom_node
const node = $my_lom_dom_context.document.createElement( this.dom_name() )

// $my_lom_dom_render:40
const text = document.createTextNode( String( view ) )

// $my_lom_theme
$my_lom_dom_context.document.documentElement.setAttribute( /*...*/ )

Run the $my_counter module build - npm start my/counter. If the mam/node_modules folder does not already have jsdom in it, you will see in the terminal the installation log of the npm package jsdom. The builder will also install typescript types, if such a package exists.

> start
> node ./mol/build/-/node.js "my/counter"

come
    time \2022-03-25T19:21:46.435Z
    place \$mol_exec
    dir \
    message \Run
    command \npm install jsdom

come
    time \2022-03-25T19:21:50.155Z
    place \$mol_exec
    dir \
    message \Run
    command \npm install @types/jsdom

The mam/my/counter/-/package.json file contains dependencies:

{
    "jsdom": "*",
    "colorette": "*"
}

The module $node depends on another module which uses the NPM-package colorlette, so this package is here.

The * is used because MAM verless' extends the NPM as well. As a last resort, if the NPM package breaks after an upgrade, you can fix the version inpackage.json`.

The file mam/my/counter/-node/deps.d.ts contains a general list of used modules - NPM and modules with which NodeJS is delivered. It is used for typescript typing.

interface $node {
    "jsdom" : typeof import( "jsdom" )
    "colorette" : typeof import( "colorette" )
    "path" : typeof import( "path" )
    "child_process" : typeof import( "child_process" )
}

How to add any files to the bundle?

Create a file module_name.meta.tree, add the line deploy \path/to/file/image.png there. The file will be created in the bundle folder under the same path, i.e. my/module/-/path/to/file/image.png. Example.

Let's add a favicon to the Counter app. Create a directory for the $my_counter_logo module and save the icon there.

In the $my_counter module, create a file counter.meta.tree with this content:

deploy \/my/counter/logo/logo.svg

Build the application module - npm start my/counter. After completion, the file with the icon mam/my/counter/-/my/counter/logo/logo.svg appeared.

Add the line <link href="my/counter/logo/logo.svg" rel="icon"> in the mam/my/counter/index.html file, inside the head tag. After rebuilding, the icon will be displayed.

How to use NPM packages in the frontend?

For the frontend, you need to add the NPM package directly to the bundle.

Install the necessary NPM package and types for it. Create a module for it, in it import this package via require - $lib_react = require('react') as typeof import('react'). Example.

There is a special repository for bindings to NPM packages. It will be better for everyone if you create a pull-request to it at once.

At the moment, the MAM builder itself handles NPM packages and includes their bundle, that can be a problem sometimes. In the future this will be handled by a builder created for NPM packages, such as webpack, etc.

Now you have to install NPM packages manually and there is no auto-update for them.

Now we will not use the Counter application, but add ReactJS to the $lib module and write a demo application. You also add another NPM package to $lib and open a pull-request yourself, as a practice.

First you need to clone the $lib repository, just run its npm start lib build. The link to it is already in mam/.meta.tree, so the builder knows where to find it.

Create a directory and ts file for the module $lib_react.

// mam/lib/react/react.ts

/// For install deps run manually: 
///     npm install react @types/react react-dom @types/react-dom
namespace $ {
    const React = require('react') as typeof import('react')

    export const $lib_react_all = React
    export const $lib_react = React.createElement

    export const $lib_react_use_state = React.useState
    export const $lib_react_use_callback = React.useCallback
    export const $lib_react_use_effect = React.useEffect
    export const $lib_react_use_effect_layout = React.useLayoutEffect
    export const $lib_react_use_ref = React.useRef
    export const $lib_react_use_memo = React.useMemo
    export const $lib_react_use_reducer = React.useReducer

    export const $lib_react_use_context = React.useContext
    export const $lib_react_context = React.createContext

    export const $lib_react_element_create = React.createElement
    export const $lib_react_element_clone = React.cloneElement
    export const $lib_react_element_check = React.isValidElement
}

The namespace are parts of the inversion control system, which we will talk about later.

We simply import react and its types. Put the original object in the $lib_react_all variable. In $lib_react we put the function to create the element to use that name in tsx:

/** @jsx $lib_react */

Part of the entities from React, we pull out into separate variables for faster use.

Now create a submodule for react-dom - $lib_react_dom. It's the same here:

// mam/lib/react/dom/dom.ts
namespace $ {

    const ReactDOM = require('react-dom') as typeof import('react-dom')

    export const $lib_react_dom = ReactDOM

}

Now create a small demo application - $lib_react_demo.

// mam/lib/react/demo/demo.tsx

/** @jsx $lib_react */
namespace $ {

    export function $lib_react_demo() {

        const [count, count_set] = $lib_react_use_state( 0 )

        return <div>
            <button onClick={()=> count_set(count - 1)}>-</button>
            <input
                type="text"
                value={count}
                onChange={e => count_set( Number( (e.target as HTMLInputElement).value ) )}
            />
            <button onClick={()=> count_set(count + 1)}>+</button>
        </div>

    }

}

Let's try running a demo application. Start dev-server and open lib/react/demo, don't forget to disable caching. We will see an error:

// Uncaught ReferenceError: process is not defined
//    at Object.<anonymous> (react.development.js:12:1)
//    at -:2:1

if (process.env.NODE_ENV !== "production") {/*...*/}

The builder does not know how to cut such parts of the code. You need to create a stub to do this. Create the module $lib_react_env.

// mam/lib/react/env/env.ts

namespace $ {

    export function $lib_react_env() {
        ;($mol_dom_context as any).process = { env: { NODE_ENV: 'development' } }
    }

    $lib_react_env()

}

We need this code to be in the bundle before the react and to be run before the react. Let's place a link to this function before react imports it.

// mam/lib/react/react.ts

namespace $ {
    $lib_react_env // +

    const React = require('react') as typeof import('react')

    // ...
}

Check the code order in web.js. There will be a function $lib_react_env and run it. Then the react code and after it the code from that file.

Refresh the page, the demo app now works. The adapter to the react still needs modifications, but a start has been made.

How to set up a Deploy to github pages?

You need to create a file .github/workflows/deploy.yml in the module to be deployed. Below is a sample config.

name: Deploy
on:
  workflow_dispatch:
  push:
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: hyoo-ru/mam_build@master2
      with:
        token: ${{ secrets.GH_PAT }}
        package: 'my/wiki'
    - uses: alex-page/blazing-fast-gh-pages-deploy@v1.1.0
      if: github.ref == 'refs/heads/master'
      with:
        repo-token: ${{ secrets.GH_PAT }}
        site-directory: 'my/wiki/-'

After sending a commit with this file to the github repository, the deplayer will start immediately. The github action source code is here.

Now let's go back to our Counter app and set it up with autodeploy on github. First, create a personal access token here. Create a secret in the my_counter repository, name it GH_PAT and put the previously created personal access token into it.

Create deploy.yml with following content. Remember to replace YOUR_NAME with your value.

# mam/my/counter/.github/workflows/deploy.yml
name: Deploy

on:
  workflow_dispatch:
  push:

jobs:
  build:

    runs-on: ubuntu-latest

    steps:

    - name: Build app
      uses: hyoo-ru/mam_build@master2
      with:
        token: ${{ secrets.GH_PAT }}
        package: 'my/counter'
        meta: |
          my https://github.com/**YOUR_NAME**/mam_my

    - name: Deploy on GitHub Pages
      if: github.ref == 'refs/heads/main'
      uses: alex-page/blazing-fast-gh-pages-deploy@v1.1.0
      with:
        repo-token: ${{ secrets.GH_PAT }}
        site-directory: 'my/counter/-'

In the meta parameter, a reference to the my namespace repository must be specified.So the script can get the my. meta.tree file and find the $my_lom module repository.

After pushing, the build will begin and once it's finished, you can open the application. You will find the address of the page on the github pages at the link https://github.com/**YOUR_NAME**/my_counter/settings/pages.

How do I include an independent module in the bundle?

The builder automatically adds to the bundle only modules that depend on the module being built. Sometimes you need to add modules on which your code does not directly depend. For example, when you create an application with a catalog of components.

Let's build the $my_lom module and see what goes into the web.js bundle.

npm start my/lom

Open the file mam/my/lom/-/web.js and you will see that there are no modules $my_lom_view, $my_lom_storage and others. But there is something there. Compare the contents of the mam/mam.ts and mam/mam.jam.js files with the contents of the web.js bundle.

Create the file mam/my/lom/lom.ts with the contents from the listing below and run the build again.

console.log('Hello web.js!')

Check web.js again and you will find the code you just added there.

There are a few simple rules:

  1. The module is included in the bundle if there is a link to it. A reference to a module implies the use of one of the FQN names.
  2. The whole module is included in the bundle. All of its files and all of the code in its files will be included in the bundle.
  3. Modules with no dependencies are not included in the bundle.
  4. Modules which are mentioned in the build command will be included in the bundle. For example npm start my/lom/storage - module code $my will be included in the bundle, module code $my_lom will be included in the bundle, module code $my_lom_storage will be included in the bundle. By code we mean its implementation files, the directories of its submodules do not belong to them. And if it has no implementation files, there is nothing to include in the bundle.

Sometimes you need to include a module in the bundle, even if it is not referenced in the module to be built.

Delete the file mam/my/lom/lom.ts. Create a directory for the module $my_lom_lib and a file inside it lib.meta.tree.

include \/my/lom/button
include \/my/lom/dom
include \/my/lom/input
include \/my/lom/lib
include \/my/lom/storage
include \/my/lom/theme
include \/my/lom/view

Build the $my_lom_lib module, run the npm start my/lom/lib command and check the mam/my/lom/lib/-/web.js file. All included modules are added to the bundle.

How to configure module auto-publish in npm?

Let's now publish the $my_lom_lib module on NPM. First, create an account or log in.

npm adduser
# or
npm login

Build the npm start my/lom/lib module, and check the name field in the mam/my/lom/lib/-/package.json file. Everyone who does this manual will have the same package name. Let's add pacakge.json with a different name.

// mam/my/lom/lib/package.json
{
    name: "my_lom_lib_test"
}

The fields from the manually added package.json will be merged with the fields of the generated package.json

Build the module again and check the package.json file in the bundle - the name has changed. Now check that this name is free in the NPM.

npm search my_lom_lib_test

If it's busy, change it to your own and go on.

Now we need a personal access token for NPM.

npm token create

Create a secret in the my_lom repository named NPM_AUTH_TOKEN.

Create a file mam/my/lom/.github/workflows/my_lom_lib.yml with this content:

name: my_lom_lib_0

on:
  workflow_dispatch:
  push:
    branches: 
      - main

jobs:
  build:

    runs-on: ubuntu-latest

    steps:

    - name: Environment Printer
      uses: managedkaos/print-env@v1.0

    - name: Build apps
      uses: hyoo-ru/mam_build@master2
      with:
        package: my/lom
        modules: lib

    - uses: PavelZubkov/npm-publish@v1
      with:
        token: ${{ secrets.NPM_AUTH_TOKEN }}
        package: ./my/lom/lib/-/package.json

Send the changes to the remote repository, wait for the build, and find the new package on NPM.

Cyclical dependencies

The builder estimates the hardness of the dependencies. The higher the hardness, the earlier the file will be included in the bundle. The hardness is now estimated by the number of indents in the line in which the dependency is located.

Now we will create some js outside of MAM and reproduce the cyclic dependency.

Create somewhere a directory cyclic and in it a file all.mjs with this content:

// cyclic/all.mjs
export class Foo {
    get bar() {
        return new Bar();
    }
}

export class Bar extends Foo {}

console.log(new Foo().bar);

Run this code node app.mjs. There is a cyclic dependency here, but the code works correctly. Now split this code into three files.

// cyclic/foo.mjs
import { Bar } from './bar.mjs';

export class Foo {
    get bar() {
        return new Bar();
    }
}
// cyclic/bar.mjs
import { Foo } from './foo.mjs';

export class Bar extends Foo {}
// cyclic/app.mjs
import { Foo } from './foo.mjs';

console.log(new Foo().bar);

To fix this, we need to add an import for bar.mjs before foo.mjs.

// cyclic/app_fix.mjs
import './bar.mjs';
import { Foo } from './foo.mjs';

console.log(new Foo().bar);

Now let's move this code to MAM. Create three modules $my_foo, $my_bar, $my_app.

// mam/my/foo/foo.ts
class $my_foo {
    get bar() {
        return new $my_bar();
    }
}
// mam/my/bar/bar.ts
class $my_bar extends $my_foo {}
// mam/my/app/app.ts
console.log(new $my_foo().bar);

Build the $my_app module and look in the js bundle.

"use strict";
class $my_foo {
    get bar() {
        return new $my_bar();
    }
}
//my/foo/foo.ts
;
"use strict";
class $my_bar extends $my_foo {
}
//my/bar/bar.ts
;
"use strict";
console.log(new $my_foo().bar);
//my/app/app.ts

The builder sticks the files together in the right order, as if we were writing code in the same file. The dependency of $my_bar on $my_foo is stricter than $my_foo on $my_bar, which means that these modules should be included in the bundle in this order: $my_foo, $my_bar, $my_app.

Using CamelCase

FQN-names, you can also write in camelCase style $myModuleHere. This is not canonical, but if you can't retrain to snake_case, this will work for you.

Using JavaScript

You can use native JavaScript for your code. In that case you must use the *.jam.js' extension. In the future, the need to writejam`(JAM - javasacript agnostic module) will go away.

Практика - взять и написать пример из МАМ Сборка фронтенда без боли.

TODO