refactor digital twin frontend to use publish tools backend [development_publish2]

[despiegk]

principle

• publishtools run as daemon (e.g. in tmux)
• publishtools check for changed content on remote git every 5 min and pull changes in, run in same process, this will require the publishtools wiki section to run as thread I think so its easy to restart
• twin frontend is proxy towards publishtools which keeps all in mem for now
• for websites we use the build function, and twin server uses the build websites

requirements

• [ ] twin server working see twin server 0.9
• [ ] document in publishtools/docs how to use
• [ ] wiki server works as proxy towards publishtools backend
• [ ] cant we do this opengraph more elegant, now too many duplicates, it should be generated out of lot less info and conventions e.g. just name of logo (not full url), description comes from site_config file, type auto, ...
• [ ] create script to launch twin server +publ backend on web & wiki small example for GITPOD !!!
• [ ] twin frontend calls rest function on publishtools to know which domains to serve
• [ ] use branch development_publish2 on all required repo's even the content ones we use in example

remarks

• DONT remove flatten function, its one of the modes which still need to be supported
• note pull/reset/branch name this makes sure the publishtools will see if new content every 5 min and pull changes in ignoring all changes and make sure is on right branch
• check if mem use stays under control if checking pulling many times
• for gitpod can use the path so it sits in the directory of gitpod which keeps the content so we don't have to use dockerfile for that part.

advantages of this approach

• can use dynamic macro's (at runtime)
• easier to deploy
• better version control

other info

• https://github.com/crystaluniverse/crystallib/issues/17

[Hamdy]

We have few comments on the issue

1- To remove publish function means we need to have a web interface int the nodejs servr with sites.json content which we can add/remove sites/wikis then after saving we reload the file and pull new content or so is this correct?

why do we need this?

how are we going to publish threefold sites/wikis in this case ?

2- Having separate publishtools daemon is more complex than doing same functionalities in nodejs server

1. if sites.json / config changed we need to call publishtools to pull or build websites or reload wikis in memory
2. inconsistency , now we gonna serve wikis from publishtools daemon memory and websites from file system?
3. separate http api to get content ? why not do flattening on when there's new change and we can cache content of wikis in the nodejs memory ?

no: we have already development functionality, so there is nithing new 2 no: we will also do websites same way eventually

OK

2- opengraph can be simplified yes

3- digitaltwin frontend was separated into separate vue.js site that can be served with publisher itself so we can add it as this is done already

{
                "name": "www_digitaltwin",
                "url":  "https://github.com/threefoldtech/www_digitaltwin",
                "branch":       "",
                "pull": false,
                "reset":        false,
                "cat":  2,
                "shortname":    "digitaltwin",
                "path_code":    "",
                "domains":      ["digitaltwin.threefold.io"],
                "descr":        "Digital twin frontend",
                "groups":       [],
                "acl":  [],
        },
}

why vuejs?

frontedn of digitaltwin from jimber is vue.js not gridsome, needed little tweak to support building