Discussion - Decentralization

[AndreasS2501]

Hello (Again) Human Connection,

Congratulations to the alpha! I discussed some ideas around decentralization here: https://github.com/HumanConnection/documentation/issues/2#issuecomment-341371041

Is there a technical roadmap for human connection?

What role do you see for decentralization in the future of HC? Have you looked at projects in the Ethereum space? Like https://akasha.world/ or userfeeds.io ?

Thanks!

[ghost]

Hi Andreas,

I am looking into holo at the moment waiting for delivery of their hardware and then do some tests to see how it works out.

HoloChain: https://medium.com/holochain/holochain-reinventing-applications-d2ac1e4f25ef Holo Project: https://holo.host

Best, Seb

[appinteractive]

We will start some features for internal school networks and might add there some decentralized features.

[realB12]

Human Connection API

Why micro container matter

Intro

When following the (legacy) discussion about "decentralization" on https://github.com/Human-Connection/API/issues/67 I have realized that "decentralization" must not been just an after-thought, but implies important architectural constraints about how to setup the project right from the beginning.

Having bee part of some "virtualisation" projects for moving travel agencies and AI research projects into the (IBM and Amazon) Cloud, I came to the conclusion, that thinking in and following best practices for micro service architectures would have made life much more easier, solutions more flexible and performance cheaper (today scaling is no longer a question of an upper limit and response time, but still a question of the money you are ready and willing to throw against hardware, because you didn't design your architecture and software for speed right from the beginning).

So when seeing, that the HC team has created HUGE Docker Containers I think the team should review and discuss its current architecture in general. This to make sure, that it has all the features and design that makes scalability and speed for low money a second nature right from the beginning.

As changing the whole thing later (once you realize that the current monoliths do not scale) is not just about code, but also about required expert skills, team-processes, risks and finally money etc.).

Dockerisation is a must have. So this is no question.

So the discussion not not about Docker, but about the content of Docker containers respectively about the question how you distribute your code into distinct, independent, small blocks that all can run and scale on their own as a service. My advise is, that all the project should follow a practice, where what ever can run in isolation should be run in isolation, and therefore in a separate container to be used as a separate service that can be deployed, scaled and optimized by its very own nature and does not have to compromiseon on an average mix of resources that all are competing on the same machine.

So, I guess you give the following text some thoughts. I am open for discussion.

Why native installations and micro container matter

A native, non-Docker Installation demands more work during setup, but comes - especially for experienced developers - with the following advantages:

Separation of (archtiectural) Concerns (SoC)

As - for instance - the current HC API- Docker Container always is starting the same versions of DB und API-Server together (and the whole thing will not run, when either component fails), a native installation allows separate handling, maintenance and development of either part.

When working on one component, in an isolated setting you are 100% sure not to impact the other (which, for debugging and repository merging, has advantages and is reducing considerable risks for chaos and conflicting with the work of others).

Further it drives the team to clear roles and responsibilities and standardized, well documented processes - which is a paramount requirements when teams are growing, skills get expert level an complexity is getting high:

Separation of Responsibilities (CbR)

Although it feels good to be master of everything, and this is best practice for small prototyping projects, this approach does most often not make much sense for larger teams and/or full scale, productive environments.

Clear roles and responsibilies with such implied agreement on well defined communication and process patterns are (and "agile" did not change that) the base for dealing with continuously growing complexity, responsibility and parallel running, real time activities.

So, even when you are just a small team in the prototype phase, it makes sense to have distinct "roles" in mind - even in case you are the only person around who has to take them all. If you do, make sure, you know in which role you currently are and close the job in your current role, before changing your hat.

Usage of cloud hosted, 3rd party maintained, common services

Running for instance just the DB in isolation on an external (cloud)-server, you will - as a developer - never have to deal with the DB's internals and further it does not consume any of your workstation's resources.

It will be easy to switch between different DB versions or different (test)-data forth and and back without shuffling Git-hub code or changing your own work in progress.

Relay on Cloud based Team Services (CbS)

Once we will have distinct, well documented and maintained components in the cloud, the architectural footprint for developing your own stuff (from GitHub hosted templates) will become much smaller, build-cycles will become much faster, troubleshooting easier, inter-team conflicts less and your developer life is no longer a walk in the fog in unknown areas, but a focused experience, where you will have everything under control what really matters for your dev-work.

Relaying in external services becomes especially important, when you local setup does not support certain components (e.g. the HC Thumbnails Server does not run on windows), or you might not have the license or the skills to run or the time to setup and troubleshoot such services on your own.

Production ready, scalable Microservice Architecture

Because in highly scalable hosting environments, for instance DB and API-Server will most probably never run in the same container, it is a good idea and it follows best practice, to keep such components isolated in development as well and sort out all problems locally, before packing them into containers. By building isolated micro-containers you will be able to run every combination of different components in parallel. You might not see this at the project start yet, but be reminded, that in full blown productive environments, you have to troubleshoot various systems for various customers for different scenarios for many platforms for different reasons and settings (training, education, production, test, demo, etc.). Building an all including Docker container for every single scenario just does not make sense.

Use your own local tooling and development process

Especially when you, as a developer are used to work with your own (not team shared), well known, specialized profiling, analyzing and debugging tools on isolated components, a native installation has its advantages.

[roschaefer]

@realB12 thank you for your input. Could you be a little more specific, what actions and measures do you have in mind?

For suggestions and ideas, we have our discord channel, here's an invitation link: https://discordapp.com/invite/NgVpvx9 @realB12 did you already join?

By the way, I know from other Github repos that discussion issues used to get closed. The reasoning is to keep the number of "open" issues small, because developers frown at repos with many open issues. Another reason is that it's unclear when such a discussion issue gets closed, ie. when a task is finished.

[Lulalaby]

How about if you take a look at our code, do a local rewrite according to your wishes and let us look at it :)

[realB12]

This is about the 10th time I provide something you are telling me that I should look to your code. This is frustrating. It's like planning the next generation city and you are sending me an invitation to have a look at your house... The wrong level I guess. Anyway: I am confused anyway where and how requirements, plans etc. are handled here. It is really difficult for me to get the big picture. I am looking for the citymap but folks are just talking about farming and housing...

[roschaefer]

@realB12 I'm sorry for you that you feel frustrated. Especially, since you want to contribute value to our project, right? Becoming frustrated early on will make it difficult for you to contribute to any open source project, not only this one. You can still try to fix one of our 'good first issues' and/or come to our discord server to get some help.

[AndreasS2501]

Its a bit sad how this turned out @realB12 seems to have a difficult attitude towards discussions. There is no single "perfect" technical way to approach to solving a generic set of problems. Its really important, especially when your building a social network to incorporate the ideas and thoughts of other people which could potentially become members of your community. If we take a step back, I think it could help ackownledge that many problems in the open source space are still cornered arround issues of how to successfully integrate collaborations from other people. In the very early days I talked with dennis hack about this too. In particular about the ethereum community. So its not only about technical issues but really about social issues.

The book Social Architecture by Peter Hintjens gives some good ideas.

Its not (only) about Code contributions but how to communicate values. Simply voting applications are not sufficent. Conflict resolutions mechanisms like in like from the Argur , Oracles or mattereum are giving some ideas where the discussions should be. HC went a long way from technical issues (the one developer code base etc..) but to make real progress to be a real social network there need to be more discussions about values, and its not a solved problem in general. But one that every open source project needs to solve eventually. So I wanted to thank you for keeping this thread so long open. I hope that In the future HC can collaborate and grow with all other open source projects.

[ghost]

You're all welcome @AndreasS2501 - the topic isn't done yet and we will need a deeper discussion at some point in the future.

[realB12]

@roschaefer Don't worry. Have been on many successful projects :-) @Andreas: I agree with your post, but I think, this thread here is not the right place to discuss generic communication issues and project management (which urgently would be required).