search

nextcloud / ansible-collection-nextcloud-admin

The ansible galaxy for your nextcloud administrative needs.
https://galaxy.ansible.com/nextcloud/admin
BSD 2-Clause "Simplified" License
138 stars 77 forks source link

Proposal: make this role an official nextcloud installation method #171

Closed staticdev closed 1 year ago

staticdev commented 1 year ago

@aalaesar @wiktor2200 I see that this role is mature and is working quite well. Nextcloud has partner projects of installations of Nextcloud namely all-in-one and nextcloudpi.

I have the feeling that by putting it under nextcloud organization we give visibility, get more users and even can share between the other official methods supported features (such as backup/restore, certificate config, proxy|load balancer config, etc) to have a better way to manage nextcloud on-prem for more advanced users. I even met the lead engineer from nextcloudpi and we plan on collaborate. What do you think? If you agree, I can reach out to Nextcloud employees and try to make it happen.

aalaesar commented 1 year ago

Hello @staticdev . That's a interesting idea. Although nextcloudpi and all-in-one are much more complete solutions than this role. Talking about collaboration, do you have this role in mind or are you giving context about your engagement in nextcloud related projects ? 😃

staticdev commented 1 year ago

About the collaboration I saw that the bare-metal solution of nextcloudpi is basically a shell script that does almost the same things as this role. It can be that we implemented more apache configs for security and they implemented other things that complement each other. If we collaborate we can try to find the best of both worlds and maybe come with a new solution. This idea is still very new and I plan to study more about both projects when I have free time for it. Of course, if any of you have interest we can even make a team/project.

wiktor2200 commented 1 year ago

Hello @staticdev @aalaesar! I really like the idea of "whole solution", but I'm not convinced to "complete solution" within Ansible role. Every tool has its own purpose, Ansible role's is to automate some steps and be easily used along with any other solution/tool.

nextcloudpi is bunch of scripts which create whole environment (build multiple images). It doesn't emphasizes Nextcloud's config itself. So after creating image you are still getting "clean" NC to configure and then you can even use this role. all-in-one is complex docker environment for NC (preferably Portainer), but still is has its pros and cons. Maintaining AiO project would end up in hundreds of egde cases which should be cover, here's just a list of few examples: different DB (mysql, mariadb, postrgres), multiple certificates (obtain by Let's Encrypt, own self-signed, other CA or wildcard), firewall methods (nftables, iptables or any other cloud provider's), a few storage solutions and even more backup. It's just a peak but I've mentioned at least several dozen of combinations and possible environments.

I would personally prefer adding more and more improvements to this role only. In my opinion, if we want some AiO based on Ansible, we should consider creating Ansible Collection to enhance our role with some other modules.

I've got lots of ideas for improvements, which I've on my TODO list when having some time. Most of them are for proper (best practices) NC config, which I used to change using occ or via NC interface. Some examples (to have it in this role):

aalaesar commented 1 year ago

Hello @wiktor2200 @staticdev, I think you made a good point @wiktor2200 in the difference of use case and tooling between those projects and the role.

They are intended as complete solutions for a given case. They have to make choices and cut corners to get a fiable, working, easy to use solution. Depending on their deployment/automation tools and code they can get a modular or monolithic system. more or less harder to maintain, update etc. (really, I haven't looked into these projects, So I don't judge anything)

The role, on the other hand, is intended to be just one step of a larger installation workflow but it comes with the advantages of ansible:

It implies to use the ansible tools-stack that may not be adequate for self-hosted, one time installation on one host. (unless, of course, if you're a tech power user and you're doing Infra as Code in your garage) Ansible is definitively useful when repeatability comes into account and shine even more on multi-host deployments.

Given all of that, I would not like the role to be "forced" upon those projects just because "ansible is better than bash".

I think their is a lack of solutions on ansible galaxy for nextcloud that would motivate more project to use ansible as an installation tool. As I talked before already on other issues, I think a nexcloud collection is a good way to extend this role exposition. But, blaming myself, I have worked on self-hosted Kubernetes and smart-home things as IaC in my garage rather than starting working heavily on this collection project. 😞

Well those where just my thoughts about the place of the role. @staticdev , I trust you will find some common ground with the other project 😃 .

regards. Aal

staticdev commented 1 year ago

@aalaesar @wiktor2200 maybe I gave some impression that I wanted to merge this role with AIO or NCP, but this is not what I meant. I think each project will remain independent and have different features/use-cases. I am not even sure they can really be combined.

Actually breaking down in concrete steps I have 2 proposals:

aalaesar commented 1 year ago

@aalaesar @wiktor2200 maybe I gave some impression that I wanted to merge this role with AIO or NCP, but this is not what I meant. I think each project will remain independent and have different features/use-cases. I am not even sure they can really be combined.

No worries. You are opening a discussion and thoughts are going all over the place. That's normal. We had to build a foundation about what we agree first, then the subject can be refocused.

Actually breaking down in concrete steps I have 2 proposals:

  • adding install_nextcloud role as a project under nextcloud organization on Github, the same way as AIO and NCP. This gives visibility and bigger user base to test and contribute. I think this project has more potential to evolve faster this way. For that, if you agree I can ask directly people I know from Nextcloud (I even met their CEO in the Nextcloud Conference this year ;) ).

I agree but by proposing a collection rather than a role. It will allow a easier expansion of use cases and codes.

(For my first modules, I'm still hitching my head around how to use occ cli in an ansible modules)

  • about crossing over with AIO/NCP it is still a very early idea, but maybe we could introduce tasks to add backup/restore capabilities in the ansible role in a way that we can reuse/adapt what is already working on both AIO and NCP. This for me would be the greatest improvement, since ability to restore NC is critical for me and streamlining it would be excellent. I am still researching how these projects work more in depth.

So you mean understand and reuse their backup process and format to write a compatible backup role with ansible ? ( I've done my own backup role btw)

wiktor2200 commented 1 year ago

I agree but by proposing a collection rather than a role. It will allow a easier expansion of use cases and codes.

Totally agree with that. Collection would be the best to share with others. 100% out-of-the-box with fool and fail-proof default configs giving A+ mark on Nextcloud check.

I've done my own backup role btw

That looks very interesting, I'll give it a try :wink:

There is still lots of features and options that we can introduce to this role. Or when we decide to build collection there would be even more. I'll try to work on my TODO list mentioned above when having some free time.

The most important is that we are getting better solution with every little step we take. :1st_place_medal:

staticdev commented 1 year ago

@aalaesar @wiktor2200 do you think this repo can be moved to https://github.com/nextcloud or should a new repository be created? (I think the advantage of moving is keep history/stars/forks)

Also regarding the idea of a collection instead.. should it already start the collection with install and backup roles? Or what would be your proposal?

wiktor2200 commented 1 year ago

Why do we need to move it at all? It can be just mentioned in NC documentation with no need to be moved. Or maybe collection could be created in that organization and then this role could be added to collection (just just yaml file)?

staticdev commented 1 year ago

Why do we need to move it at all? It can be just mentioned in NC documentation with no need to be moved. Or maybe collection could be created in that organization and then this role could be added to collection (just just yaml file)?

It does not NEED. It is a proposal and it was well received by Nextcloud and also NCP and AIO are in Nextcloud Github org. IMO it definitely increases visibility for Nextcloud users.

aalaesar commented 1 year ago

Hello there !

it was well received by Nextcloud

@staticdev does this mean you asked some nextcloud member to if they may took the role in their organization ?

Do you think this repo can be moved to https://github.com/nextcloud or should a new repository be created? (I think the advantage of moving is keep history/stars/forks)

Keeping the stats is a good point. I'll see if I can merge the collection's few commits to a new collection dev branch in this repo

staticdev commented 1 year ago

Hello there !

it was well received by Nextcloud

@staticdev does this mean you asked some nextcloud member to if they may took the role in their organization ?

Do you think this repo can be moved to https://github.com/nextcloud or should a new repository be created? (I think the advantage of moving is keep history/stars/forks)

Keeping the stats is a good point. I'll see if I can merge the collection's few commits to a new collection dev branch in this repo

Yes, you are right. I talked to them. And Andy Scherzinger from Nextcloud can help us with that once you give the go.

aalaesar commented 1 year ago

Hello there ! @staticdev , thank you again for your initiative, it is definitively a good idea. Still, I need more infos about implications like:

Is there some doc relative to what happen to projects integrated under the nextcloud organization ? Or maybe should we make a call with a nextcloud community manager or someone related ? regards. Aal.

aalaesar commented 1 year ago

Anyway. The repository next big evolution will be to become a collection. I've made some good progress on some fundation code and module: run_occ_tests

staticdev commented 1 year ago

@aalaesar I will try tot get answers ASAP.

AndyScherzinger commented 1 year ago

Hi @staticdev @aalaesar just invited you to the Nextcloud org on Github and also replied to your mail @staticdev .

To answer the questions raised up to now:

licensing change (if any)

Not needed

rights managements changes on the repository

I just make you an admin and than you take it from there

CI & bots integration

Haven't seen anything that needs to be discussed, so all good I would say

...

Just shoot the questions :)

Is there some doc relative to what happen to projects integrated under the nextcloud organization ?

In a way nothing except the "home" as-in having it moved into the Nexctcloud Github organization. You would still be the owner(s) and can do what you need.

staticdev commented 1 year ago

Thanks a lot @AndyScherzinger =)

@aalaesar here are instructions I got via email also:

The way to transfer it is relatively simple (while it seems a bit
strange from the outside). @aalaesar would simply need to transfer to
the repository to a person me or one of our team leads since we are the
only ones with transfer power to the Nextcloud organization (this way
all your issues, etc. will also move), So it is a "transfer to
@AndyScherzinger" and I then "transfer to Nextcloud Github
Organization". After that I can simply make @aalaesar an Admin of that
repo and then he can grant permissions as needed/wanted. Of course if
you give me a list of Github handles and which role they would need on
the repo I can also do that for you.
aalaesar commented 1 year ago

Hello there ! @staticdev @AndyScherzinger @wiktor2200 I have some fundamental questions before moving the repo to the nextcloud organization.

As said before, I want it to be now a collection of roles and modules for administration. modules and roles will be called with the fully-qualified collection name (FQCN) in playbooks so it is kind of significant the FQCN is <namespace>.<collection-name>.<module or role name> so choosing the collections namespace and name is important as it won't be easy to change it later. (my suggestions are at top of the lists)

  1. what namespace the collection should be in ?

    • obviously become 'nextcloud'
    • stay 'aalaesar'
    • suggest an other idea

  2. What name should the collection have?

    • 'admin' (so FQCN could be nextcloud.admin.something 😃 )
    • 'nextcloud' (but FQCN could be nextcloud.nextcloud.something 😞 )
    • suggest an other idea

  3. Should the repository be renamed in the context of the nextcloud organisation?

    • yes, something like 'ansible_admin'
    • no, stay install_nextcloud
    • suggest an other option

Thank you in advance for your your feedbacks . 😃

Regards, Aal.

staticdev commented 1 year ago

@aalaesar great points.. I like the suggestion to be nextcloud.admin, the name of the repo is not critical since the metadata file is what is important for Ansible Galaxy.. to distinguish I would call it ansible-collection-nextcloud-admin (this is similar to the convention I use for my Ansible roles and also from many of the roles/collections in Galaxy. Alternatively you can also call it nextcloud.admin, this is more of personal taste.

Second point, to have it nextcloud.admin we need to create use nextcloud on Ansible Galaxy (if it is still allowed). For that we are lucky, I just verified it does not exist. But I am really not sure how organizations can log in into Galaxy... maybe only admins on Github such as @AndyScherzinger will be able to login in Galaxy as Nextcloud. Then we will need the API Key to set it as a secret for the new repo. And the rest is CI/CD magic =)

aalaesar commented 1 year ago

Second point, to have it nextcloud.admin we need to create use nextcloud on Ansible Galaxy (if it is still allowed). For that we are lucky, I just verified it does not exist. But I am really not sure how organizations can log in into Galaxy... maybe only admins on Github such as @AndyScherzinger will be able to login in Galaxy as Nextcloud. Then we will need the API Key to set it as a secret for the new repo. And the rest is CI/CD magic =)

I created #186 with a link to galaxy doc to manage that. Given the doc, I don't think a special user is required, it depend on the organisation one is part of. But even if I have accepted @AndyScherzinger invitation, I can't see the nextcloud namespace yet in galaxy. Probably because I don't have any right on any repository in the organization.

aalaesar commented 1 year ago

@staticdev @wiktor2200 about Point 4 on my previous questions. How should this role be renamed in the future collection? Do you have suggestions ? I have to rename it for #183

wiktor2200 commented 1 year ago

If it's possible I would leave the name as it is. Name may be associated with knowledge of the solution among all users of role. Seeing the old name even when collection/FQCN is changed, may be more obvious choice for some users (just using old, proven method). But if there is a need to change it, I don't see any other contraindications :)

aalaesar commented 1 year ago

If it's possible I would leave the name as it is. Name may be associated with knowledge of the solution among all users of role. Seeing the old name even when collection/FQCN is changed, may be more obvious choice for some users (just using old, proven method). But if there is a need to change it, I don't see any other contraindications :)

That's a really good point. You convinced me. So I'll keep the role name and continue.

Thank you @wiktor2200 !

AndyScherzinger commented 1 year ago

Hi @aalaesar, been on vacation the last days / week, so couldn't reply earlier than today. Looping in @blizzz and @mejo- about the Ansible Galaxy part, also because I don't have absolute admin rights on the Nextcloud Github organization.

Probably because I don't have any right on any repository in the organization.

Being a member of the Nextcloud org on Github, you should have write permission on any repo I think.

staticdev commented 1 year ago

If it's possible I would leave the name as it is. Name may be associated with knowledge of the solution among all users of role. Seeing the old name even when collection/FQCN is changed, may be more obvious choice for some users (just using old, proven method). But if there is a need to change it, I don't see any other contraindications :)

That's a really good point. You convinced me. So I'll keep the role name and continue.

Thank you @wiktor2200 !

As I understood the idea of the collection we would not just focus on install but also do other operations no? Maybe keeping install_nextcloud is not good for that but maybe moving to just nextcloud fit better as it is standard for names in Ansible Galaxy.

aalaesar commented 1 year ago

As I understood the idea of the collection we would not just focus on install but also do other operations no? Maybe keeping install_nextcloud is not good for that but maybe moving to just nextcloud fit better as it is standard for names in Ansible Galaxy.

Yes my point was about the role name only. The collection namespace will be nextcloud and it's name admin Given that, the role fqcn would be nextcloud.admin.install_nextcloud. Keeping the same role name for all current users

aalaesar commented 1 year ago

Hi @aalaesar, been on vacation the last days / week, so couldn't reply earlier than today. Looping in @blizzz and @mejo- about the Ansible Galaxy part, also because I don't have absolute admin rights on the Nextcloud Github organization. ... Being a member of the Nextcloud org on Github, you should have write permission on any repo I think.

Hello @AndyScherzinger . No problem 😃

i don't think galaxy will be a big problem. We will have time to manage this subject after the transfert to nextcloud.

AndyScherzinger commented 1 year ago

Sounds good @aalaesar 👍 Please let me know if I can be of any help 😃

aalaesar commented 1 year ago

Hello @AndyScherzinger another question came into my mind relative to the repo name. will I be able to rename it after I transfered it to you/nextcloud ? (maybe I should rename it before handling it if it is not possible)

AndyScherzinger commented 1 year ago

Hi @aalaesar, not sure to be honest, but renaming should be possible either by you or one of our oganization admins after the move, by me during the move or by you before the move.

AndyScherzinger commented 1 year ago

I would think having Ansible as part of the repo name might help to point to the technology behind it. In case that ever change one might rather create a new repo I would assume.

staticdev commented 1 year ago

I would recommend for repo name ansible-collection-nextcloud ;)

aalaesar commented 1 year ago

Hello there ! @AndyScherzinger The collection in coming very soon and I'll transfer the repository to you just after its release.

Once this repository is transfered, updated with new infos and properly imported in ansible Galaxy, Would the nextcloud Org wish to communicate about this ? Ansible is a very popular and robust deployment tool. This would add a new string to the project's bow for visibility.

AndyScherzinger commented 1 year ago

Hello @aalaesar, looking forward to the release 😃 As for the communication part I have no idea, so looping in @jospoortvliet

staticdev commented 1 year ago

@aalaesar before communication would be best if we already have the repo transfered, deployed to galaxy and tested from there, right?

aalaesar commented 1 year ago

@aalaesar before communication would be best if we already have the repo transfered, deployed to galaxy and tested from there, right?

Yep that what I said 😃 Just being busy this week-end.

aalaesar commented 1 year ago

Hello there !

I've just sent the transfert request to you @AndyScherzinger

regards

AndyScherzinger commented 1 year ago

Hi @aalaesar, all

moved the repo to https://github.com/nextcloud/install_nextcloud and granted you admin permissions 👍

Cheers

staticdev commented 1 year ago

Great, thanks everyone for the team work.

We can try to do some alpha releases to see how it goes on Ansible Galaxy now =)

aalaesar commented 1 year ago

And thank you @staticdev for the proposal. :+1: