Packaging v3 wishlist

[alexAubin]

• Handle system confs declaratively, similar to the work done on resources:
  • nginx
  • systemd service
  • php
  • fail2ban
  • cron job
  • app confs
• Rework the entire install/remove/upgrade/backup/restore paradigm, probably with a build/init/migrate?/?? paradigm where we limit the number of stuff ran as root : https://github.com/YunoHost/issues/issues/1606#issuecomment-952290204
• Rework app.src format of ynh_setup_source : https://github.com/YunoHost/issues/issues/1978
  • mentioned PR has been merged but there's still some stuff to declarative-ize such as the --keep option
• Declare what technologies an app relies on, something like nodejs version being a resource
• Because systemd conf should now be handled declaratively, it should be possible to have a clear relation between systemd services and applications
• Mandatory systemd hardening ?
• For nginx confs, standardize fastCGI and proxy_pass stuff (proxy headers etc)
• For nginx confs (and maybe others?), explore systematically having an include $app.d/*.conf ? And also stuff like __YNH_DEFAULT_CSP_HEADER__ or __YNH_PORTAL_BUTTON__
• Declarative format for relation to log files and the logrotate story : https://github.com/YunoHost/issues/issues/310
• Clarify the whole story between config panels, app settings, and ynh_add_config
  • Clarify the story of storing config checksums and "people may manually edit stuff" (eg during backup restore, shall we recreate the conf from scratch or ...)
• Moving app install folders to something like /apps or /var/apps/ or idk (and maybe /home/yunohost.app to somewhere else too ... to be discussed .. needs to be thinked about in relation to eg /var/lib/mysql and postgresql too). Basically there's a whole story of "how to could people move the large data locations to some other disk somewhat easily"
• Clarify the whole thing about multimedia ... https://github.com/YunoHost/issues/issues/1981
• Declare what kind of integration is needed for SSO/LDAP : not_relevant (= no SSO info transmitted), user(?) (= only the remote user info), full (= user+password+???)
• Explore the idea that app should have an uuid-like id like foobar-84fe312a instead of just foobar such that restoring foobar on another system isnt blocked by the fact that a different foobar is already installed ...
• chattr -i/+i at least on the settings.yml file
• Explore the idea of upstream being able to have a .ynh/ folder (or similar) with yunohost package-related stuff, similar to debian/ folder
• Explore appimage format to ship app builds with all their deps and theferore simplifies install time and solve the issue of "apps may conflict with each other"
• Should we switch to full jinja for templates x_x ... Or conditional / dynamic properties in manifest ?
• CODEOWNERS ?
• App hooks management is a mess for many reason, we should just use regular ynh_add_config (or ynh_add_config_hook) (or actually in a declarative way in manifest just like for nginx etc)

[zamentur]

• declarative change_url https://github.com/YunoHost/issues/issues/2150

[Salamandar]

• Provide common directories for build :

  • build cache directory (for npm build, cargo, etc), that way admins can decide "ok I delete the build cache dir of apps XX" (also, not backed up)
  • temporary directory (deleted for sure at the end of the step)

• Revamp the logs, with "hierarchical" logs (for now all log files are at the same level, hard to decide which sub-command is about which main command)

[Josue-T]

Hello,

Some others good thing would be:

Would be good to have more standard filesystem hierarchy for app. Except the big discussion about where we should store app code there are also some directories whish are currently completely not standardized like logs and config file.

By example for log should be really good to be able as same as for data to have a standard directory for each app and to have logrotate automatically managed.

On config side for me where are also some more standardization which could be done. Firtly as same as for logs would be good to have a standard dir like /etc/yunohost.app/$app or /srv/yunohost/$app/config. And secondly actually we have 2 different way to manage configuration on yunohost. We have one specific system for the core and an other one for app. For me (and for users) it would be really good if we are able to have one standard configuration system which apply for the core and for the app with the same comportment. And finally it would be really nice to have jinja as the template system for app to have a more standardized and flexible template system.

[Salamandar]

By example for log should be really good to be able as same as for data to have a standard directory for each app and to have logrotate automatically managed.

I fully agree. TBH I would love to see removed [resource.install_dir] and [resource.data_dir] and see, instead some more generic form:

[resources.dirs.install_dir]
# sane defaults for well-known install_dir
[resources.dirs.data_dir]
# sane defaults for well-known data_dir
[resources.dirs.log_dir]
# sane defaults for well-known log_dir
[resources.dirs.downloads_dir]
path = "{{install_dir}}/downloads"
owner = "www-data:rwx"

Firtly as same as for logs would be good to have a standard dir like /etc/yunohost.app/$app or /srv/yunohost/$app/config

Unfortunately, every software comes with its own way configuration "paradigm". gitea uses custom/conf/app.ini, mediawiki uses LocalSettings.php, etc… It's kinda impossible to standardize (or we would need to patch apps… don't want that)

it would be really good if we are able to have one standard configuration system which apply for the core and for the app with the same comportment

Agreed. What could be done though, would be to edit files in /etc/yunohost.app/$app and then call "yunohost app reconfigure $app" or something related.

[orhtej2]

• Split apt deps into runtime dependencies (required, referenced by phony deb) and install time dependencies (like crystal, build-essentials etc)
• Allow restoring with modified settings (database name and especially domain/path) Make way for 'helper' apps - stuff like mongodb could be installed as dedicated app with all the necessary hacks required to make it work.
• Another helper would be bridge helpers to allow mautrix_* to be installed with all of Matrix servers (currently crucial script is a part of Synapse_ynh and bridges are hardwired to use Synapse)

[Josue-T]

Unfortunately, every software comes with its own way configuration "paradigm". gitea uses custom/conf/app.ini, mediawiki uses LocalSettings.php, etc… It's kinda impossible to standardize (or we would need to patch apps… don't want that)

Patching app is not a great solution. But sometime we juste use symbolic link to fix this. And for app which don't have any working solutions this is not mandatory. It could also be managed as currently if needed. But I think most of app should have a solution I think.

[Josue-T]

• Make way for 'helper' apps - stuff like mongodb could be installed as dedicated app with all the necessary hacks required to make it work. Another helper would be bridge helpers to allow mautrix_* to be installed with all of Matrix servers (currently crucial script is a part of Synapse_ynh and bridges are hardwired to use Synapse)

Yes adding a solution to link apps and to manage app dependancy would be really useful but I think a features like this need reflexion beaucaue there are a lot of use cases and there are also a lot of different features that we can add in this context.

[orhtej2]

Yes adding a solution to link apps and to manage app dependancy would be really useful but I think a features like this need reflexion beaucaue there are a lot of use cases and there are also a lot of different features that we can add in this context.

Oh, that was exactly my point :) Make sure dex is uninstalled when outline gets deleted and prevent it from being uninstalled (or at least warn) if doing so would brick another app.

[Josue-T]

Oh, that was exactly my point :) Make sure dex is uninstalled when outline gets deleted and prevent it from being uninstalled (or at least warn) if doing so would brick another app.

But about Dex for me OIDC whoud be added on the core beaucause more and more app will need it.

[Josue-T]

I was also tempted to add in the which list the OIDC support but it's a bit out of scoops. But yes for me more and more app will need this for authentication (synapse by example will soon need it) and it might be good to have it natively. There are by example this library which could do it https://github.com/IdentityPython/pyop

[Salamandar]

Tracked ownership of files installed outside of $install_dir, kinda like real packages from package managers.

Like a replacement of ynh_add_config with ynh_install_file that will save the path of this file in some database, just like apt dependencies. This way, backup/restore/remove can be automatically handled.

ynh_install_file could have several arguments:

• --ignore-changes -> do not save checksum etc
• --no-backup -> do not backup/restore the content, could be only useful for directories like /var/log/$app
• ?

[selfhoster1312]

Should we switch to full jinja for templates x_x ... Or conditional / dynamic properties ?

It's not clear what you mean with "dynamic properties", but jinja is a very good idea. Jinja-like templates are very easy to understand for non-programmers, very well documented, and have several implementations (with minor differences) in many programming languages.

[Salamandar]

Have a "under maintenance" 503 page during upgrade / change_url like some apps do (discourse wordpress etherpad lutim acropolis pihole my_webapp).

[selfhoster1312]

Have a "under maintenance" 503 page during upgrade / change_url like some apps do (discourse wordpress etherpad lutim acropolis pihole my_webapp).

In the same vein, maybe add a setting to define what to do when the app backend is unreachable. Right now i think somehow we end up back on SSOWat homepage which is really confusing. Maybe some app just get a generic 503, maybe some have an integrated maintenance/status page, etc...

[Salamandar]

A suggestion of an $install_dir topology, considering what we learned:

$install_dir/
├── bin/
│   ├── corepack binaries
│   ├── composer
│   └── what else?
├── sources/ (ynh_setup_source)
├── www/ or dist/
├── cache/
│   ├── things that wont be backed up
│   ├── and can be deleted automatically by yunohost 
│   └── (nodejs cache, composer cache, cargo cache, etc)
└── garbage files because it's a $HOME so .bashrc etc

Also directories could be created automatically. For example ynh_use_node creates cache/npm etc.

The bin directory could also contain things to run yunohost like aliases for yunohost app shell