search

danielmarschall / oidplus

OIDplus 2.0 - An OpenSource online Registration Authority for OIDs and other Object Types
https://www.oidplus.com
Apache License 2.0
10 stars 6 forks source link

Multi-Tenancy capability #44

Closed danielmarschall closed 2 weeks ago

danielmarschall commented 4 months ago

Based on an idea by @wehowski : https://startforum.de/content/perma?id=2039

A few basic ideas:

... Multiple RA using one OIDplus codebase and one database

... It simplifies "Hosting" of OIDplus systems

A lot of things need to be considered / decided:

... Plugins: Should Tenancy-Owners be able to install plugins (just for their system)? Then we need to take care that the plugin-loader only loads plugins that are enabled for the Tenancy.

... Should the web visitor see that there are multiple users in one system, i.e. the system has a "IANA PEN" root, which you can extend, and then you see "RA 123", "RA 456", etc. ? One one hand, this might be nice because it might look like a "Master OIDplus system" eventually, but on the other hand, it has the disadvantage that the Tenancy owner cannot choose their own design that represents their RA. For example, this RA has a pretty nice and unique design: https://oid.r74n.com/ So I rather would say Multi-Tenancy means that the RAs are using one codebase, one database, one server folder, but they have different URLs and Designs. (If we allow multiple per-tenancy-plugins, then it is required to have different URLs, because the plugins might change the menu structure)

danielmarschall commented 4 months ago

@wehowski Please add all your thoughts in re the Multi-Tenancy-Topic into this GitHub issue. Thank you!

danielmarschall commented 4 months ago

At "hosting.oidplus.com" I have a few standalone systems, so these would be a perfect possibility to test and develop the multi-tenancy.

So, I think a tenant should have:

I have the following idea how to do this change without a lot of effort:

{
"https://hosted.viathinksoft.de/abc/" => ["userdata_abc", "userdata_pub_abc"],
"https://hosted.viathinksoft.de/def/" => ["userdata_def", "userdata_pub_def"],
"https://hosted.viathinksoft.de/xyz/" => ["userdata_xyz", "userdata_pub_xyz"],
"default" => ["userdata", "userdata_pub"]
}

What do you think?

wehowski commented 4 months ago

... Should the web visitor see that there are multiple users in one system, i.e. the system has a "IANA PEN" root, which you can extend, and then you see "RA 123", "RA 456", etc. ?

I would suggest to decide how an OID can be REGISTERED vs. how it can be ADMINISTRATED. To have a single root OID would prevent to add other arcs, e.g. I would suggest to offer a registration formualr, where we can do the validation for PEN automatically (e.g. by confirmation email to the registered RA). This way we could provide (automatically) registered OIDs are ~valid.

So, I think a tenant should have:...

Yes, that sounds good! Maybe it would make sense to first seperate config/db/dirs/paths as the question about to customize plugins is one with different possible solutions?

On a tenancy enabled system, there will a file tenants.json. This file might look like this:...

Yes. Maybe it yould make sense to use a php file or to have the tenants.json in a none-readable dir? The question is if it would be reasonable to save (addtional) tenancy config in (central) DB? To get the tenant from request in common there are

For cron/console maybe we could use an argument for tenant-ID/Domain/path like the PHP build in web-server?

Software Updates should be done by admin only, or it should be on a seperated hosting multi-tenant system as you have now on hosted.oidplus? Because we have a single runtime code an cannot personalize it for tenancy in a comfortable way?

danielmarschall commented 4 months ago

Removing the hardcoded userdata/ and userdata_pub/ will be a lot of work. We could do it step-by-step as preparation for the multi-tenancy. As soon as the hardcoded strings are replaced by OIDplus::getUserDataDir(), we have freedom to implement this method as we want.

$ grep -r "userdata/" | cut -d ':' -f 1 | sort | uniq | grep -v pristine
changelog.json.php
dev/find_copyright_missing
dev/test_database_plugins
doc/config_values.md
doc/developer_notes/database/firebird_notes.txt
doc/developer_notes/database/oracle_test_vm.txt
doc/install_alpine_linux.md
doc/install_facebook_oauth2.md
doc/install_fail2ban.md
doc/install_google_oauth2.md
doc/install_ldap.md
doc/oidplus_custom_guid.md
favicon.png.php
.gitignore
.idea/workspace.xml
includes/classes/OIDplusAuthUtils.class.php
includes/classes/OIDplusBaseConfig.class.php
includes/classes/OIDplus.class.php
includes/classes/OIDplusConfig.class.php
includes/classes/OIDplusConfigInitializationException.class.php
includes/oidplus_limits.inc.php
nginx.conf
oidplus.min.css.php
phpstan.neon.dist
plugins/frdl/publicPages/1276945_rdap/OIDplusRDAP.class.php
plugins/viathinksoft/adminPages/010_notifications/OIDplusPageAdminNotifications.class.php
plugins/viathinksoft/adminPages/050_oobe/oobe.php
plugins/viathinksoft/adminPages/110_system_config/OIDplusPageAdminSystemConfig.class.php
plugins/viathinksoft/adminPages/120_registration/OIDplusPageAdminRegistration.class.php
plugins/viathinksoft/adminPages/900_software_update/OIDplusPageAdminSoftwareUpdate.class.php
plugins/viathinksoft/adminPages/900_software_update/private/gen_serverside_git
plugins/viathinksoft/adminPages/900_software_update/private/gen_serverside_svn
plugins/viathinksoft/adminPages/901_vnag_version_check/vnag.php
plugins/viathinksoft/adminPages/902_systemfile_check/OIDplusPageAdminSystemFileCheck.class.php
plugins/viathinksoft/adminPages/920_nostalgia/export_dos.php
plugins/viathinksoft/adminPages/920_nostalgia/export_win.php
plugins/viathinksoft/captcha/vts_challenge/OIDplusCaptchaPluginVtsClientChallenge.class.php
plugins/viathinksoft/database/sqlite3/OIDplusDatabaseConnectionSQLite3.class.php
plugins/viathinksoft/database/sqlite3/OIDplusDatabasePluginSQLite3.class.php
plugins/viathinksoft/design/default/oidplus_base.css
plugins/viathinksoft/design/ironbase/oidplus_base.css
plugins/viathinksoft/language/dede/messages.xml
plugins/viathinksoft/logger/300_userdata_logfile/OIDplusLoggerPluginUserdataLogfile.class.php
plugins/viathinksoft/objectTypes/gs1/barcode.php
plugins/viathinksoft/publicPages/000_objects/OIDplusPagePublicObjects.class.php
plugins/viathinksoft/publicPages/000_objects/welcome$dede.html
plugins/viathinksoft/publicPages/000_objects/welcome.html
plugins/viathinksoft/publicPages/092_forgot_password_admin/OIDplusPagePublicForgotPasswordAdmin.class.php
plugins/viathinksoft/publicPages/095_attachments/OIDplusPagePublicAttachments.class.php
plugins/viathinksoft/publicPages/500_resources/OIDplusPagePublicResources.class.php
plugins/viathinksoft/publicPages/500_resources/show_icon.php
polyfill.min.js.php
res/ATTENTION.TXT
setup/includes/setup_base.js
setup/index.php
setup/setup.min.css.php
userdata/attachments/info.txt
userdata/baseconfig/info.txt
userdata/cache/info.txt
userdata/cache/master_changelog.json
userdata/cache/translation_d96733c0588ec4e1450d9b821cda9859.ser
userdata/database/info.txt
userdata/favicon/info.txt
userdata/info.txt
userdata/logs/info.txt
userdata/private/info.txt
userdata_pub/info.txt
userdata/resources/info.txt
userdata/styles/info.txt
userdata/welcome/info.txt

$ grep -r "userdata_pub/" | cut -d ':' -f 1 | sort | uniq | grep -v pristine
plugins/viathinksoft/adminPages/902_systemfile_check/OIDplusPageAdminSystemFileCheck.class.php
danielmarschall commented 3 weeks ago

From internal email communication (edited code):

Als Vorbereitung würde ich den CORE so ändern, dass alle Zugriffe auf userdata und userdata_pub gekapselt werden. Um es sogar noch mehr flexibler zu machen würde ich es so machen:

Vorher:

$irgendeinedatei = file_get_contents(OIDplus::localpath()."userdata/baseconfig/test123.php");

Neu:

$irgendeinedatei = file_get_contents(OIDplus::getUserDataDir("baseconfig")."test123.php");

Dadurch kann die Funktion OIDplus::getUserDataDir() entscheiden, dass beispielsweise "baseconfig" Mandant-Abhängig wäre, aber "cache" nicht.

Die Funktion OIDplus::getUserDataDir können wir im Anschluss in Absprache miteinander implementieren, ggf. sogar Plugin-Spezifisch.

Eine mögliche Implementierung könnte sein:

function tenantSubDirName(): string {
  return $_SERVER['HTTP_HOST'];
}

function getUserDataDir(string $subdir, bool $public=false): string {
  $localdir = OIDplus::localpath(); // contains trailing dir separator
  $priv_or_pub = $public ? "userdata_pub/" : "userdata/";
  $tenant_dir = "tenant/".tenantSubDirName()."/";

  // Tenancy dependant dir `userdata/tenant/<tenantSubDirName>/$subdir` existing? Then use this.
  $candidate1 = $localdir.$priv_or_pub.$tenant_dir.$subdir.'/';
  if (is_dir($candidate1)) return $candidate1;

  // General dir `userdata/$subdir` existing? Then use this.
  $candidate2 = $localdir.$priv_or_pub.$subdir.'/';
  if (is_dir($candidate2)) return $candidate2;

  if (is_dir($localdir.$priv_or_pub.$tenant_dir)) {
    // This is a tenancy-enabled system. Therefore, create tenancy-dependant dir by default
    mkdir($candidate1);
    return $candidate1;
  } else {
    // This is a non-tenancy-enabled system. Therefore, create a general dir by default
    mkdir($candidate2);
    return $candidate2;
  }
}

VORTEILE: Dadurch, dass userdata abstrahiert wird, haben wir den Vorteil, dass wirklich ALLES (bis auf die Plugins) individuell gestaltbar ist. Cache, Public Key, Welcome Pages, Baseconfig, einfach alles!

Bzgl. Datenbank-Tabellen: In meiner Variante würde ich dem Administrator die Wahl lassen, das userdata Verzeichnis so bestücken, wie er möchte. Wenn man möchte, dass der DB-Tabellen-Präfix automatisch generiert wird, dann kann man ja gerne Dinge machen wie setValue('TABLENAMEPREFIX', 'oidplus'.md5($mydomain).'_') Die Tabellen werden bei OIDplus ja automatisch erzeugt, wenn die Datenbankverbindung OK ist. Deshalb müssten wir da nix großartig beachten?

@wehowski Is that OK for you?

danielmarschall commented 2 weeks ago

The change was easier than I thought... Just took 3 hours.

I have now replaced all occurences of userdata/ with OIDplus::getUserDataDir().

Theoretically, with this approach, multi-tenancy should already be possible.

In userdata/info.txt , I have added the following explanation:

Tenants
-------

Multiple domains can use the same OIDplus installation but have different
`userdata/` and `userdata_pub/` directories. Therefore, simply create
a directory named `userdata/tenant/www.example.com` where `www.example.com` is
your host name.
Then, if the user visits this domain name, then the data inside
`userdata/tenant/www.example.com` will be chosen rather than `userdata/`.
Each domain can therefore have different database connections (or simply different
table prefixes), different resources (welcome page), etc.

@wehowski Can you please give me feedback on commit 773f4c1 ? Do you think it is useful and can you integrate this userdata abstraction in your tenant plugin, or doesn't it fit in your plugin development? As I said earlier, I try to avoid that we develop two different solutions. So I hope you can integrate parts of my solution into your solution. Feel free to request any changes if you need some!

More things to do:

wehowski commented 2 weeks ago

Hello Daniel, although I did not fully test yet, so far, it looks PERFECT!

danielmarschall commented 2 weeks ago

@wehowski Hello, you can either download the latest ZIP file from Github (or clone it), or download the SVN snapshot here https://www.viathinksoft.com/download/235/oidplus2_svn.tar.gz

When testing, please also look at this method: https://github.com/danielmarschall/oidplus/blob/master/includes/classes/OIDplus.class.php#L2106 This method is very questionable at the moment, so I am not fully sure about it. You know that I have several systems at https://hosted.oidplus.com/.../ where each OIDplus system has an own directory. I kinda want multi-tenancy here (without defining sub domains), but don't know how to do it. Probably something symlink-related is the solution?

danielmarschall commented 2 weeks ago

Also a very funny line of code: https://github.com/danielmarschall/oidplus/blob/master/cron.sh#L37

danielmarschall commented 2 weeks ago

Please wait with the testing. There are a few things broken which I am currently fixing...

danielmarschall commented 2 weeks ago

Great news! I have now solved a lot of problems and you are free to test the beta version (download the ZIP at GitHub or the SVN snapshot at oidplus.com).

The following systems are now all tenancy systems, meaning I have now only one OIDplus installation which I need to care about!

How does it work in my case?

All directories except one are symlinks:

[22:28]oidplus@viathinksoft.de:~/hosted$ ls
insgesamt 12K
lrwxrwxrwx  1 oidplus oidplus   16 12. Jun 14:21 cjydev -> viathinksoft/
-rw-r--r--  1 oidplus oidplus   52 25. Nov 2023  index.php
lrwxrwxrwx  1 oidplus oidplus   16 12. Jun 14:21 infernostars -> viathinksoft/
lrwxrwxrwx  1 oidplus oidplus   16 12. Jun 12:38 r74n -> viathinksoft/
-rw-r--r--  1 oidplus oidplus  178  2. Dez 2023  robots.txt
drwxr-xr-x 15 oidplus oidplus 4,0K 12. Jun 22:08 viathinksoft/
lrwxrwxrwx  1 oidplus oidplus   16 12. Jun 13:41 viathinksoft_demo -> viathinksoft/

The userdata is as follows:

[22:28]oidplus@viathinksoft.de:~/hosted$ ls viathinksoft/userdata/tenant/
insgesamt 16K
drwxr-xr-x 13 oidplus oidplus 4,0K 12. Jun 14:53 hosted.oidplus.com__cjydev/
drwxr-xr-x 13 oidplus oidplus 4,0K 12. Jun 15:00 hosted.oidplus.com__infernostars/
drwxr-xr-x 13 oidplus oidplus 4,0K 12. Jun 12:34 hosted.oidplus.com__r74n/
drwxr-xr-x 13 oidplus oidplus 4,0K 12. Jun 14:54 hosted.oidplus.com__viathinksoft_demo/
lrwxrwxrwx  1 oidplus oidplus   24 12. Jun 12:39 oid.r74n.com -> hosted.oidplus.com__r74n/

ALSO FIXED:

EDIT: Released as 2.0.1.22 to the public (still "beta" though)