google / earthenterprise

Google Earth Enterprise - Open Source
Apache License 2.0
2.67k stars 887 forks source link

Slow publish root configuration during upgrades #1910

Closed tst-jlarocco closed 3 years ago

tst-jlarocco commented 3 years ago

Describe the bug

Installing OpenGEE RPMs with a lot (terrabytes) of data in the publish root (/gevol/published_dbs) causes upgrades to take a very long time.

To Reproduce

  1. Install OpenGEE using RPMs created with the Gradle script in /earth_enterprise/rpms.
  2. Create a project and publish a large database.
  3. Create new RPMs
  4. Upgrade OpenGEE using the new RPMs
  5. While the upgrade is running use top or ps -faux to observe the running processes. At some point geconfigurepublishroot, chown -R, and chmod -R will run for significant amounts of time. The larger the publish root, the longer it will run.
  6. Manually run geconfigurepublishroot against the publish root using sudo /opt/google/bin/geconfigurepublishroot --noprompt --path=/gevol/published_dbs/. This will take a long time for a large publish root, even though there should be no changes happening.

Expected behavior

Upgrading or configuring an existing publish root should be fast if nothing is changing.

Desktop

Additional context

The root of the problem is that both geconfigurepublishroot and the installer are changing ownership and permissions on a large number of files, even if it's not necessary to do so. To make matters worse, the installer changes the permissions, calls geconfigurepublishroot, and then changes the permissions again.

One scenario where these changes would be necessary during the install would be to handle the scenario where OpenGEE is installed, a database published, OpenGEE is removed and the geapacheuser and gefusionuser users removed and the UIDs re-assigned. At a later time OpenGEE is re-installed and pointed at the old publish root, but it would now have a different owner and potentially different permissions, so the chown and chmod would be necessary.