search

clearlinux / swupd-server

Software update server (deprecated)
Other
12 stars 16 forks source link

www/*/delta directory needed? #86

Open pohly opened 6 years ago

pohly commented 6 years ago

Under which circumstances does the swupd client need the individual files from the "delta" directory?

I think "swupd update" looks for a pack--from-.tar, which already contains the individual delta files. So that doesn't need the individual files, does it?

Example:

/tmp/swupd/www/18150/
├── delta
│   ├── 18140-18150-206d61ff6418533a8bc82efc0d0c440ca2d7cfcb82b613757dfb03ebda3df4ae-7d892d8129792794e38b85456eb5a22fb7030c42c612cd4ac444c1fa39ee9237
│   └── 18140-18150-65deb3f9ac1b156c7d49b1bdcdfa16453fa5cd6df3e379a86c97ab73e445597b-55f2651b387f26b7855778a81e84a13607d6ea9481e5c407c5cf8fb33dcbbd69
├── files
│   ├── 48b7bf6f095612b3f665a06be334f1dcea32260d117f2c3591dc64090183593a.tar
│   ├── 55f2651b387f26b7855778a81e84a13607d6ea9481e5c407c5cf8fb33dcbbd69.tar
│   ├── 7d892d8129792794e38b85456eb5a22fb7030c42c612cd4ac444c1fa39ee9237.tar
│   ├── c2061a1d767a4e55d6289db93d30fffeb14d2de89b3ee7dbc058a30011a638ea.tar
│   └── df607e9fb157bc045f209d17b450a6b78235bf2965cafed325413839666406bd.tar
├── format
├── Manifest.full
├── Manifest.full.tar
├── Manifest.MoM
├── Manifest.MoM.tar
├── Manifest.os-core
├── Manifest-os-core-delta-from-18140
├── Manifest.os-core.tar
├── pack-os-core-from-0.tar
├── pack-os-core-from-18140.tar
└── swupd-server-src-version

2 directories, 18 files
pohly@pohly-desktop:/fast/build/refkit/intel-corei7-64$ tar tf /tmp/swupd/www/18150/pack-os-core-from-18140.tar 
delta/
delta/18140-18150-65deb3f9ac1b156c7d49b1bdcdfa16453fa5cd6df3e379a86c97ab73e445597b-55f2651b387f26b7855778a81e84a13607d6ea9481e5c407c5cf8fb33dcbbd69
delta/18140-18150-206d61ff6418533a8bc82efc0d0c440ca2d7cfcb82b613757dfb03ebda3df4ae-7d892d8129792794e38b85456eb5a22fb7030c42c612cd4ac444c1fa39ee9237
staged/
staged/c2061a1d767a4e55d6289db93d30fffeb14d2de89b3ee7dbc058a30011a638ea/
staged/df607e9fb157bc045f209d17b450a6b78235bf2965cafed325413839666406bd
staged/48b7bf6f095612b3f665a06be334f1dcea32260d117f2c3591dc64090183593a
Manifest-os-core-delta-from-18140
phmccarty commented 6 years ago

The www/*/delta directories are not needed at the moment. Historically, swupd-client downloaded individual deltas for an update if they were not present in delta packs. It no longer has this capability.

phmccarty commented 6 years ago

However... this being said, keeping www/*/delta around on the server where swupd-server runs is an optimization, because swupd_make_pack will use previously created deltas for delta packs in the current build (as necessary).

pohly commented 6 years ago

If the "delta" dir is only for swupd-server, should it really be under "www"? It makes hosting the "www" directory more expensive on a content delivery network.

phmccarty commented 6 years ago

@pohly No, they do not need to be under "www", but I'm not seeing a good home for them at the moment; the "image" directory is primarily for swupd-server input (chroot content), and "www" is for its output. The standalone deltas could be considered intermediate files given the current implementation, since they are created and consumed by swupd_make_pack.

Do you have any ideas about where swupd-server should store the files? As a workaround, you could set up an rsync filter rule to exclude www/*/delta when publishing the content.

pohly commented 6 years ago

@phmccarty perhaps introduce a "cache" directory alongside "image" and "www"? Then use it for anything that is worth preserving across builds, but with the guarantee that its content can be re-created at any time and thus does not need to be backed up with the same level of reliability as "www" (if it gets backed up at all).

Yes, special rsync rules could achieve the same goals, but then using swupd becomes harder and requires more knowledge about its internals.