search

serhepopovych / simple-cdd-meta

Debian and it's derivatives Simple-CDD preseed configuration with profiles
MIT License
6 stars 3 forks source link

debian-cd does not add packages mentioned in Recommends and Suggests tags #19

Closed serhepopovych closed 4 years ago

serhepopovych commented 4 years ago

It is discovered that on Debian 9 (Stretch) and Debian 10 (Buster) not all packages present in target ISO images making installed system package set differ whenever access to network repository available or not.

In first case system installed with packages mentioned in Recommends and Suggests tags of package control file. In later, when no network available, system installed with limited functionality missing number of packages as they not available on installation media ISO image.

Symptoms like ones described for Debian Edu project here. There is branch for fix candidate in debian-cd, however it neither applies correctly for the current debian-cd master nor looks as correct for me.

Further investigation shows that problem does not affect earlier Debian releases (7 and 8). However using debian-cd from 9/10 does not make issue reproducible on these releases.

Need more in-depth look at sort_deps from debian-cd as this is origin of issue according to contents of packages and packages.amd64 files that created by tools/sort_deps and does not contain recommended packages.

serhepopovych commented 4 years ago

In general problem is solved in private repository of debian-cd. However implementation raises additional problems like too deep recursion of add_missing() helper in tools/sort_deps for which perl complains with warning.

This is caused by sub optimal dependency resolution process where same package passed in $list to add_missing() multiple times.

Helper variable used to track packages ($pkgin) from upper calls of add_missing() does not help much.

Furthermore $list is array and therefore is_in_dep_list() has O(n) complexity, together with deep recursion making performance even worse. Making it hash isn't trivial. It is very likely that $parents must be turned into hash, mirroring contents of $list for O(1) lookups.

Take note that hash should be with package name as key and array as value holding multiple package descriptions as "Version" and "CmpOp" might differ.

serhepopovych commented 4 years ago

Improving performance of add_missing() with hash tables for $parents as described in comment 1 is far beyond of this ticket. Now even with perl complains about too deep recursion original problem is fixed. However it is possible that one day we might be hit by perl recursion problem. Let's keep it for future (upstream)? work.

serhepopovych commented 4 years ago

Deep recursion problem now solved. There is no need in more complex hash indirection.

Main contributors to deep recursion problem was add_recommends() and add_suggests() subroutines that run add_missing() recursively for every package already present in the list.

Rather than only package whose Recommens and Suggests processed.

Pushed directly to master with commit 1bb7890d6acb ("simple-cdd-meta/debian: Update debian-cd submodule to include sort_deps fixes").