pkgdev manifest does nothing, prints nothing, exits 0 anyway

[nethershaw]

The action of writing out the Manifest file is expected not to be sensitive to prior states of the Manifest file.

I'm in a working directory, a git repository, full of well-formed ebuilds that repoman (which wasn't broken, and only has an incomplete replacement, but has been removed from Portage anyway, so thanks to upstream for that forced decision) used to have no trouble with. Manifest is absent. Have wasted an hour contemplating precisely what new and arcane expectation pkgdev has which I am now inexplicably violating because it refuses to tell me.

pkgdev manifest exits 0, prints nothing.

pkgdev manifest -f exits 0, prints nothing.

pkgdev manifest -v exits 0, prints nothing.

pkgdev manifest --debug exits 0, prints nothing.

pkgdev manifest <portage-category>/<portage-package> exits 0, prints nothing.

pkgdev manifest <the complete path to the actual ebuild> exits 0, prints nothing.

No Manifest is ever written with any combination of the above arguments.

Relative location in the repository makes no difference.

Maybe output something to tell me what you're trying to do, especially with -v or --debug. Anything in relation to any of the ebuilds in the current working directory. Or the one whose path I've given you. Throw an exception. Anything at all.

If I do ebuild foo.ebuild manifest just once in foo.ebuild's directory, even if I delete the resulting Manifest file afterward, subsequent invocations of pkgdev manifest -f behave as expected; pkgdev manifest frustratingly exits 0 and does nothing, but the only indication this is the intended behavior when a manifest is already present -- by inference from the existence of the -f argument -- is in the manual page or the manifest subcommand's help output.

[thesamesam]

I understand you may be frustrated but it doesn't lead to a bug report that's easy to act on.

There are two bugs here:

1. Something led to your manifest not being written.
2. The lack of output making it hard to understand the reasons for a decision taken, even if it is (or isn't) correct. I agree more output is useful, at the very least with -v or --debug, even if the conclusion of pkgdev is to do nothing.

I have a few questions:

1. Could you share an example ebuild directory (inc. its contents) (and give the full path name to it) including the Manifest when it's been generated and an ebuild?
2. Is this a repository with thin manifests (see metadata/layout.conf)?
3. What is in the Manifest when it does get generated with pkgdev manifest -f? (I'm interested in particular in the types of entries)
4. What do you expect it to do when a Manifest is present that it isn't doing for you? (In ::gentoo, if you run pkgdev manifest, it will indeed regenerate the Manifest to remove/add entries as required, even if a Manifest file does or does not already exist. The only time it's done nothing for me when things have changed are: 1. the ebuild is broken and can't be sourced, or 2. the repository is actually thin-manifests = true and therefore pkgdev won't regenerate EBUILD and such lines in the Manifest)

[radhermit]

Taking a total shot in the dark since it's been awhile, I'm guessing this might occur when running against repos using thick manifests (e.g. https://github.com/pkgcore/pkgdev/issues/78).

Personally, I never saw the point of a development repo using thick manifests after the gentoo repo moved to git and thin manifests started being used (which is one reason why I barely touched the old, thick manifest code in pkgcore). However, I do agree pkgdev should either handle them, auto-convert them, or throw an error.

[r7l]

As a long term repoman user i felt kinda frustrated for the removal as well. But i am not here to complain about changes. The removal forced me to check out other tools and some of them are really useful (like pkgcheck) and i am glad to have noticed them.

One issue i've noticed in pkgdev that prevents it from writing Manifest files is having typos / errors within the Ebuild. But as stated it does not complain or do anything about it. It simply ends. So you end up without a Manifest but also have no clue about why it didn't write a file. Using ebuild does print out the actual issue.

It would be great to have pkgdev to print out errors if it doesn't write a Manifest file.

[arthurzam]

One issue i've noticed in pkgdev that prevents it from writing Manifest files is having typos / errors within the Ebuild. But as stated it does not complain or do anything about it. It simply ends. So you end up without a Manifest but also have no clue about why it didn't write a file. Using ebuild does print out the actual issue.

Can you please add an example for that situation. I'm failing to create one myself, and with having an example, I can tackle it and find where I'm suppressing the log by mistake.

[r7l]

I've copied over some dependencies from one Ebuild to another and missed to remove quotation marks. So having something like this made it silently fail for me:

RDEPEND="
    dev-python/GitPython
    dev-python/python-gnupg
    ""

You might compare the resulting output from ebuild.

[correabuscar]

From my tests, the exit with no message happens due to this:

https://github.com/pkgcore/pkgcore/blob/1499b0d68307042476402890890f16bd6d060932/src/pkgcore/ebuild/repository.py#L116-L118

            # Manifest file is current and not forcing a refresh
            if not force and manifest.distfiles.keys() == pkgdir_fetchables.keys():
                continue

that seems to check only fetchable(from online) files, such as firefox-118.0.1.source.tar.xz, firefox-118.0.1-kn.xpi etc. but not any existing in-current-dir files like the ebuild(s) such as firefox-118.0.1.ebuild.

therefore the later line which does update the Manifest isn't reached, this one, that is: https://github.com/pkgcore/pkgcore/blob/1499b0d68307042476402890890f16bd6d060932/src/pkgcore/ebuild/repository.py#L165

                manifest.update(sorted(all_fetchables.values()), chfs=write_chksums)

and this line not only does update the fetchables in Manifest but also, internally, on its own, updates the files that are in the current dir like the ebuild files such as firefox-118.0.1.ebuild. So if you were to run it with empty args like manifest.update(set()) it would update the Manifest only for the current-dir ebuild and other existing on-disk files, but no fetchables(they'll be removed from manifest).

So, at the very least, there could be inserted a message there before this continue to say that it won't update the manifest: https://github.com/pkgcore/pkgcore/blob/1499b0d68307042476402890890f16bd6d060932/src/pkgcore/ebuild/repository.py#L116-L118 or, just somehow attempt to update the manifest at that point in the code for only the files on disk, without touching the fetchables from the Manifest.

Index: /var/tmp/portage/sys-apps/pkgcore-0.12.23/work/pkgcore-0.12.23/src/pkgcore/ebuild/repository.py
===================================================================
--- .orig/var/tmp/portage/sys-apps/pkgcore-0.12.23/work/pkgcore-0.12.23/src/pkgcore/ebuild/repository.py
+++ pkgcore-0.12.23/src/pkgcore/ebuild/repository.py
@@ -115,6 +115,9 @@ class repo_operations(_repo_ops.operatio

             # Manifest file is current and not forcing a refresh
             if not force and manifest.distfiles.keys() == pkgdir_fetchables.keys():
+                observer.error(
+                        f"Not updating manifest due to no --force and all fetchables are up to date. https://github.com/pkgcore/pkgdev/issues/108#issuecomment-1754024761"
+                        )
                 continue

             # fetch distfiles

EDIT: Can use something like ebuild file command from man ebuild:

manifest Updates the manifest file for the package. This creates checksums for all of the files found in the same directory as the current ebuild as well as the recursive contents of the files subdirectory. It also creates checksums for all of the files listed in SRC_URI for each ebuild. For further information regarding the behavior of this command, see the documentation for the assume-digests value of the FEATURES variable in make.conf(5). See the --force option if you would like to prevent digests from being assumed.

So, a command like ebuild rust-1.75.0-r1.ebuild manifest would update the ./Manifest in current dir with the proper sizes and checksums for any of the already existing(in ./Manifest) changed files such as ./metadata.xml etc., while pkgdev manifest would just exit instead.