Traceback when publishing with a skipped package

[soininen]

I created new releases while skipping spine_items:

conduct release --exclude spine_items --bump patch -c release.toml

When trying to publish with

conduct publish --pkgtags pkgtags.json -c release.toml

encountered this Traceback:

┌─────────────────────────────── Traceback (most recent call last) ────────────────────────────────┐
│ spine-conductor\orchestra\__main__.py:78 in publish                   │
│                                                                                                  │
│    75 ):                                                                                         │
│    76 │   """Push Git tags to GitHub and publish packages to PyPI"""                             │
│    77 │   conf = read_conf(f"{config}")                                                          │
│ >  78 │   publish_tags_whls(conf, pkgtags)                                                       │
│    79                                                                                            │
│    80                                                                                            │
│    81 _xtest_doc_ref = (                                                                         │
│                                                                                                  │
│ ┌─────────────────────────────────────────── locals ───────────────────────────────────────────┐ │
│ │    conf = {                                                                                  │ │
│ │           │   'packagename_regex': 'spine(toolbox|(db){0,1}[_-][a-z]+)',                     │ │
│ │           │   'pkgname_re': re.compile('spine(toolbox|(db){0,1}[_-][a-z]+)'),                │ │
│ │           │   'repos': {'spinetoolbox': '.', 'spine_items': '../spine-items',                │ │
│ │           'spine_engine': '../spine-engine', 'spinedb_api': '../Spine-Database-API'},        │ │
│ │           │   'dependency_graph': {'spinetoolbox': ['spine_items', 'spine_engine',           │ │
│ │           'spinedb_api'], 'spine_items': ['spinetoolbox', 'spine_engine', 'spinedb_api'],    │ │
│ │           'spine_engine': ['spinedb_api'], 'spinedb_api': []},                               │ │
│ │           │   'default_branch': 'master',                                                    │ │
│ │           │   'branches': {'spinetoolbox': 'release', 'spine_items': 'release',              │ │
│ │           'spine_engine': 'release', 'spinedb_api': 'release'},                              │ │
│ │           │   'workflow': {'repo': 'spine-tools/spine-conductor', 'file':                    │ │
│ │           'test-n-publish.yml'}                                                              │ │
│ │           }                                                                                  │ │
│ │  config = WindowsPath('release.toml')                                                        │ │
│ │     ctx = <click.core.Context object at 0x0000026A2B245E80>                                  │ │
│ │ pkgtags = WindowsPath('pkgtags.json')                                                        │ │
│ └──────────────────────────────────────────────────────────────────────────────────────────────┘ │
│                                                                                                  │
│ spine-conductor\orchestra\publish.py:111 in publish_tags_whls         │
│                                                                                                  │
│   108 │   tags = json.loads(pkgtags.read_text())                                                 │
│   109 │   for pkg, repo_path in CONF["repos"].items():                                           │
│   110 │   │   repo = Repo(repo_path)                                                             │
│ > 111 │   │   push_tags(CONF, pkg, repo, tags[remote_name(repo)])                                │
│   112 │   dispatch_workflow(CONF, pkgtags)                                                       │
│   113                                                                                            │
│                                                                                                  │
│ ┌─────────────────────────────────────────── locals ───────────────────────────────────────────┐ │
│ │      CONF = {                                                                                │ │
│ │             │   'packagename_regex': 'spine(toolbox|(db){0,1}[_-][a-z]+)',                   │ │
│ │             │   'pkgname_re': re.compile('spine(toolbox|(db){0,1}[_-][a-z]+)'),              │ │
│ │             │   'repos': {'spinetoolbox': '.', 'spine_items': '../spine-items',              │ │
│ │             'spine_engine': '../spine-engine', 'spinedb_api': '../Spine-Database-API'},      │ │
│ │             │   'dependency_graph': {'spinetoolbox': ['spine_items', 'spine_engine',         │ │
│ │             'spinedb_api'], 'spine_items': ['spinetoolbox', 'spine_engine', 'spinedb_api'],  │ │
│ │             'spine_engine': ['spinedb_api'], 'spinedb_api': []},                             │ │
│ │             │   'default_branch': 'master',                                                  │ │
│ │             │   'branches': {'spinetoolbox': 'release', 'spine_items': 'release',            │ │
│ │             'spine_engine': 'release', 'spinedb_api': 'release'},                            │ │
│ │             │   'workflow': {'repo': 'spine-tools/spine-conductor', 'file':                  │ │
│ │             'test-n-publish.yml'}                                                            │ │
│ │             }                                                                                │ │
│ │       pkg = 'spine_items'                                                                    │ │
│ │   pkgtags = WindowsPath('pkgtags.json')                                                      │ │
│ │      repo = <git.repo.base.Repo 'spine-items\\.git'>          │ │
│ │ repo_path = '../spine-items'                                                                 │ │
│ │      tags = {                                                                                │ │
│ │             │   'Spine-Toolbox': '0.8.4',                                                    │ │
│ │             │   'spine-engine': '0.24.3',                                                    │ │
│ │             │   'Spine-Database-API': '0.31.5'                                               │ │
│ │             }                                                                                │ │
│ └──────────────────────────────────────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘
KeyError: 'spine-items'

[suvayu]

I think the current behaviour is that you have to repeat the --exclude <pkg> argument also for publish. But you are right in not expecting that, since pkgtags.json clearly specifies which packages to release.

[soininen]

conduct publish --help does not mention the --exclude flag so I though it would pick the packages automatically from pkgtags.json in any case.

[suvayu]

Oh! That's a double bug then!

[soininen]

Does it even make sense to have --exclude in publish? What happens if you exclude packages that you did not exclude when running release?

[suvayu]

I think this isn't actually deliberate. When I fixed #22 I think what I did (I don't remember the details any more :stuck_out_tongue:) was incorporate the --exclude or --only flags into the config file parsing. I think that is why it expects the flags to be repeated because the config file has all packages defined.

You are right of course, the correct behaviour should be,

• no need for any flags
• take the set from pkgtags.json
• verify against config file: for pkg in json: pkg in config
• publish

But the common flags not showing in the help message may still be a bug, but irrelevant for the proper solution for the original problem of this issue.