Error on syncing `issues` from GitLab repo #53

Closed pnadolny13 closed 2 years ago

In GitLab by @HassanS6000 on Jun 16, 2021, 16:51

Config:

{
  "start_date": "2000-01-01T00:00:00.000Z",
  "api_url": "https://gitlab.com/api/v4",
  "private_token": "*******",
  "projects": "HassanS6000/****"
}

Stacktrace:

CRITICAL 'NoneType' object has no attribute 'metadata'
Traceback (most recent call last):
  File "/home/envs/gitlab/bin/tap-gitlab", line 8, in <module>
    sys.exit(main())
  File "/home/envs/gitlab/lib64/python3.7/site-packages/tap_gitlab/__init__.py", line 858, in main
    raise exc
  File "/home/envs/gitlab/lib64/python3.7/site-packages/tap_gitlab/__init__.py", line 855, in main
    main_impl()
  File "/home/envs/gitlab/lib64/python3.7/site-packages/tap_gitlab/__init__.py", line 850, in main_impl
    do_sync()
  File "/home/envs/gitlab/lib64/python3.7/site-packages/tap_gitlab/__init__.py", line 810, in do_sync
    sync_project(pid)
  File "/home/envs/gitlab/lib64/python3.7/site-packages/tap_gitlab/__init__.py", line 718, in sync_project
    mdata = metadata.to_map(stream.metadata)
AttributeError: 'NoneType' object has no attribute 'metadata'

In GitLab by @HassanS6000 on Jun 16, 2021, 16:52

changed the description

In GitLab by @HassanS6000 on Jun 16, 2021, 16:59

changed the description

In GitLab by @DouweM on Jun 16, 2021, 17:14

@HassanS6000 This suggests that CATALOG.get_stream("branches") returns None here: https://gitlab.com/meltano/tap-gitlab/-/blob/master/tap_gitlab/__init__.py#L310.

Are you doing anything special with the catalog?

In GitLab by @DouweM on Jun 16, 2021, 17:14

assigned to @aaronsteers

In GitLab by @HassanS6000 on Jun 16, 2021, 17:16

@DouweM To save space I think once users select relevant streams we don't provide the rest of the catalog to the tap. Seems like a similar assumption in this tap as the Singer SDK issue I filed a while back: https://gitlab.com/meltano/sdk/-/issues/105

In GitLab by @DouweM on Jun 16, 2021, 17:25

@HassanS6000 I agree it shouldn't be necessary for every stream to be in the catalog. If it's not there, the tap should just treat it as deselected. Would you be up for submitting a patch?

In GitLab by @hsyyid on Jun 17, 2021, 10:25

mentioned in merge request !40

In GitLab by @DouweM on Jun 17, 2021, 11:02

assigned to @HassanS6000 and unassigned @aaronsteers

In GitLab by @aaronsteers on Jul 19, 2021, 13:29

mentioned in commit b8d45d1c3068579799c35a01dd2717d2b9393de0

MeltanoLabs / tap-gitlab

Error on syncing `issues` from GitLab repo #53