search

airbytehq / PyAirbyte

PyAirbyte brings the power of Airbyte to every Python developer.
https://docs.airbyte.com/pyairbyte
Other
231 stars 41 forks source link

Something went wrong in the connector - "source-github" #399

Open obaidmuneer opened 1 month ago

obaidmuneer commented 1 month ago 
import airbyte as ab
source = ab.get_source(
    "source-github",
    install_if_missing=True,
    config={
        "repositories": ["airbytehq/quickstarts"],
        "credentials": {
            "personal_access_token": ab.get_secret("GITHUB_PERSONAL_ACCESS_TOKEN"),
        },
    },
)
source.check()
source.set_streams(["pull_requests", "issues", "reviews", "stargazers"])
cache = ab.get_default_cache()
result = source.read(cache=cache)

error:

Writing PyAirbyte logs to file: C:\Users\OBAIDM~1\AppData\Local\Temp\airbyte\logs\2024-09-27\airbyte-log-J8SKMYTPJ.log
Writing `source-github` logs to file: C:\Users\OBAIDM~1\AppData\Local\Temp\airbyte\logs\source-github\source-github-log-J8SKMYTQ6.log
Connection check succeeded for `source-github`.
C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\github.py:17: DeprecationWarning: The 'set_streams' method is deprecated and will be removed in a future version. Please use the 'select_streams' method instead.
  source.set_streams(["pull_requests", "issues", "reviews", "stargazers"])
ERROR: Something went wrong in the connector. See the logs for more details.
Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\github.py", line 17, in <module>
    source.set_streams(["pull_requests", "issues", "reviews", "stargazers"])
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\sources\base.py", line 99, in set_streams
    self.select_streams(streams)
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\sources\base.py", line 147, in select_streams
    available_streams = self.get_available_streams()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\sources\base.py", line 219, in get_available_streams
    return [s.name for s in self.discovered_catalog.streams]
                            ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\sources\base.py", line 325, in discovered_catalog
    self._discovered_catalog = self._discover()
                               ^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\sources\base.py", line 209, in _discover
    for msg in self._execute(["discover", "--config", config_file]):
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\site-packages\airbyte\_connector_base.py", line 440, in _execute
    raise exc.AirbyteConnectorFailedError(
airbyte.exceptions.AirbyteConnectorFailedError: Connector failed. (AirbyteConnectorFailedError)
------------------------------------------------------------
AirbyteConnectorFailedError: Connector failed.
    Please review the log file for more information.
    Connector Name: 'source-github'
    Exit Code: 1
    Log file: C:\Users\OBAIDM~1\AppData\Local\Temp\airbyte\logs\source-github\source-github-log-J8SKMYTQ6.log

would be apricated for help

obaidmuneer commented 1 month ago 
Log file: C:\Users\OBAIDM~1\AppData\Local\Temp\airbyte\logs\source-github\source-github-log-J8SKMYTQ6.log
2024-09-27 16:25:25 - INFO - Check succeeded
2024-09-27 16:25:28 - INFO - [WinError 267] The directory name is invalid: 'C:\\Users\\OBAIDM~1\\AppData\\Local\\Temp\\tmpuvxwz6tc\\comments.sqlite'
Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 252, in __subject__
    return self.cache
           ^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 176, in __getattribute__
    return _oga(self, attr)
           ^^^^^^^^^^^^^^^^
AttributeError: 'JsonRef' object has no attribute 'cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\jsonref.py", line 178, in callback
    base_doc = self.loader(uri)
               ^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 33, in __call__
    with open(uri) as f:
         ^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: ''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\entrypoint.py", line 117, in run
    yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.discover(source_spec, config))
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\entrypoint.py", line 152, in discover
    catalog = self.source.discover(self.logger, config)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\abstract_source.py", line 74, in discover
    streams = [stream.as_airbyte_stream() for stream in self.streams(config=config)]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\abstract_source.py", line 74, in <listcomp>
    streams = [stream.as_airbyte_stream() for stream in self.streams(config=config)]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\streams\core.py", line 286, in as_airbyte_stream
    json_schema=dict(self.get_json_schema()),
                     ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\streams\core.py", line 281, in get_json_schema
    return ResourceSchemaLoader(package_name_from_class(self.__class__)).get_schema(self.name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 140, in get_schema
    return self._resolve_schema_references(raw_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 156, in _resolve_schema_references
    resolved = resolve_ref_links(resolved)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 58, in resolve_ref_links
    return {k: resolve_ref_links(v) for k, v in obj.items()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 58, in <dictcomp>
    return {k: resolve_ref_links(v) for k, v in obj.items()}
               ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 58, in resolve_ref_links
    return {k: resolve_ref_links(v) for k, v in obj.items()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 58, in <dictcomp>
    return {k: resolve_ref_links(v) for k, v in obj.items()}
               ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\sources\utils\schema_helpers.py", line 49, in resolve_ref_links
    obj = resolve_ref_links(obj.__subject__)
                            ^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 176, in __getattribute__
    return _oga(self, attr)
           ^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 254, in __subject__
    self.cache = super(LazyProxy, self).__subject__
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 240, in __subject__
    return self.callback()
           ^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\jsonref.py", line 180, in callback
    self._error("%s: %s" % (e.__class__.__name__, unicode(e)), cause=e)
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\jsonref.py", line 223, in _error
    raise JsonRefError(
jsonref.JsonRefError: FileNotFoundError: [Errno 2] No such file or directory: ''

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 620, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\OBAIDM~1\\AppData\\Local\\Temp\\tmpuvxwz6tc\\comments.sqlite'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 878, in onerror
    _os.unlink(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\OBAIDM~1\\AppData\\Local\\Temp\\tmpuvxwz6tc\\comments.sqlite'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Scripts\source-github.exe\__main__.py", line 7, in <module>
    sys.exit(run())
             ^^^^^
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\source_github\run.py", line 17, in run
    launch(source, sys.argv[1:])
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\entrypoint.py", line 234, in launch
    for message in source_entrypoint.run(parsed_args):
  File "C:\Users\Obaid Muneer\Desktop\prosp\ai-sdk-internal-knowledge-base\backend_py\.venv-source-github\Lib\site-packages\airbyte_cdk\entrypoint.py", line 99, in run
    with tempfile.TemporaryDirectory() as temp_dir:
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 904, in __exit__
    self.cleanup()
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 908, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 622, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 881, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors)
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 603, in _rmtree_unsafe
    onerror(os.scandir, path, sys.exc_info())
  File "C:\Users\Obaid Muneer\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 600, in _rmtree_unsafe
    with os.scandir(path) as scandir_it:
         ^^^^^^^^^^^^^^^^
NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\OBAIDM~1\\AppData\\Local\\Temp\\tmpuvxwz6tc\\comments.sqlite'
2024-09-27 16:25:28 - ERROR - Something went wrong in the connector. See the logs for more details.
2024-09-27 16:25:28 - INFO - {"type":"TRACE","trace":{"type":"ERROR","emitted_at":1727436328748.815,"error":{"message":"Something went wrong in the connector. See the logs for more details.","internal_message":"[WinError 267] The directory name is invalid: 'C:\\\\Users\\\\OBAIDM~1\\\\AppData\\\\Local\\\\Temp\\\\tmpuvxwz6tc\\\\comments.sqlite'","stack_trace":"Traceback (most recent call last):\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 252, in __subject__\n    return self.cache\n           ^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 176, in __getattribute__\n    return _oga(self, attr)\n           ^^^^^^^^^^^^^^^^\nAttributeError: 'JsonRef' object has no attribute 'cache'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\jsonref.py\", line 178, in callback\n    base_doc = self.loader(uri)\n               ^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 33, in __call__\n    with open(uri) as f:\n         ^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory: ''\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\entrypoint.py\", line 117, in run\n    yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.discover(source_spec, config))\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\entrypoint.py\", line 152, in discover\n    catalog = self.source.discover(self.logger, config)\n              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\abstract_source.py\", line 74, in discover\n    streams = [stream.as_airbyte_stream() for stream in self.streams(config=config)]\n              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\abstract_source.py\", line 74, in <listcomp>\n    streams = [stream.as_airbyte_stream() for stream in self.streams(config=config)]\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\streams\\core.py\", line 286, in as_airbyte_stream\n    json_schema=dict(self.get_json_schema()),\n                     ^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\streams\\core.py\", line 281, in get_json_schema\n    return ResourceSchemaLoader(package_name_from_class(self.__class__)).get_schema(self.name)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 140, in get_schema\n    return self._resolve_schema_references(raw_schema)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 156, in _resolve_schema_references\n    resolved = resolve_ref_links(resolved)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 58, in resolve_ref_links\n    return {k: resolve_ref_links(v) for k, v in obj.items()}\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 58, in <dictcomp>\n    return {k: resolve_ref_links(v) for k, v in obj.items()}\n               ^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 58, in resolve_ref_links\n    return {k: resolve_ref_links(v) for k, v in obj.items()}\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 58, in <dictcomp>\n    return {k: resolve_ref_links(v) for k, v in obj.items()}\n               ^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\sources\\utils\\schema_helpers.py\", line 49, in resolve_ref_links\n    obj = resolve_ref_links(obj.__subject__)\n                            ^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 176, in __getattribute__\n    return _oga(self, attr)\n           ^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 134, in wrapper\n    return method(self, *args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 254, in __subject__\n    self.cache = super(LazyProxy, self).__subject__\n                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 134, in wrapper\n    return method(self, *args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 240, in __subject__\n    return self.callback()\n           ^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 134, in wrapper\n    return method(self, *args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\jsonref.py\", line 180, in callback\n    self._error(\"%s: %s\" % (e.__class__.__name__, unicode(e)), cause=e)\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\proxytypes.py\", line 134, in wrapper\n    return method(self, *args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\jsonref.py\", line 223, in _error\n    raise JsonRefError(\njsonref.JsonRefError: FileNotFoundError: [Errno 2] No such file or directory: ''\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 620, in _rmtree_unsafe\n    os.unlink(fullname)\nPermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\OBAIDM~1\\\\AppData\\\\Local\\\\Temp\\\\tmpuvxwz6tc\\\\comments.sqlite'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 878, in onerror\n    _os.unlink(path)\nPermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\\\Users\\\\OBAIDM~1\\\\AppData\\\\Local\\\\Temp\\\\tmpuvxwz6tc\\\\comments.sqlite'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"<frozen runpy>\", line 198, in _run_module_as_main\n  File \"<frozen runpy>\", line 88, in _run_code\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Scripts\\source-github.exe\\__main__.py\", line 7, in <module>\n    sys.exit(run())\n             ^^^^^\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\source_github\\run.py\", line 17, in run\n    launch(source, sys.argv[1:])\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\entrypoint.py\", line 234, in launch\n    for message in source_entrypoint.run(parsed_args):\n  File \"C:\\Users\\Obaid Muneer\\Desktop\\prosp\\ai-sdk-internal-knowledge-base\\backend_py\\.venv-source-github\\Lib\\site-packages\\airbyte_cdk\\entrypoint.py\", line 99, in run\n    with tempfile.TemporaryDirectory() as temp_dir:\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 904, in __exit__\n    self.cleanup()\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 908, in cleanup\n    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 890, in _rmtree\n    _shutil.rmtree(name, onerror=onerror)\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 759, in rmtree\n    return _rmtree_unsafe(path, onerror)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 622, in _rmtree_unsafe\n    onerror(os.unlink, fullname, sys.exc_info())\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 881, in onerror\n    cls._rmtree(path, ignore_errors=ignore_errors)\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\tempfile.py\", line 890, in _rmtree\n    _shutil.rmtree(name, onerror=onerror)\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 759, in rmtree\n    return _rmtree_unsafe(path, onerror)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 603, in _rmtree_unsafe\n    onerror(os.scandir, path, sys.exc_info())\n  File \"C:\\Users\\Obaid Muneer\\AppData\\Local\\Programs\\Python\\Python311\\Lib\\shutil.py\", line 600, in _rmtree_unsafe\n    with os.scandir(path) as scandir_it:\n         ^^^^^^^^^^^^^^^^\nNotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\\\Users\\\\OBAIDM~1\\\\AppData\\\\Local\\\\Temp\\\\tmpuvxwz6tc\\\\comments.sqlite'\n","failure_type":"system_error","stream_descriptor":null}}}
aaronsteers commented 1 month ago

Hi, @obaidmuneer . Thanks for raising this! I have not seen this issue before but I do see this error which is probably related to root cause:

NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\OBAIDM~1\\AppData\\Local\\Temp\\tmpuvxwz6tc\\comments.sqlite'

This appears to be originating from the source connector itself (source-github) and is likely related to some python cross-OS compatibility issues for how Windows treats temp files.

If you have access to WSL or a linux box, it is likely this would run correctly on one of those platforms. If this is the correct root cause, a "full" fix likely would require an update to the source itself.

We do try to keep Windows working well for PyAirbyte, and we have a robust test matrix covering Windows as well as Linux, but the connectors themselves sometimes will have OS-level compatibility issues.

Let me know if this sounds right, and if you have any other environments you can test from.