Unify smbprotocol deps and bump to latest

seanstory commented 1 month ago

Closes https://github.com/elastic/connectors/issues/2851

This bumps our smbprotocol dependency to 1.10.1. "Latest" is 1.14.0, but seems to have made some backwards-incompatible changes.

It also moves this to the framework requirements file, as it was the same dep being used between aarch64 and x86-64, which doesn't make sense to keep separate.

Checklists

Pre-Review Checklist

[x] this PR does NOT contain credentials of any kind, such as API keys or username/passwords (double check config.yml.example)
[x] this PR has a meaningful title
[x] this PR links to all relevant github issues that it fixes or partially addresses
[x] this PR has a thorough description
[x] Covered the changes with automated tests
[ ] Tested the changes locally
[x] Added a label for each target release version (example: v7.13.2, v7.14.0, v8.0.0)

Release note

Fixes a bug where Network Drive connectors might have issues connecting to SMB 3.1.1 shares

seanstory commented 1 month ago

updating to 1.14.0 doesn't seem to have worked:


2024-09-25 18:10:06 UTC | [FMWK][18:10:06][ERROR] [Connector id: network_drive, index name: search-network_drive, Sync job id: q9ZgKpIBzFqNd8iuMb9b] Extractor failed with an error: 'SMBDirEntryInformation' object has no attribute 'fields'
-- | --
  | 2024-09-25 18:10:06 UTC | [FMWK][18:10:06][CRITICAL] [Connector id: network_drive, index name: search-network_drive, Sync job id: q9ZgKpIBzFqNd8iuMb9b] Document extractor failed
  | 2024-09-25 18:10:06 UTC | Traceback (most recent call last):
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/es/sink.py", line 490, in run
  | 2024-09-25 18:10:06 UTC | await self.get_docs(generator)
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/es/sink.py", line 542, in get_docs
  | 2024-09-25 18:10:06 UTC | async for count, doc in aenumerate(generator):
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/utils.py", line 856, in aenumerate
  | 2024-09-25 18:10:06 UTC | async for elem in asequence:
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/logger.py", line 244, in __anext__
  | 2024-09-25 18:10:06 UTC | return await self.gen.__anext__()
  | 2024-09-25 18:10:06 UTC | ^^^^^^^^^^^^^^^^^^^^^^^^^^
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/es/sink.py", line 524, in _decorate_with_metrics_span
  | 2024-09-25 18:10:06 UTC | async for doc in generator:
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/sync_job_runner.py", line 457, in prepare_docs
  | 2024-09-25 18:10:06 UTC | async for doc, lazy_download, operation in self.generator():
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/sync_job_runner.py", line 493, in generator
  | 2024-09-25 18:10:06 UTC | async for doc, lazy_download in self.data_provider.get_docs(
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/sources/network_drive.py", line 897, in get_docs
  | 2024-09-25 18:10:06 UTC | async for document in self.traverse_diretory(
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/utils.py", line 594, in wrapped
  | 2024-09-25 18:10:06 UTC | raise e
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/utils.py", line 589, in wrapped
  | 2024-09-25 18:10:06 UTC | async for item in func(*args, **kwargs):
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/sources/network_drive.py", line 471, in traverse_diretory
  | 2024-09-25 18:10:06 UTC | yield self.format_document(file=file)
  | 2024-09-25 18:10:06 UTC | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | 2024-09-25 18:10:06 UTC | File "/opt/buildkite-agent/builds/bk-agent-prod-gcp-1727287414931206720/elastic/connectors/connectors/sources/network_drive.py", line 428, in format_document
  | 2024-09-25 18:10:06 UTC | file_details = file._dir_info.fields
  | 2024-09-25 18:10:06 UTC | ^^^^^^^^^^^^^^^^^^^^^
  | 2024-09-25 18:10:06 UTC | AttributeError: 'SMBDirEntryInformation' object has no attribute 'fields'

seanstory commented 1 month ago

This seems to be hitting some ftest flakeyness that won't go away. But the same branch on nightly didn't have issues with those ftests. :weird: https://buildkite.com/elastic/connectors-nightly/builds/2539

github-actions[bot] commented 1 month ago

💚 Backport PR(s) successfully created

Status	Branch	Result
✅	8.x	https://github.com/elastic/connectors/pull/2855
✅	8.15	https://github.com/elastic/connectors/pull/2856

The backport PRs will be merged automatically after passing CI.

elastic / connectors