Closed PeeBee66 closed 8 months ago
@axelfahy I think you were the one who created the conenctor. Would you mind having a look?
@nino-filigran, sorry, but I have never used this connector.
my bad then! I did look in the wrong place. @daemitus / @Arcelone / @rlynch-ironnet / @maertv I see that you all have done some PRs in the past with this connector. Would any of you mind having a look at this?
Description
Consistent restarting of the backup connector when using AlienVault
Backup connector https://github.com/OpenCTI-Platform/connectors/tree/master/stream/backup-files
INFO Backup processed event 1701520853906-0 in 20231202T124100Z / relationship--fc89d0ab-3968-4f05-8437-641af617d002 | timestamp=2023-12-17T21:40:39.642305Z name=pycti.connector Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/pycti/connector/opencti_connector_helper.py", line 530, in run self.callback(msg) File "/opt/opencti-connector-backup-files/backup-files.py", line 75, in _process_message data = json.loads(msg.data) ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/init.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 353, in raw_decode obj, end = self.scan_once(s, idx) ^^^^^^^^^^^^^^^^^^^^^^ json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 841 (char 840) Terminated
Environment
Reproducible Steps
Steps to create the smallest reproducible scenario:
Expected Output
I expect the backup feed to keep backing up with out restarting every 10 sec.
i have a number of other opencti instances and feeds backing up and not one other seems to have this issue
Actual Output
The backup connector restarts every 10 sec and shows this error
AV-connectoer log.txt
Additional information
`###------------------- AV CONNECTOR 1 of 2 PORT 4000-----------------------### version: '3' services:
------------------- REDIS SEARCH -----------------------
redis: image: redis:7.2.3 ports:
redisdata:/data
------------------- ELASTIC SEARCH -----------------------
elasticsearch: image: docker.elastic.co/elasticsearch/elasticsearch:8.11.3 volumes:
Comment-out the line below for a cluster of multiple nodes
Uncomment the line below below for a cluster of multiple nodes
- cluster.name=docker-cluster
"9200:9200" restart: always ulimits: memlock: soft: -1 hard: -1 nofile: soft: 65536 hard: 65536
------------------- MINIO -----------------------
minio: image: minio/minio:RELEASE.2023-11-15T20-43-25Z volumes:
"9001:9001" environment: MINIO_ROOT_USER: ${MINIO_ROOT_USER} MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
command: server --address ":9000" --console-address ":9001" /data healthcheck: test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"] interval: 30s timeout: 20s retries: 3 restart: always
------------------- RABITMQ -----------------------
rabbitmq: image: rabbitmq:3.12-management privileged: true environment:
amqpdata:/var/lib/rabbitmq restart: always
------------------- OPENCTI -----------------------
opencti: image: opencti/platform:${OPENCTI_VERSION} environment:
------------------- Worker -----------------------
worker: image: opencti/worker:${OPENCTI_VERSION} environment:
------------------- AV BACKUP -----------------------
c.av-backup-files: image: opencti/connector-backup-files:${OPENCTI_VERSION}
privileged: true
------------------- AV FEED -----------------------
connector-alienvault: image: opencti/connector-alienvault:${OPENCTI_VERSION} environment:
Screenshots (optional)