search

karthikkbala / MISP-QRadar-Integration

The Project can be used to integrate QRadar with MISP Threat Sharing Platform
37 stars 29 forks source link

Could not POST IOCs to QRadar (Failure) #9

Open siddarthtalupula opened 2 years ago

siddarthtalupula commented 2 years ago

Hello, I am trying to integrate MISP with our QRadar instance. GET of feeds from MISP is successful, however, the POST to QRadar is failing. Please advise.

[user@fkqc4 MISP-QRadar-Integration]# sudo python3 integration.py /usr/local/lib/python3.6/site-packages/apscheduler/util.py:94: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new t ime zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html if obj.zone == 'local': /usr/local/lib/python3.6/site-packages/apscheduler/util.py:165: PytzUsageWarning: The localize method is no longer necessary, as this time zone supports the f old attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migra tion.html return tz.localize(datetime_, is_dst=None) /usr/local/lib/python3.6/site-packages/apscheduler/triggers/interval.py:66: PytzUsageWarning: The normalize method is no longer necessary, as this time zone s upports the fold attribute (PEP 495). For more details on migrating to a PEP 495-compliant implementation, see https://pytz-deprecation-shim.readthedocs.io/en /latest/migration.html return self.timezone.normalize(next_fire_time) 14:28:45 -- Checking HTTPS Connectivity to QRadar 14:28:45 -- (Success) HTTPS Connectivity to QRadar 14:28:45 -- Checking HTTPS Connectivity to MISP 14:28:45 -- (Success) HTTPS Connectivity to MISP 14:28:45 -- Validating if reference set MISP_Event_IOC exists 14:28:45 -- Validating reference set MISP_Event_IOC - (Success) 14:28:45 -- Identifying Reference set MISP_Event_IOC element type 14:28:45 -- Reference set element type = ALN (Success) 14:28:45 -- Initiating, GET data from MISP on 10.127.2.22 14:28:59 -- MISP API Query (Success) 14:28:59 -- 283558 IOCs imported 14:28:59 -- Initiating, IOC POST to QRadar 14:29:00 -- Could not POST IOCs to QRadar (Failure)

14:29:00 -- Could not POST IOCs to QRadar (Failure) ^CTraceback (most recent call last): File "integration.py", line 121, in scheduler.start() File "/usr/local/lib/python3.6/site-packages/apscheduler/schedulers/blocking.py", line 19, in start self._main_loop() File "/usr/local/lib/python3.6/site-packages/apscheduler/schedulers/blocking.py", line 28, in _main_loop self._event.wait(wait_seconds) File "/usr/lib64/python3.6/threading.py", line 551, in wait signaled = self._cond.wait(timeout) File "/usr/lib64/python3.6/threading.py", line 299, in wait gotit = waiter.acquire(True, timeout) KeyboardInterrupt

Thanks in advance Siddarth

michal0000000 commented 1 year ago

Hi, did you manage to resolve this? Stumbled on the same thing.. Thanks!

michal0000000 commented 1 year ago

Kind of resolved this issue:

Upon further investigation of the /var/log/qradar.error file I found the following Postgres error:

Dec 30 11:01:42 ::ffff:IP_ADDRESS [tomcat.tomcat] [admin@IP_ADDRESS (7024) /console/JSON-RPC/QRadar.addToReferenceSet QRadar.addToReferenceSet] org.apache.openjpa.lib.jdbc.ReportingSQLException: Batch entry 0 select * from refdata_insert_reference_data_element(13, ?, 'admin') as result was aborted: ERROR: insert or update on table "reference_data_element" violates foreign key constraint "reference_data_element_fkey"

Basically what happens is that for some reason, QRadar doesn't insert data into "reference_data_key" after some point. This causes an error when inserting data into "reference_data_element" table, since the referenced foreign key doesn't exist.

Although I haven't found the reason for this behavior, I found a "fix" that makes insertion into given Reference Set possible again:

  1. psql -U qradar
  2. Find out the missing id (from the error message), as well as the Reference Set id you're working with (PK in table reference_data)
  3. Add entry to reference_data_element table: INSERT INTO reference_data_key (id,rd_id,domain_info) VALUES (13,27,<get_val_from_other_entries>);
  4. \q to quit Postgres

After this, insertion should work again. Hope this helps.