search

OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
6.24k stars 922 forks source link

Deploy the NLP connector to SaaS users #5652

Open nino-filigran opened 8 months ago

nino-filigran commented 8 months ago

Goal of the task is to make the solution avaialble to SaaS users.

iFrozenPhoenix commented 5 months ago

@Jipegien I think the best and easiest way for you to leverage NER (Named Entity Recognition) to extract entities and relations out of unstructured texts is to use AI integration (OpenAI or your custom trained Mistral model). I already use such an approach before ingestion. Before I used this approach I tried it with well-known python frameworks with medium good results. After moving to NER with an AI model (First tried OpenAI, then Mistral; stayed at mistral because OpenAI gets to expensive if you run this on random web scraped content) the results were pretty good. Hope it helps a little.

Jipegien commented 5 months ago

Yes we are considering this possibility, but in a context of multiple users, it comes rapidly with huge costs and latencies. Results are often not so consistent over time. Thanks :)