Error check SSL - microsoftgraph

[fpsilva-source]

Hello everybody, Can you help me. I enabled the option misp_verifycert = True and when running the script it generates the error below regarding the certificate.

[Rafiot]

Are you using a self-signed certificate on the server? It is either that, or the certificate cannot be validated with your local certificate directory because it is too recent (or your local certificate directory is too old).

[fpsilva-source]

@Rafiot

Yes, I generated a temp by letsencrypt. I'm integrating MISP with Sentinel using Graph API that Microsoft releases. The error only happens when I change the misp_verifycert option to True.

[Rafiot]

yes, if you pass False to misp_verifycert, it disables the check, so you won't get that error. But we use lets encrypt with MISP (and PyMISP) , and it works.

Based on the code above, you seem to pass an IP instead of a domain. For the TLS check validation to pass, you need to use the domain.

[fpsilva-source]

@Rafiot

The code above was just an example to show the parameter. In the original configuration is the domain. What do you suggest so I can fix this and enable SSL validation? Can you accept me on linkedin?

Thank you very much in advance.

[Rafiot]

I suggest you to run curl -vvv https://<your.misp.domain> from the machine you're running the script and paste the response in this issue, it will help understand what is going on with your TLS certificate, and probably why python-requests/PyMISP doesn't want to validate the certificate.

Note that you didn't mask the domain completely in the first screenshot, and this one is unreachable so I cannot test myself with curl.

I do not handle PyMISP related issues on LinkedIn so please keep using this channel. If you need private debugging sessions, please refer to the professional services.

[fpsilva-source]

Mu laboratory was off and now it is already on.

URL: https://misphmg.procaci.com/users/login

[Rafiot]

Okay, so that's an odd one: it seems that Letsencrypt changed something in their certificate chain and it's not working on my machine either. But it works fine in the browsers (at least Firefox and Chrome, as they have their own certificate list).

I just extracted the pem from my browser (unzip that file to get it: misphmg-procaci-com-chain.zip), and this command works: curl -vvv --cacert ~/Downloads/misphmg-procaci-com-chain.pem https://misphmg.procaci.com

In order to get PyMISP to work, you can pass the path to the pem file to misp_verifycert (misp_verifycert=<path_to_pem>, instead of False), and it should be able to connect. It's not a great solution, but this is the only way to get that to work until the system certificates are updated, sorry for that.

[fpsilva-source]

So this error is related to the Letsencrypt certificate chain and not directly to the MISP and Microsoft Graph API, correct?

[fpsilva-source]

One question, as the MISP service is running on a server different from the API, will I have to download the certificate according to the command sent and point the path to the downloaded certificate locally on the API server?

[Rafiot]

Yes, this error is related to the certificate and for some reason the ca-certificates package is missing something, it has nothing to do with MISP nor Microsoft Graph API.

I'm not sure I understand the 2nd question. You will need to have the pem file on the server you run the PyMISP script from.

[fpsilva-source]

@Rafiot ,

You understood correctly, the question is whether I have the pem file on the host where I run the Microsoft Graph API?

Can I use the command you mentioned to download the certificate to another host?

Commnad: curl -vvv --cacert ~/Downloads/misphmg-procaci-com-chain.pem https://misphmg.procaci.com

[Rafiot]

That command with curl is only to confirm the certificate it working, it does nothing with MISP/PyMISP.

What you need to do in the script that uses PyMISP will be something like that:

misp_connector = PyMISP(url="https://misphmg.procaci.com/", key="your_api_key", ssl="~/Downloads/misphmg-procaci-com-chain.pem")

I have no idea what the Microsoft Graph API is, all I can do is telling you how to get PyMISP to connect to a MISP instance.

[fpsilva-source]

When I talk about Microsoft Graph API, and a script responsible for integrating importing the IOC from the feed to the SIEM, available at this link https://github.com/microsoftgraph/security-api-solutions. In the security-api-solutions/Samples/MISP/ folder, there is the config.py file where I insert parameters related to MISP and Azure AD referring to the application that I registered in my tenant. Then I run the script.py located in this same folder.

Sorry for not understanding, because I'm new to this MISP environment. When you say PyMISP connect to an instance of MISP, are you referring to a specific configuration file?

[Rafiot]

okay, so I looked at the code and the solution for you should simply be to replace False by the path to the pem file on the machine you run the script.

If you want to make it as simple as possible, put the pem file in the same directory as there the script and the config.py file is and replace False by "./misphmg-procaci-com-chain.pem"

[fpsilva-source]

Thanks, I'll test it next week and report back.

[fpsilva-source]

@Rafiot

Hi,

I ran the script fetching feeds from another MISP instance that has the certificate in operation and after waiting for the execution, the error below occurred. Is this error related to the amount of data to be imported? I saw that in this post you comment on this same error, but the person who posted the doubt did not return with the answer.

https://github.com/MISP/PyMISP/issues/766

Can you help me with limiting the page so that I can get all the IOC from the database.

[Rafiot]

When you have an error like that, something is failing on MISP side. For more details, you need to go look at the logfiles of the MISP instance.

In general, it is because the search query it getting too many responses. The solution when you use PyMISP is to paginate with the limit and page keys and iterate until you got all you need. But as you are using an other project, it will have to be implemented on microsoftgraph's side. So I invite you to open an issue in this repository. If you want to implement the fix yourself, running the request in a loop and iterating until you're done will do the trick.

[fpsilva-source]

When you say create a loop would it be using the limit and page keys? Could you help me by showing an example of how this query will be?

[Rafiot]

That's what I mean with iterating until done:

i = 1
while True:
    r = misp.search(controller='events', return_format='json', limit=10, page=i)
    if not r:
        # no more responses, quit loop
        break

    <Do something with the response>

    i += 1

[fpsilva-source]

In this case, do I configure the loop inside the file where I make the call?

graph_auth = { 'tenant': 'xxxxxxxx', 'client_id': 'xxxxxx', 'client_secret': 'xxxxx', } targetProduct = 'Azure Sentinel'

misp_event_filters = { i = 1 while True: r = misp.search(controller='events', return_format='json', limit=10, page=i) if not r:

no more responses, quit loop

    break

i += 1

} action = 'alert' passiveOnly = False days_to_expire = 30 misp_key = 'xxxxxx misp_domain = 'xxxxxxx misp_verifycert = True

[Rafiot]

No, really not. The current code of microsoftgraph doesn't supports pagination at all (that's why I invited you to open an issue in their repository), so changing the config won't solve your issue.

You, or the maintainer of this code, will need to go edit this file: https://github.com/microsoftgraph/security-api-solutions/blob/master/Samples/MISP/script.py, and change the _get_events method so it iterates over the responses (and probably aggregates the events in one single list).

[fpsilva-source]

@Rafiot

Okay, I'll let the project manager know.

Through your suggestion in another post, I can not get around this problem? If the answer is yes, is this change made in the script.py or config.py file?

misp.search(controller='events', return_format='json', limit=10, page=1)

[Rafiot]

My suggestion in the other post is a piece of pseudo code that will iterate over a paginated query, it will work, but as you can read, it uses misp.search, which is the call used in the _get_event method: https://github.com/microsoftgraph/security-api-solutions/blob/master/Samples/MISP/script.py#L13, not in the config file.

[fpsilva-source]

@Rafiot

I have seen more people with this same problem than mine.

Would it be possible for you to describe how the change in the _get_event method would look so that I can indicate it to the maintainer or maybe even yourself to take credit for the suggested improvement through this link https://github.com/microsoftgraph/security-api-solutions/ blob/master/CONTRIBUTING.md.

As I understand it, without this change I won't be able to import the feeds due to the size of the database, correct? To get around I would have to delimit as your suggestion in the pseudocode?

[Rafiot]

I'll patch the code so it works for you. As you're not a paid customer, I'll work on that on a best-effort basis.

[fpsilva-source]

@Rafiot

Thanks. When the new code is available please let me know.

[Rafiot]

You should still open an issue in the Microsoft repo, they're the maintainers of the code and will probably have an easier (and maybe faster) time than be to fix your problem.

[fpsilva-source]

@Rafiot

Ok, I will notify you now via email, but I would also like to test the correction you will make since doubts in the maintainer's repository take a long time to be answered and in some cases there is no answer.