Integration in LOKI

[Neo23x0]

Hi @hasherezade,

Great tool! I'd like to integrate PE-Sieve in my LOKI scanner. https://github.com/Neo23x0/Loki

I have already tested an integration (PE-Sieve started as sub process, processing the output) and failed due to the PAUSE that is integrated after the output summary.

I would like to use your releases and don't want to fork the repo.

Could you make the pause optional or remove it completely in your compiled releases?

https://github.com/hasherezade/pe-sieve/pull/2

[Neo23x0]

Ah, and please make the dumps optional

e.g. /nodump or /dump

https://github.com/hasherezade/pe-sieve/pull/3

[hasherezade]

Hi! I am happy that you like it and it's cool that you want to integrate it :) Regarding #2 - no problem, I was planning to do it anyways in my first official release, but I can do it right away. However, I will prefer to keep the pause when the project is built in the debug mode. Regarding #3 - I want the dump to be made by default, so rather I will go for an option /nodump

[hasherezade]

Please check the latest updates and let me know.

[Neo23x0]

Ah, great! Yes, perfect. Did you update the pre-compiled binaries? I'll check asap.

[hasherezade]

Yes, I just updated.

[Neo23x0]

Great - it works!

This will be really useful.

[hasherezade]

Great! Please let me know after all the tests if everything works fine. And if you need anything more, you can always ping me.

[hasherezade]

BTW - if you use it strictly to detect malicious processes, I must emphasize on this: "Some in-memory patches are done by legitimate applications, so, in the future version I will provide capability of whitelisting defined patches." (https://hshrzd.wordpress.com/pe-sieve/). Especially WOW64 processes will have some patched DLLs and this is normal.

[Neo23x0]

Ok, I did some testing ... and crashed PE-Sieve. Sorry.

I run some 32bit applications in user context on a Win7 x64.

Notepad++ Console-2.00b148-Beta_32bit.zip link

[hasherezade]

I checked this app (md5=4ec4a7d78ff127fcef4809f0d8ba9780) on Windows7 64bit and it was not crashing PE-sieve. Can you send me some more info about the crash itself? As far as I can see on your console, it was not exactly a PE-sieve crash, but the process was killed due to timeout. Am I right? Can you run the PE-sieve as standalone, scan the same process and show me the log from PE-sieve, so that I will know on which part of processing it takes too much time? Eventually, I will send you later a version built with more debug strings, so that we will see more.

[Neo23x0]

I have uploaded a crash dump of PE-Sieve running on one of the processes.

https://keybase.pub/johngalt/Share/AppCrash_pe-sieve64.exe.zip

Both crashes happen while (or after) scanning "normaliz.DLL". (VT link)

To your questions: I added a timeout that prevents PE-Sieve from running endlessly. It is not the timeout that kills the process. As a workaround I would also kill all processes named "WerFault.exe" to avoid users seeing the message windows for too long but maybe we can figure out the reason for these crashes.

[Neo23x0]

Ah and regarding your note:

BTW - if you use it strictly to detect malicious processes, I must emphasize on this: "Some in-memory patches are done by legitimate applications, so, in the future version I will provide capability of whitelisting defined patches."

I totally understand. LOKI just prints "warnings" and "notices" to mark elements as malicious, suspicious or noteworthy. I would start with LOKI's "notice" level messages for processes with "hooked" and "suspicious" findings and LOKI's "warning" level messages for processes where "replaced" is not "0".

Hooked != 0 > Notice Suspicious != 0 > Notice Replaced != 0 > Warning

Do you think this is adequate?

[hasherezade]

Thank you for the valuable feedback! I reproduced the crash and it will be fixed today.

[Neo23x0]

Ah, awesome!

[Neo23x0]

Have you played with the "Releases" here in github?

https://github.com/hasherezade/pe-sieve/releases

You could draft a new "release" of the compiled binaries there. When drafting a release you attach both EXE files in a ZIP (or maybe the two EXE files).

This way I could integrate the PE-Sieve executables in the update process, because the link to the "latest" release would never change. This way the loki-upgrader.exe could always download the latest versions of your compiled executables and they would always stay up-to-date.

^ but this is optional - otherwise I would integrate the most recent files manually

[hasherezade]

No worries, I know what are the github "Releases" ;) If you see my other projects you will find that I use them. The reason why I don' use it with PE-sieve is that, the current builds are just a preview of the project before the official release (https://twitter.com/hasherezade/status/946035029439860737). BTW - I already found and fixed the bug that caused crashing - so the new build will be ready in few minutes. And just a side note - the links to the google drive are also static.

[hasherezade]

New builds uploaded! Please check if it solves the problem.

[Neo23x0]

Great. it works like a charm.

I'll continue with the integration.

[hasherezade]

Great! I am happy to hear :) Also, I added a new option /quiet - I thought you may find it useful

[Neo23x0]

Ok, almost done.

One last thing: Is it possible to suppress the creation of that sub directory for each analyzed process?

[hasherezade]

Yes, sure. I may include it as a part of the /quiet mode

[hasherezade]

Ok, done. Please check the latest build.

[Neo23x0]

Done - I've created a new release v0.26, include references in README and code, included your license in the ./docs folder and published a tweet.

Thanks so much for your tool and help! I think that in combination with LOKI many people will be able to run your tool on thousands of end systems scanning millions of running processes for anomalies. A small step for us - one giant leap for the community.

[hasherezade]

Great. Please remember that PE-sieve is a very young project and I had no time to make proper testing, so I can give no guarantee that all will always work exactly as intended. If any problem ever pops up, please let me know immediately and I will fix. But for sure integrating it with LOKI gives opportunity to expose PE-sieve to wide range of cases and it will help me also to make it better.

[hasherezade]

I was testing the latest LOKI today and it works with PE-sieve pretty well, I run some recent malware samples and I got them immediately detected, i.e. QuantLoader: TrickBot:

In my opinion, if the module withing a process is found replaced it should be reported as "alert" not "warning" - because this will almost always indicate malware. Fix me if I am wrong, but I don't think any legitimate application replaces a loaded module in the memory... Another idea that came in my mind is that re-running PE-sieve for each process is a huge waste of time - rather than this, I can just export the scanning function. Then, you can load PE-sieve just once, and then call this function on each process.

[Neo23x0]

I have seen a lot of weird software, but you are right. I'll change it to "ALERT".

Then, you can load PE-sieve just once, and then call this function on each process.

I have never tried that before, but I think, that I can do that, yes.

[hasherezade]

From now PE-sieve can be also build as a DLL: https://github.com/hasherezade/pe-sieve/commit/3ac22f4650963044a721161a89ef97a3f5b4c333 The next thing that I am gonna do soon, is writing Python bindings, so that you can easily use the API from LOKI. If you have any remarks about the API please let me know.

[Neo23x0]

Ah, great! I would have tried to use cytpes to load and call the function from Python.

https://stackoverflow.com/questions/252417/how-can-i-use-a-dll-file-from-python

[hasherezade]

OK, I made some experimental builds. Please check:

• DLL 64bit
• DLL 32bit And this is how it works, my sample scanner in C++: https://gist.github.com/hasherezade/69160abc5ae4ca6798fd228e01a0a378

[Neo23x0]

Oh, there's a snag. I compile loki.exe with Pyinstaller as a 32bit application as it makes no difference during file system scans. The problem is that from that 32bit executable I cannot load the pe-sieve64.dll. I would have to reorganize my build environment with an additional 64bit Python version, to build a loki64.exe, which is able to make use of the pe-sieve64.dll.

[hasherezade]

ok, I have another idea to solve this. I can make an EXE that would be either 32 or 64 bit, importing PE-sieve as a library. It will be communicating with LOKI over the pipe. LOKI would be sending it PID, and it will scan the process and send it back the results. By this way, we will have a separate process, but it will be deployed only once. What do you think?

[Neo23x0]

Ah, yes, awesome. I haven't thought about that.

[dnides]

First, super cool project!!!! Is the DLL posted above still the latest release? I also want to play with using in Python if possible.

[hasherezade]

@dnides - thank you, but please make a new issue if you want to discuss something else than LOKI. you can also catch me on twitter, my DM is open.