Question | Rekall installation / support

[jarvisjarb]

As I understand this, you (as in the GitHub GRR) no longer support rekall.

1. Does this mean AnalyzeClientMemory Functionality is no longer available?
2. After installing GRR Server and running the GRR Client on the target workstation is there somethng regarding Rekall that I have to manually install myself?

[grrrrrrrrr]

The functionality is for the moment still present in GRR, you just need to enable it in the config - for example during the initial setup - and will be able to use AnalyzeClientMemory. However, if you run into any issues, we will not be able to help you fix them.

Re 2: You do not need to manually install anything. If you have the GRR client running and Rekall support enabled you should be able to use it right away with no additional setup.

I'll close this issue as it's not really an issue.

[jarvisjarb]

Thats a shame. One of the best feature of this platform is the remote memory analysis. I'm unable to make this work at the moment. When trying to run pslist plugin it is 'unrecognized' by the server..

[mbushkov]

For the record: GRR does have a remote memory analysis capability - with YARA.

Cheers, Misha

On Wed, Jun 13, 2018 at 8:56 PM, jarvisjarb notifications@github.com wrote:

Thats a sham. One of the best feature of this platform is the remote memory analysis. I'm unable to make this work at the moment. When trying to run pslist plugin it is 'unrecognized' by the server..

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/google/grr/issues/593#issuecomment-397048057, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3PFa0QLJ3gJPS2CsUb3xYaw9lupDIPks5t8WBLgaJpZM4UcR3Z .

[jarvisjarb]

UPDATE: (For those who face the same problem as I do) To fix the error mentioned above I have: 1) Downloaded the rekall profile repository to the server machine. 2) Served it with SimpleHTTPServer on port 1212 3) Edited the /etc/grr/server.local.yaml file and added Rekall.profile_repository: 'http://localhost:1212 4) Ran grr_config_updater download_missing_rekall_profiles

This seems to solve the issue with profiles not being recognized. I still face problems with: 1) Rekall on Windows7 x86 workstations since Winpmem causes BSOD. 2) Rekall on WIndows7 x64 workstations with error in rekall modules generate_memory_ranges (..) collect AttributeError: 'Module object has no attribute 'Deduplicate'

Bottom line Rekall does not work (yet) on my installation machine.

[jarvisjarb]

LAST UPDATE:

You can close this topic. I have stopped working on making Rekall work with / without grr since Rekall is no logger supported by the Grr-Team , and there are serious problems with Rekall on by itself.

If someone does read this and succeeds with deploying Rekall+GRR, ping me..

Good luck.

[grrrrrrrrr]

Sorry for your bad experience with Rekall. The kind of problems you encountered here is essentially why we decided to stop supporting Rekall in GRR.

The main issue is that in today's OSs, memory layout changes super quickly. If you look at the Rekall issue tracker, there is a small bugfix every day in response to some OS change. Now, Rekall has this philosophy of moving fast, breaking things and then fixing all bugs super quickly (but to be fair, all those OS changes can't be predicted and would break the algorithms from time to time anyways). This approach works ok when you have a memory image on your machine to analyze. All you need to do is a pip upgrade, your bug will be fixed, and you can continue your analysis.

With GRR, it's a different story. In order to fix a bug in Rekall, you need to get the new version to all your clients somehow. So, if there is a new small issue every day, you will spend all your time updating GRR clients which is completely impractical.

If you also consider that memory acquisition is inherently dangerous - see the BSOD you got - and that the results even when Rekall works are not very stable, we decided to not offer those capabilities anymore and instead only provide signature search in process memory and process memory dumping. I hope this explains a bit the reasoning behind that decision. If you have any specific use case that you would like to solve with Rekall but can't with GRR, please let us know, maybe we can provide some alternative.

[scudette]

I would suggest you just package Rekall as its own binary inside GRR and launch it as a separate process. We are doing the same with osquery and it should work for Rekall as well.

Can I ask what kind of analysis you want to run with Rekall? Do you want to collect images? Do you want to do live analysis (Either using the API or Memory analysis?).

[jarvisjarb]

Can I ask what kind of analysis you want to run with Rekall? Do you want to collect images? Do you want to do live analysis (Either using the API or Memory analysis?).

The idea was to run live analysis - psxview, netscan, filescan, handles, hooks_iat, hooks_inline, ldrdrll, malfind on large scale and forward it to ELK. Doing this side by side with GRR current functionalities would have been perfect.

[scudette]

This kind of shotgun approach is not very effective. Dont get me wrong - I mean memory analysis is still a very powerful technique. We recently were working a case where the bad guys used Cobalt Strike. All the script kiddies are using it now and it makes techniques which were considered very advanced a few years ago very easy. Take a look for example: https://www.cobaltstrike.com/help-malleable-c2 under the heading "In-memory Evasion and Obfuscation" you will see that it specifically defeats malfind like plugins by doing simple things like freeing injected memory after execution is finished or dll stomping which does not have the typical RWX signature that malfind is looking for. The bad guys are actively aware of the techniques memory analysis uses and are quite capable of defeating them - then they sell their tools to script kiddies and everyone is suddenly l33t :-).

Most attacks tend to happen purely in memory and use powershell a lot (so filescan and handles are very useful). For example psxview is pretty useless now because people dont need to unlink lists in kernel space any more to hide processes (process injection makes hiding in plain sight more effective anyway - your process may be called explorer.exe but you are running the CS beacon inside it). The hooks* modules are very expensive to run and really easy to bypass and also produce a ton of false positives - so they make it hard to run at scale (they are ok for image analysis and sometimes reveal useful stuff).

On the other hand with the appearance of script kiddie tools like CS we tend to get way better success with yara sigs. Because most of these tools end up using parts of metasploit's shellcode generators to some extent (its really hard to write small tight shell code) this makes a very effective signature (also add the base64 version of the shellcode as a sign to get the initial exploit vector). With yara sigs it is more reliable and effective to just scan memory using debug privs rather than memory analysis because then you dont need to deal with paging etc and profile support is better. Tools like CS just piece together shellcode blocks to make the payload so any block can trigger a sig - and the sig may appear anywhere in memory too (in the heap when the payload is decoded/decrypted or in VADs when it is injected).

So I guess I am saying in the end it is a tradeoff between stability and efficiency with endpoint monitoring and the benefits that memory analysis gives. In practical terms memory analysis at scale is hard to make work and not that effective - and costs a lot in resource load on the client. I certainly would not be doing it to just collect data for baseline analysis - I would be using the API provided info for that (e.g. WMI, process yara scanning etc). If you want to get more sophisticated then execution tracing is very effective (e.g. Carbon Black style tracing).

Memory analysis is very useful to have in order to acquire an image if you know there is something wrong on the endpoint. An image is absolutely invaluable to prove what was going on but not so much as detection. I would just run something line winpmem or linpmem on the endpoint to grab the image on demand. You can pipe a memory image to a cloud bucket without touching the disk if you want using the latest winpmem for example (providing you have the bandwidth for that to make sense). Alternatively you can just launch winpmem to grab a local image and then use GRR to grab it slowly later.