search

google / grr-doc

Documentation for the GRR Rapid Reponse framework
https://grr-doc.readthedocs.io/
Apache License 2.0
300 stars 154 forks source link

AnalyzeClientMemory Flow Fails Systematically #63

Closed agentX13 closed 7 years ago

agentX13 commented 7 years ago

Hi.

Memory analysis issues. I tried these plugins : arp , pslist, netscan, dns, all without extra arguments. This worked only once, the target was a Win7x64 VM. It has failed everytime since on multiple hosts (physical boxes & VM, Win7x64). So far every non-memory-analysis related features works well.

Here is a debug report from a Win7x64 physical box. Agent installed from GRR_3.1.0.2_amd64.exe

Plugin pslist Client urn aff4:/C.5a5c4e8d5429643a

[DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\inventory.gz [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Logging level set to 10 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Unable to open \.\pmem: (2, 'CreateFile', 'The system cannot find the file specified.') [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Loading driver from c:\windows\system32\grr\3.1.0.2\components\grr-rekall\0.4\resources\WinPmem\winpmem_x64.sys [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Removing service pmem [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] pmem service does not exist. [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Created service pmem [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Running plugin (pslist) with args (()) kwargs ({}) Metadata: session
state
profile
0
1 FileName ept
0
1 IntParser filename
0
1 FileName timezone
0 UTC 1 TimeZone session_name
0
1 String pagefile
0
1 FileName mro GrrRekallSession:Session:object id 327 session_id 2 cookie 329 tool_name rekall plugin_name pslist tool_version 1.5.2.rc1 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Will detect profile using these Detectors: linux_index,nt_index,osx,pe,windows_kernel_file,rsds,ntfs,linux [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\Linux\index.gz [INFO (2016-11-03 19:13:33 UTC) rekall.2] Loaded profile Linux/index from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.018000125885 sec) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\nt\eprocess_index.gz [INFO (2016-11-03 19:13:33 UTC) rekall.2] Loaded profile nt/eprocess_index from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.0460000038147 sec) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\nt\index.gz [INFO (2016-11-03 19:13:33 UTC) rekall.2] Loaded profile nt/index from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.0520000457764 sec) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\OSX\index.gz [INFO (2016-11-03 19:13:33 UTC) rekall.2] Loaded profile OSX/index from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.000999927520752 sec) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\pe.gz [INFO (2016-11-03 19:13:33 UTC) rekall.2] Loaded profile pe from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.0239999294281 sec) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Trying method pe, offset 0 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Trying method windows_kernel_file, offset 0 [INFO (2016-11-03 19:13:33 UTC) rekall.2] Found RSDS in kernel image: 41859C34E0F14EE1B63BDA4607E028162 (ntkrnlmp.pdb) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Skipped profile nt/GUID/41859C34E0F14EE1B63BDA4607E028162 from None (Not in inventory) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Trying method rsds, offset 0 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Trying method linux_index, offset 0 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] LinuxIndexDetector:DetectFromHit(0) = None [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Trying method nt_index, offset 0 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] nt/GUID/A02D90EADA7E4195BD50672CD0A0ABD52 matched offset 0x99b4e+0xfffff80003466000=0xfffff800034ffb4e ('\x90') [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] nt/GUID/A02D90EADA7E4195BD50672CD0A0ABD52 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/F72F5B948181401796C8747CDBD02A0B2 matched offset 0x48aa3f+0xfffff80003466000=0xfffff800038f0a3f ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/F72F5B948181401796C8747CDBD02A0B2 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/80B56B0EBF4A45F8827658E0826FE4ED2 matched offset 0x47d7af+0xfffff80003466000=0xfffff800038e37af ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/80B56B0EBF4A45F8827658E0826FE4ED2 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/437E0D7F5BAD46D2BEDAABBDFBA554A71 matched offset 0x4160bf+0xfffff80003466000=0xfffff8000387c0bf ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/437E0D7F5BAD46D2BEDAABBDFBA554A71 matches 1/11 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/31E8E2145E6F4397B542859183FF79072 matched offset 0x2467c+0xfffff80003466000=0xfffff8000348a67c ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/31E8E2145E6F4397B542859183FF79072 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/706A01493491438C9E1D97ADBB9950C12 matched offset 0x6a43f+0xfffff80003466000=0xfffff800034d043f ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/706A01493491438C9E1D97ADBB9950C12 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/2E3403F0AAF2487FBD3B41763D6DB26F2 matched offset 0x2a6b1c+0xfffff80003466000=0xfffff8000370cb1c ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/2E3403F0AAF2487FBD3B41763D6DB26F2 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/F79084B4F72C4D508F0F4924B2AE188F2 matched offset 0x30fd2f+0xfffff80003466000=0xfffff80003775d2f ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/F79084B4F72C4D508F0F4924B2AE188F2 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/76280CA49D9549AD9030AAB3B2BB97B32 matched offset 0x5503e8+0xfffff80003466000=0xfffff800039b63e8 ('IRP_MN_QUERY_DEVICE_TEXT') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/76280CA49D9549AD9030AAB3B2BB97B32 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/B883868B88CB415A92EC010CF6A115A52 matched offset 0x110a4d+0xfffff80003466000=0xfffff80003576a4d ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/B883868B88CB415A92EC010CF6A115A52 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/6DFDC02F680D41D38E56E1EF5A71F16B2 matched offset 0x19c62+0xfffff80003466000=0xfffff8000347fc62 ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/6DFDC02F680D41D38E56E1EF5A71F16B2 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/156B9EB6B75E403B901BC4E40653F2D82 matched offset 0x17dd9+0xfffff80003466000=0xfffff8000347ddd9 ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/156B9EB6B75E403B901BC4E40653F2D82 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/9722FFCB56A54FCFA86444471E118CE42 matched offset 0x8f16f+0xfffff80003466000=0xfffff800034f516f ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/9722FFCB56A54FCFA86444471E118CE42 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/10165C7EAE314C94B245DBA6C764BD151 matched offset 0x83668+0xfffff80003466000=0xfffff800034e9668 ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/10165C7EAE314C94B245DBA6C764BD151 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/253DE844208F4973B0890F15157027E11 matched offset 0x2f253f+0xfffff80003466000=0xfffff8000375853f ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/253DE844208F4973B0890F15157027E11 matches 1/11 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/100945A3A7F04EDE894A49BD9FEF988B1 matched offset 0x151cef+0xfffff80003466000=0xfffff800035b7cef ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/100945A3A7F04EDE894A49BD9FEF988B1 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/CF6A1D28362E4610946B4EBA29A3CFAE2 matched offset 0x20df+0xfffff80003466000=0xfffff800034680df ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/CF6A1D28362E4610946B4EBA29A3CFAE2 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/37AEA986314C4921ACCFC606BD6FC77C2 matched offset 0x99b4e+0xfffff80003466000=0xfffff800034ffb4e ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/37AEA986314C4921ACCFC606BD6FC77C2 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/8DCA302B311D4D60A7DA738353336B6C1 matched offset 0x648a7+0xfffff80003466000=0xfffff800034ca8a7 ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/8DCA302B311D4D60A7DA738353336B6C1 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] ntdll/GUID/B005A3B66ED64473B02451207DEFC0802 matched offset 0x9d571+0xfffff80003466000=0xfffff80003503571 ('\xcc') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] ntdll/GUID/B005A3B66ED64473B02451207DEFC0802 matches 1/2 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/44C6B918C126440083600BDD67F31BF82 matched offset 0x207e7+0xfffff80003466000=0xfffff800034867e7 ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/44C6B918C126440083600BDD67F31BF82 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] ntdll/GUID/0CB7245D955042C79948F7F767BBA0041 matched offset 0x6be9f+0xfffff80003466000=0xfffff800034d1e9f ('\xcc') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] ntdll/GUID/0CB7245D955042C79948F7F767BBA0041 matches 1/1 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\ntdll\GUID\0CB7245D955042C79948F7F767BBA0041.gz [INFO (2016-11-03 19:13:34 UTC) rekall.2] Loaded profile ntdll/GUID/0CB7245D955042C79948F7F767BBA0041 from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.348999977112 sec) [INFO (2016-11-03 19:13:34 UTC) rekall.2] Detection method nt_index yielded profile <I386 profile ntdll/GUID/0CB7245D955042C79948F7F767BBA0041 (Ntdll)> Table: PPID Thds Hnds Sess Wow64 Start Exit [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Listed 0 processes using PsActiveProcessHead [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Listed 0 processes using CSRSS [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Listed 0 processes using PspCidTable [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Listed 0 processes using Sessions [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Listed 0 processes using Handles EOM

scudette commented 7 years ago

You can see the following debug messages:

[INFO (2016-11-03 19:13:33 UTC) rekall.2] Found RSDS in kernel image: 41859C34E0F14EE1B63BDA4607E028162 (ntkrnlmp.pdb) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Skipped profile nt/GUID/41859C34E0F14EE1B63BDA4607E028162 from None (Not in inventory)

Means that Rekall wants to use the profile 41859C34E0F14EE1B63BDA4607E028162 but you dont have it in your repository. Did you sync recently? GRR copies the profile repository into its own data store - you can sync it using the info in this bug

https://github.com/google/grr/issues/448

Thanks Michael.

On 03/11/2016, agentX13 notifications@github.com wrote:

Hi.

Memory analysis issues. I tried these plugins : arp , pslist, netscan, dns, all without extra arguments. This worked only once, the target was a Win7x64 VM. It has failed everytime since on multiple hosts (physical boxes & VM, Win7x64). So far every non-memory-analysis related features works well.

Here is a debug report from a Win7x64 physical box. Agent installed from GRR_3.1.0.2_amd64.exe

Plugin pslist Client urn aff4:/C.5a5c4e8d5429643a

[DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\inventory.gz [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Logging level set to 10 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Unable to open .\pmem: (2, 'CreateFile', 'The system cannot find the file specified.') [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Loading driver from c:\windows\system32\grr\3.1.0.2\components\grr-rekall\0.4\resources\WinPmem\winpmem_x64.sys [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Removing service pmem [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] pmem service does not exist. [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Created service pmem [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Running plugin (pslist) with args (()) kwargs ({}) Metadata: session
state
profile
0
1 FileName ept
0
1 IntParser filename
0
1 FileName timezone
0 UTC 1 TimeZone session_name
0
1 String pagefile
0
1 FileName mro GrrRekallSession:Session:object id 327 session_id 2 cookie 329 tool_name rekall plugin_name pslist tool_version 1.5.2.rc1 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Will detect profile using these Detectors: linux_index,nt_index,osx,pe,windows_kernel_file,rsds,ntfs,linux [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\Linux\index.gz [INFO (2016-11-03 19:13:33 UTC) rekall.2] Loaded profile Linux/index from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.018000125885 sec) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\nt\eprocess_index.gz [INFO (2016-11-03 19:13:33 UTC) rekall.2] Loaded profile nt/eprocess_index from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.0460000038147 sec) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\nt\index.gz [INFO (2016-11-03 19:13:33 UTC) rekall.2] Loaded profile nt/index from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.0520000457764 sec) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\OSX\index.gz [INFO (2016-11-03 19:13:33 UTC) rekall.2] Loaded profile OSX/index from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.000999927520752 sec) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\pe.gz [INFO (2016-11-03 19:13:33 UTC) rekall.2] Loaded profile pe from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.0239999294281 sec) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Trying method pe, offset 0 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Trying method windows_kernel_file, offset 0 [INFO (2016-11-03 19:13:33 UTC) rekall.2] Found RSDS in kernel image: 41859C34E0F14EE1B63BDA4607E028162 (ntkrnlmp.pdb) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Skipped profile nt/GUID/41859C34E0F14EE1B63BDA4607E028162 from None (Not in inventory) [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Trying method rsds, offset 0 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Trying method linux_index, offset 0 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] LinuxIndexDetector:DetectFromHit(0) = None [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] Trying method nt_index, offset 0 [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] nt/GUID/A02D90EADA7E4195BD50672CD0A0ABD52 matched offset 0x99b4e+0xfffff80003466000=0xfffff800034ffb4e ('\x90') [DEBUG (2016-11-03 19:13:33 UTC) rekall.2] nt/GUID/A02D90EADA7E4195BD50672CD0A0ABD52 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/F72F5B948181401796C8747CDBD02A0B2 matched offset 0x48aa3f+0xfffff80003466000=0xfffff800038f0a3f ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/F72F5B948181401796C8747CDBD02A0B2 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/80B56B0EBF4A45F8827658E0826FE4ED2 matched offset 0x47d7af+0xfffff80003466000=0xfffff800038e37af ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/80B56B0EBF4A45F8827658E0826FE4ED2 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/437E0D7F5BAD46D2BEDAABBDFBA554A71 matched offset 0x4160bf+0xfffff80003466000=0xfffff8000387c0bf ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/437E0D7F5BAD46D2BEDAABBDFBA554A71 matches 1/11 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/31E8E2145E6F4397B542859183FF79072 matched offset 0x2467c+0xfffff80003466000=0xfffff8000348a67c ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/31E8E2145E6F4397B542859183FF79072 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/706A01493491438C9E1D97ADBB9950C12 matched offset 0x6a43f+0xfffff80003466000=0xfffff800034d043f ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/706A01493491438C9E1D97ADBB9950C12 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/2E3403F0AAF2487FBD3B41763D6DB26F2 matched offset 0x2a6b1c+0xfffff80003466000=0xfffff8000370cb1c ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/2E3403F0AAF2487FBD3B41763D6DB26F2 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/F79084B4F72C4D508F0F4924B2AE188F2 matched offset 0x30fd2f+0xfffff80003466000=0xfffff80003775d2f ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/F79084B4F72C4D508F0F4924B2AE188F2 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/76280CA49D9549AD9030AAB3B2BB97B32 matched offset 0x5503e8+0xfffff80003466000=0xfffff800039b63e8 ('IRP_MN_QUERY_DEVICE_TEXT') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/76280CA49D9549AD9030AAB3B2BB97B32 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/B883868B88CB415A92EC010CF6A115A52 matched offset 0x110a4d+0xfffff80003466000=0xfffff80003576a4d ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/B883868B88CB415A92EC010CF6A115A52 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/6DFDC02F680D41D38E56E1EF5A71F16B2 matched offset 0x19c62+0xfffff80003466000=0xfffff8000347fc62 ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/6DFDC02F680D41D38E56E1EF5A71F16B2 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/156B9EB6B75E403B901BC4E40653F2D82 matched offset 0x17dd9+0xfffff80003466000=0xfffff8000347ddd9 ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/156B9EB6B75E403B901BC4E40653F2D82 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/9722FFCB56A54FCFA86444471E118CE42 matched offset 0x8f16f+0xfffff80003466000=0xfffff800034f516f ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/9722FFCB56A54FCFA86444471E118CE42 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/10165C7EAE314C94B245DBA6C764BD151 matched offset 0x83668+0xfffff80003466000=0xfffff800034e9668 ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/10165C7EAE314C94B245DBA6C764BD151 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/253DE844208F4973B0890F15157027E11 matched offset 0x2f253f+0xfffff80003466000=0xfffff8000375853f ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/253DE844208F4973B0890F15157027E11 matches 1/11 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/100945A3A7F04EDE894A49BD9FEF988B1 matched offset 0x151cef+0xfffff80003466000=0xfffff800035b7cef ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/100945A3A7F04EDE894A49BD9FEF988B1 matches 1/12 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/CF6A1D28362E4610946B4EBA29A3CFAE2 matched offset 0x20df+0xfffff80003466000=0xfffff800034680df ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/CF6A1D28362E4610946B4EBA29A3CFAE2 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/37AEA986314C4921ACCFC606BD6FC77C2 matched offset 0x99b4e+0xfffff80003466000=0xfffff800034ffb4e ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/37AEA986314C4921ACCFC606BD6FC77C2 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/8DCA302B311D4D60A7DA738353336B6C1 matched offset 0x648a7+0xfffff80003466000=0xfffff800034ca8a7 ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/8DCA302B311D4D60A7DA738353336B6C1 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] ntdll/GUID/B005A3B66ED64473B02451207DEFC0802 matched offset 0x9d571+0xfffff80003466000=0xfffff80003503571 ('\xcc') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] ntdll/GUID/B005A3B66ED64473B02451207DEFC0802 matches 1/2 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/44C6B918C126440083600BDD67F31BF82 matched offset 0x207e7+0xfffff80003466000=0xfffff800034867e7 ('\x90') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] nt/GUID/44C6B918C126440083600BDD67F31BF82 matches 1/13 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] ntdll/GUID/0CB7245D955042C79948F7F767BBA0041 matched offset 0x6be9f+0xfffff80003466000=0xfffff800034d1e9f ('\xcc') [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] ntdll/GUID/0CB7245D955042C79948F7F767BBA0041 matches 1/1 comparison points [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Opened local file C:\Windows\System32\GRR\3.1.0.2\rekall_profiles\v1.0\ntdll\GUID\0CB7245D955042C79948F7F767BBA0041.gz [INFO (2016-11-03 19:13:34 UTC) rekall.2] Loaded profile ntdll/GUID/0CB7245D955042C79948F7F767BBA0041 from Local Cache Directory:C:\Windows\System32\GRR\3.1.0.2\rekall_profiles (in 0.348999977112 sec) [INFO (2016-11-03 19:13:34 UTC) rekall.2] Detection method nt_index yielded profile <I386 profile ntdll/GUID/0CB7245D955042C79948F7F767BBA0041 (Ntdll)> Table: PPID Thds Hnds Sess Wow64 Start Exit [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Listed 0 processes using PsActiveProcessHead [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Listed 0 processes using CSRSS [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Listed 0 processes using PspCidTable [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Listed 0 processes using Sessions [DEBUG (2016-11-03 19:13:34 UTC) rekall.2] Listed 0 processes using Handles EOM

You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/google/grr-doc/issues/63

agentX13 commented 7 years ago

If by sync you mean: " git clone --depth 1 https://github.com/google/rekall-profiles.git " ... well it did download 1.7GB of data and I changed the contents of ~./rekallrc ... but that did not work. So I checked inside the guid.txt file (/home/some_user_name/rekall-profiles/v1.0/src/guid.txt) and mine (41859C34E0F14EE1B63BDA4607E028162) is not in there. An attempt to add it manually failed as the file "build_profile_repo.py" was not found on the system. ( python ~/rekall/tools/profiles/build_profile_repo.py src/guids.txt )

Anyways, unless you have some instructions for me I will just send the GUID by email to rekall-discuss@googlegroups.com .

scudette commented 7 years ago

GRR copies the profile repository into the data store (i.e. mysql or sqlite) so if you git clone it you still need to sync it to your data store. This should happen automatically by the GRR server but if you do not have internet access (for example egress filtering) you will need to manually copy it using the console (just run grr_console):

from grr.lib import rekall_profile_server

rekall_profile_server.GRRRekallProfileServer().GetMissingProfiles()
lodr74 commented 7 years ago

So where does rekall_profile_server.GRRRekallProfileServer().GetMissingProfiles() get the local files from? I looked in the source for https://github.com/google/grr/blob/6ee1c18e64ccb03c24d1a4c1140563ca16b613ae/grr/lib/rekall_profile_server.py

and it isnt explicit. Basically if we create a CentOS profile and we want to submit the JSON and Kernel Object file + update the guid.txt what directory do we put our GIT clone into ? I see in the code it is expecting a zip file file, so I am assuming this is the gzip file for the JSON. Yet it is not clear how we structure the data to get imported. We fully intend to add to the Rekall profiles, but if we are doing a hunt then we will need to update quickly and not be able to wait for it to be updated publicly.

scudette commented 7 years ago

The rekall profile server is just a proxy for the repository served at Rekall.profile_repository config parameter: https://github.com/google/grr/blob/6ee1c18e64ccb03c24d1a4c1140563ca16b613ae/grr/lib/rekall_profile_server.py#L91

You can set up your own repository by cloning the public one (and serve it over http:// or even file:// url).

You can then add profiles to your own repository when you want. Please be aware that the repository is managed using the rekall manage_repo plugins. New debug builds (i.e. the output of calling make in the rekal/tools/linux directory) are added to the v1.0/src directory and the manage_repo plugin generates the json file and most importantly updates the index. Without the index it is not possible for Rekall to automatically choose the correct profile and since you can not specify it for each of the clients it will probably fail.

destijl commented 7 years ago

Relevant documentation here:

https://github.com/google/grr-doc/blob/a01a68eaf6cf069f573353a0cba7187674157e82/troubleshooting.adoc#missing-rekall-profiles

As mentioned there, you can avoid serving the git repo itself if you have a machine that can run the GRR console (i.e. connect to the DB) and has internet access. It will put the profiles into the DB for you directly.