Windows.Memory.Acquisition - initial setup issues

tosinator commented 5 years ago

First of all - great project! I'm a big fan of your work and I'm very interested to see how this evolves. Thank you for your continued work on awesome open source DFIR projects!

This one is a follow up to #70. That bug report helped me tremendously to understand how memory acquisition works in Velociraptor. When playing around with it, I had to apply a few workarounds to make it work.

I am using latest release, 0.3.5.

"public_path" was created in config automatically, but the folder was not auto generated. When I ran Windows.Utils.DownloadBinaries, I received error messages. After creating the directory, DownloadBinaries ran fine.
Windows.Utils.DownloadBinaries seems to have generated this line in inventory.csv (note no presence of filename): WinPmem,.,,6ca71c13f98eeed8ffc0a9edb8eea787624b90018bd4f64672439af35c3ec820 I have replaced it with: WinPmem,.,winpmem_v3.3.rc2.exe,6ca71c13f98eeed8ffc0a9edb8eea787624b90018bd4f64672439af35c3ec820 Due to this, the download was failing, e.g.:
```
[INFO] 2019-10-12T19:50:15Z http_client: Downloading https://github.com/Velocidex/c-aff4/releases/download/v3.3.rc2/winpmem_v3.3.rc2.exe into /tmp/tmp827506525.exe
[INFO] 2019-10-12T19:50:16Z copy: Copying file from /tmp/tmp827506525.exe into /home/tomas/velociraptor/datastore/public
[INFO] 2019-10-12T19:50:16Z copy: Failed to open /home/tomas/velociraptor/datastore/public for writing: open /home/tomas/velociraptor/datastore/public: is a directory
```
I ran the Windows.Utils.DownloadBinaries after the change and this time it downloaded files. I was also able to run Windows.Memory.Acquisition afterwards.

As a side thing to this, it would be nice to get WinPmem upgraded to v3.3.rc3 in Velociraptor.

Again - very nice tool, thanks for releasing this to the public! Tomas

tosinator commented 5 years ago

Side question: upload of the memory dump from client to server seems to happen in "bursts", with somewhat limited upload speed. Is there a way to adjust this?

scudette commented 5 years ago

Thanks for reporting and great debugging work! This should be fixed by #123

File uploads happen via the normal comms mechanism - the file is chunked into small parts then it gets pushed to the server via POST messages. It is not as efficient as streaming the file and depends on the server load may be limited by the server bandwidth too.

If it suits you, you can directly upload the image from the endpoint to a cloud bucket by using the upload_gcs() VQL plugin - just copy the artifact and use upload_gcs() instead of upload(). You can see how that works in this post: https://medium.com/velociraptor-ir/triage-with-velociraptor-pt-3-d6f63215f579?source=friends_link&sk=193c6b36fd29db6d063907d1ccf7f647

scudette commented 5 years ago

The client in head has much improved network comms now - if should upload the file much faster.

tosinator commented 4 years ago

@scudette Just to confirm: no issues at all with the latest builds, all of these are now fixed. The upload is much faster too. Thank you very much - great job! I did not expect this to be addressed so fast.

scudette commented 4 years ago

Awesome! Thanks for the update.

On Tue, Dec 10, 2019, 12:02 Tomas Šiaulys notifications@github.com wrote:

@scudette https://github.com/scudette Just to confirm: no issues at all with the latest builds, all of these are now fixed. The upload is much faster too. Thank you very much - great job! I did not expect this to be addressed so fast.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Velocidex/velociraptor/issues/122?email_source=notifications&email_token=AA5NRIW7GJBHO7IMLWKADYDQX3E63A5CNFSM4JAGZXA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGLALKQ#issuecomment-563479978, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA5NRIT7EB6ZZKGQ7XDXVSDQX3E63ANCNFSM4JAGZXAQ .

Velocidex / velociraptor

Windows.Memory.Acquisition - initial setup issues #122