search

barncastle / CASC-Bruteforcer

Bruteforce tool for Jenkins and Salsa
9 stars 13 forks source link

Jenkins bruteforcing on an NVIDIA GPU (error -9999) #1

Open Marlamin opened 6 years ago

Marlamin commented 6 years ago

Can't seem to bruteforce Jenkins on an NVIDIA GPU.

Some Googling reveals that -9999 == clEnqueueNDRangeKernel (Illegal read or write to a buffer)

Probably an issue in the underlying library, but might be an easy fix if there's a compatibility mode or something. If not, feel free to close.


  GeForce GTX 980 (GPU)

Loading kernel - 15552 hashes. This may take a minute...

Unhandled Exception: Cloo.ComputeException: OpenCL error code detected: -9999.
   at Cloo.ComputeException.ThrowOnError(ComputeErrorCode errorCode)
   at Cloo.ComputeProgram.Build(ICollection`1 devices, String options, ComputeProgramBuildNotifier notify, IntPtr notifyDataPtr)
   at OpenCLlib.OpenCL.SetKernel(String OpenCLBody, String EntryPoint) in D:\Martin\Documents\GitHub\CASC-Bruteforcer\OpenCLlib\OpenCL.cs:line 267
   at OpenCLlib.MultiCL.SetKernel(String Kernel, String Method) in D:\Martin\Documents\GitHub\CASC-Bruteforcer\OpenCLlib\MultiCL.cs:line 30
   at CASCBruteforcer.Bruteforcers.Jenkins96.Run(Int32 m) in D:\Martin\Documents\GitHub\CASC-Bruteforcer\CASCBruteforcer\Bruteforcers\Jenkins96.cs:line 162
   at CASCBruteforcer.Bruteforcers.Jenkins96.Start() in D:\Martin\Documents\GitHub\CASC-Bruteforcer\CASCBruteforcer\Bruteforcers\Jenkins96.cs:line 106
   at CASCBruteforcer.Program.Main(String[] args) in D:\Martin\Documents\GitHub\CASC-Bruteforcer\CASCBruteforcer\Program.cs:line 39```
barncastle commented 6 years ago

It turns out NVIDIA has a 64kb constant memory limit which means a maximum of ~8000 unnamed hashes can be loaded at anyone time. I have a few solutions however I'm not sure the best route to go down:

  1. Move it to global memory which will mean a slightly slower hash rate but will allow for ridiculously large amount of hashes.
  2. Create an exception list of hashes that no one is interested in i.e. beta builds and single occurrence files.
  3. Filter the hashes by filetype. For this file extensions will have to be enforced in templates (and I'll have to write a scraper for your site to get this info unless you can whip something together?).

What do you think?

Marlamin commented 6 years ago

Maybe a combination of 2 and 3. I can add an optional argument to the unknown lookups to only select stuff from the latest build or a specific build/something like that. Should atleast filter out files that have long been removed. And I can definitely whip something together for 3 as well. Although we'll probably have to talk about the format and stuff. Poke me on Discord or IRC in a few hours if you're still around.

On Mon, Jan 29, 2018, 14:54 barncastle notifications@github.com wrote:

It turns out NVIDIA has a 64kb constant memory limit which means a maximum of ~8000 unnamed hashes can be loaded at anyone time. I have a few solutions however I'm not sure the best route to go down:

  1. Move it to global memory which will mean a slightly slower hash rate but will allow for ridiculously large amount of hashes.
  2. Create an exception list of hashes that no one is interested in i.e. beta builds and single occurrence files.
  3. Filter the hashes by filetype. For this file extensions will have to be enforced in templates (and I'll have to write a scraper for your site to get this info unless you can whip something together?).

What do you think?

ā€” You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/barncastle/CASC-Bruteforcer/issues/1#issuecomment-361251979, or mute the thread https://github.com/notifications/unsubscribe-auth/AADEJLZ8RaoknuIn6i8fR4u1YPRnjcpvks5tPc2ugaJpZM4Rv33J .

Marlamin commented 6 years ago

I added a product option to the listfile script like so:

https://bnet.marlam.in/listfile.php?unk=1&product=wow

Supported products are wow, wowt and wow_beta. wow and wowt respectively only return about 5k of unknown hashes currently, while wow_beta is at 14443. Not specifying a product will get you 14606 (which includes old files not available in any branch anymore). I'm expecting wow_beta's count to drop under 10k, though not sure we can make it to 8k just yet.

We'll probably need something along the lines of option 3 as well to get under 10k for wow_beta... maybe I should ignore ogg's by default. šŸ˜›

Marlamin commented 6 years ago

Also added an exclude option just now that lets you exclude files in a comma seperated list like so: https://bnet.marlam.in/listfile.php?unk=1&product=wow_beta&exclude=ogg,unk,mp3

This gets me at 7980 unknown lookups.

item/objectcomponents/head/helm_cloth_zandalardungeon_c_01_%%%%.m2 finds these:

  GeForce GTX 980 (GPU)

Loading kernel - 7980 hashes. This may take a minute...
Starting Jenkins Hashing :: 2313441 combinations
Completed in 0,13 secs
Found 27:
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_HU_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_GN_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_BE_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_GN_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_DR_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_OR_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_DW_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_OR_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_TA_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_ZA_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_TR_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_TR_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_BE_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_GO_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_GO_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_TA_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_PA_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_WO_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_SC_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_DW_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_NI_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_SC_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_DR_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_WO_F.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_NI_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_HU_M.M2
  ITEM\OBJECTCOMPONENTS\HEAD\HELM_CLOTH_ZANDALARDUNGEON_C_01_PA_F.M2
barncastle commented 6 years ago

That's some grand work, I didn't expect the turn around to be so quick! The changes you've made will cover everything that we need to filter these hashes down. The only tweak that would be nice is if the exclude filter could be independent of the product filter e.g. all unnamed hashes from all products except for mp3s - but we can just default the product if not šŸ˜›.

I'll update the code to facilitate this this afternoon, time/work permitted.

Also the first list of successfully found names!

Marlamin commented 6 years ago

I'll have a look and see if I can move the exclusion stuff outside of the product check. Maybe I'll add an include option as well to only include certain types.

Also, I'm not sure if this is an NVIDIA issue again but OpenCL throws an OutOfResources when doing bruteforces with a LOT of wildcards. I get that'll take a while but it probably shouldn't crash while doing it. Maybe some kind of batching is required?