Closed Yusufkulcu closed 2 weeks ago
@jehna please help me
Can you run humanify using --verbose
flag and post the output? It gives a lot more debug data to work with
Can you run humanify using
--verbose
flag and post the output? It gives a lot more debug data to work with
I'm trying right now
@jehna
@jehna
Do you have any suggestions on what I can do to fix the problem?
Hmm. Seems like an issue with some of the dependencies, at least that error does not originate from humanify's code. In hindsight it seems that using pkgroll was a bad idea, as it seems to mess with the dependencies' stack traces. I think this would need some extra debugging to solve.
Hmm. Bazı bağımlılıklarla ilgili bir sorun gibi görünüyor, en azından bu hata humanify'ın kodundan kaynaklanmıyor. Geriye dönüp bakıldığında pkgroll'u kullanmanın kötü bir fikir olduğu anlaşılıyor, çünkü bağımlılıkların yığın izlerini bozuyor gibi görünüyor. Bunun çözülmesi için biraz daha hata ayıklamaya ihtiyaç olduğunu düşünüyorum.
@jehna
So what can I do as a solution? Solving this code is very important to me
How can I send you error details
Whoops, closed this by accident. You could check which part of the bundle throws the error, that would help
Whoops, closed this by accident. You could check which part of the bundle throws the error, that would help
@jehna
When I check, the "transformFromAstAsync" function in the "@babel/core" package gives an error. Since I'm not familiar with the package, I have no idea how to fix the problem. I would be grateful if you could help
@jehna
Please help me
@jehna
Will you help?
@Yusufkulcu thanks for the additional info, but unfortunately I'm not able to respond to bigger work immediately (not being paid for doing ppen source etc)
I think I'll be able to investigate more within a few weeks when I have some time
I understood. I would be very happy if I know when the study is concluded. We hope that it will be concluded as soon as possibleI understood. I would be very happy if I know when the study is concluded. Hoping to conclude it as soon as possible 😊
When I check, the "transformFromAstAsync" function in the "@babel/core" package gives an error. Since I'm not familiar with the package, I have no idea how to fix the problem.
@Yusufkulcu I haven't looked too deeply into this, but I suspect the issue is likely not going to be directly within transformFromAstAsync
within @babel/core
; but more with how it is being used by either humanify
's AST parsing code, or how the webcrack
project it depends on is using it.
Looking up that "Failed to stringify code"
error leads to this source within humanify
:
It looks like it throws that when the result from transformFromAstAsync
has a falsy value for stringified?.code
Depending on your skill at debugging, it might be useful to see what the value of ast
being passed in is, as well as the full value of stringified
that is being returned.
Depending on the sensitivity of the code you're running this on, it could also be useful if you're able to attach the code that consistently triggers this area; even better if you're first able to reduce it to a minimal example that triggers the error; as that would make it a lot easier for others to help debug/understand what might be causing it.
When I check, the "transformFromAstAsync" function in the "@babel/core" package gives an error. Since I'm not familiar with the package, I have no idea how to fix the problem.
@Yusufkulcu I haven't looked too deeply into this, but I suspect the issue is likely not going to be directly within
transformFromAstAsync
within@babel/core
; but more with how it is being used by eitherhumanify
's AST parsing code, or how thewebcrack
project it depends on is using it.Looking up that
"Failed to stringify code"
error leads to this source withinhumanify
:It looks like it throws that when the result from
transformFromAstAsync
has a falsy value forstringified?.code
Depending on your skill at debugging, it might be useful to see what the value of
ast
being passed in is, as well as the full value ofstringified
that is being returned.Depending on the sensitivity of the code you're running this on, it could also be useful if you're able to attach the code that consistently triggers this area; even better if you're first able to reduce it to a minimal example that triggers the error; as that would make it a lot easier for others to help debug/understand what might be causing it.
I didn't understand exactly what I was supposed to do:(
I didn't understand exactly what I was supposed to do:(
@Yusufkulcu At a minimum, if you are able to attach the code you're trying to decrypt, that would be really useful to help narrow things down.
Tam olarak ne yapmam gerektiğini anlamadım :(
@YusufkulcuEn azından, şifresini çözmeye çalıştığınız kodu ekleyebilirseniz, bu, işleri daraltmanıza yardımcı olmak için gerçekten yararlı olacaktır.
Yes, you are right, I have attached the file below. Github I uploaded it to my Google drive account because it doesn't allow javascript file uploading.
https://drive.google.com/file/d/1H9_g6L32xMchrpi-dIRksz7MyQkmWm2a/view?usp=drivesdk
@jehna @0xdevalias
I uploaded it to my Google drive account because it doesn't allow javascript file uploading.
@Yusufkulcu Thanks. If you just rename the .js file to .txt that would usually let you upload it directly to GitHub as well.
Javascript dosya yüklemeye izin vermediği için Google Drive hesabıma yükledim.
@YusufkulcuTeşekkürler. .js dosyasının adını .txt olarak değiştirirseniz genellikle onu doğrudan GitHub'a da yükleyebilirsiniz.
I uploaded it to my Google drive account because it doesn't allow javascript file uploading.
@Yusufkulcu Thanks. If you just rename the .js file to .txt that would usually let you upload it directly to GitHub as well.
I uploaded it to my Google drive account because it doesn't allow javascript file uploading.
@Yusufkulcu Thanks. If you just rename the .js file to .txt that would usually let you upload it directly to GitHub as well.
Thanks for the info. I reloaded it as txt. I will be waiting for your help. It's an important project for me main.txt
Thanks for the info. I reloaded it as txt.
@Yusufkulcu Thanks for that. I ran it through the web version of and it seemed to unpack correctly there, which makes it a higher likelihood that the issue is within
humanify`:
Depending on your use case, I wonder if you might be able to use that unpacked/unmangled version as a starting point for your analysis; even though it doesn't have the 'nice variable renames' that humanify
adds on top? You might also be able to run humanify
against individual files from that extracted version in the interim as well.
@Yusufkulcu Looking at your screenshot from https://github.com/jehna/humanify/issues/111#issuecomment-2365764329, it looks like the error is occuring on file 18:
Looking at the code where this happens, it seems to be within unminify
, just after webcrack
has run to extract the files, and in the main loop where those files are being processed:
Since even with --verbose
enabled, humanify
doesn't output the filenames it's processing, it makes it a bit harder to debug; but running it through a debugger we can see what 'file 18' of extractedFiles
corresponds with anyway (assuming webcrack
consistently unpacks the code the same way); which since the output log shows i + 1
, that makes it actually entry 17 in the array (because it's 0-indexed); which seems to be the extracted file 1.js
:
Looking at the contents of output/1.js
, we can see that it's an empty file:
⇒ cat output/1.js | wc -m
0
As part of the main unminify
loop, the code then calls plugins.reduce
on the plugins
passed in to unminify
:
When run as humanify openai
, these plugins are:
The Failed to stringify code
error you're seeing comes from the visitAllIdentifiers
function, just after calling babel
's transformFromAstAsync
:
Which is called by the openaiRename
plugin that is passed to unminify
:
transformFromAstAsync
accepts a parsed ast
, and returns an object with code
, map
, and ast
.
Presumably, with an empty file as input, that would result in an empty ast, which when passed to transformFromAstAsync
, would result in an empty string returned in the code
field.
Given the check that ends up throwing the error you're seeing is looking for a falsy value in stringified.code
; the empty string would definitely trigger that:
@Yusufkulcu @jehna So the above seems to be the debugging as to the 'why' of what is going on; though I'm not familiar enough with the codebase to suggest where the best point of implementing the fix is.
If I was to suggest the most minimal/localised fix, it would be to the if (!stringified?.code)
check within visitAllIdentifiers
(Ref)
Though an argument could also be made that filtering out empty/similar files at the unminify
level could have it's merits (Ref); though that would then make it less generic/flexible, as theoretically a plugin in the chain could make it a non-empty file before the openaiRename
actually needs to process it.
So I still think the best fix is probably within visitAllIdentifiers
.
In fact, in looking through related issues, this just seems to be a variation of the same root cause that I raised in this issue:
I also think the general CLI output should be improved to explain what is happening better. eg. a message before running webcrack
, outputting the actual filenames (not just the file number), etc.
These issues also seem to tangentially relate to some aspect of this latter suggestion:
Thanks for the info. I reloaded it as txt.
@Yusufkulcu Thanks for that. I ran it through the web version of
and it seemed to unpack correctly there, which makes it a higher likelihood that the issue is within
humanify`:Depending on your use case, I wonder if you might be able to use that unpacked/unmangled version as a starting point for your analysis; even though it doesn't have the 'nice variable renames' that
humanify
adds on top? You might also be able to runhumanify
against individual files from that extracted version in the interim as well.
Thanks for the suggested solution. It works largely without "beautifying variable" values. But my goal is to change it completely. It seems to take a long time to decode the files in the "outputs" folder one by one with Humanify. Is there a shortcut to this?
It seems to take a long time to decode the files in the "outputs" folder one by one with Humanify. Is there a shortcut to this?
@Yusufkulcu Do you mean that the process takes a long time in general? Or that having to manually run it against each file takes a long time?
If the latter, you could probably extract them with webcrack
, then write a bash script/similar that runs humanify
on each of those files individually. I'm not 100% sure how the renames works across module boundaries/for exports/similar.. but I think it already reads/parses each of the split files individually anyway, so I don't think it would be much worse to run it this way..
Actually.. it will probably still be a bit slower because of the startup boilerplate, and then trying to run webcrack
against the file each time (even though they are already split)..
While it's not currently a feature, if you're going to need to be re-running the variable renaming process on future versions of the same code, this feature request would probably also be relevant to you:
@jehna It might be a neat/useful feature to be able to run humanify
against a source file (or multiple files/directory of files/etc) while bypassing the webcrack
step; that way it could be used on files that had already been unpacked/etc with webcrack
(or another tool like wakaru
) previously.
Edit: I split this last part into its own issue:
It seems to take a long time to decode the files in the "outputs" folder one by one with Humanify. Is there a shortcut to this?
@Yusufkulcu Do you mean that the process takes a long time in general? Or that having to manually run it against each file takes a long time?
If the latter, you could probably extract them with
webcrack
, then write a bash script/similar that runshumanify
on each of those files individually. I'm not 100% sure how the renames works across module boundaries/for exports/similar.. but I think it already reads/parses each of the split files individually anyway, so I don't think it would be much worse to run it this way..Actually.. it will probably still be a bit slower because of the startup boilerplate, and then trying to run
webcrack
against the file each time (even though they are already split)..While it's not currently a feature, if you're going to need to be re-running the variable renaming process on future versions of the same code, this feature request would probably also be relevant to you:
@jehna It might be a neat/useful feature to be able to run
humanify
against a source file (or multiple files/directory of files/etc) while bypassing thewebcrack
step; that way it could be used on files that had already been unpacked/etc withwebcrack
(or another tool likewakaru
) previously.Edit: Split this last part into its own issue:
Thank you for your detailed explanation. I'm going to try the parsed source code separately by writing a script that implements the humanify command. I have one more question.
What is the reason why the "Webcrack" package breaks the code into pieces. I divide it into many files in the Outputs folder and I don't know what they are. I wonder if you can explain that. Thank you in advance
Thank you for your detailed explanation. I'm going to try the parsed source code separately by writing a script that implements the humanify command.
@Yusufkulcu It might actually be quicker for you to just clone the humanify
repo and temporarily patch the code to skip trying to run against empty files.
I haven't tried it, but a quick/dirty hack fix could just be to add an if
check to skip empty files within unminify
:
Something like:
export async function unminify(
filename: string,
outputDir: string,
plugins: ((code: string) => Promise<string>)[] = []
) {
ensureFileExists(filename);
const bundledCode = await fs.readFile(filename, "utf-8");
const extractedFiles = await webcrack(bundledCode, outputDir);
for (let i = 0; i < extractedFiles.length; i++) {
console.log(`Processing file ${i + 1}/${extractedFiles.length}`);
const file = extractedFiles[i];
const code = await fs.readFile(file.path, "utf-8");
+ if (code) {
const formattedCode = await plugins.reduce(
(p, next) => p.then(next),
Promise.resolve(code)
);
verbose.log("Input: ", code);
verbose.log("Output: ", formattedCode);
await fs.writeFile(file.path, formattedCode);
+ }
+ else {
+ verbose.log(`Skipping empty file ${i + 1} (${file.path})`);
+ }
}
}
While this wouldn't really be suited for a proper fix, it should hopefully be enough to do what you need to work around this bug in the meantime.
What is the reason why the "Webcrack" package breaks the code into pieces. I divide it into many files in the Outputs folder and I don't know what they are. I wonder if you can explain that.
@Yusufkulcu The original source file you provided is a webpack bundle of code (or similar), where the original many code files have been combined into a single file. When webcrack
runs on it, it 'undoes' this bundling, and so splits the single file back into many files similar to how it was originally.
ensureFileExists(filename); const bundledCode = await fs.readFile(filename, "utf-8"); const extractedFiles = await webcrack(bundledCode, outputDir); for (let i = 0; i < extractedFiles.length; i++) { console.log(`Processing file ${i + 1}/${extractedFiles.length}`); const file = extractedFiles[i]; const code = await fs.readFile(file.path, "utf-8"); + if (code) { const formattedCode = await plugins.reduce( (p, next) => p.then(next), Promise.resolve(code) ); verbose.log("Input: ", code); verbose.log("Output: ", formattedCode); await fs.writeFile(file.path, formattedCode); + } + else { + verbose.log(`Skipping empty file ${i + 1} (${file.path})`); + } }
Thanks for the suggestion, it doesn't give that error anymore, but now I'm getting another error. I get the following error in file 63. I've added the original file and the file where I got the error below. It's a pretty long mistake, so I added some more pictures.
Thanks for the suggestion, it doesn't give that error anymore, but now I'm getting another error. I get the following error in file 63. I've added the original file and the file where I got the error below. It's a pretty long mistake, so I added some more pictures.
..snip..
..snip..
@Yusufkulcu I'm glad that workaround helped get around the first issue :)
SyntaxError: Unexpected token, expected "," (74:70) 72 | }); 73 | }; 74 | subscribeHandler.prototype._subscribe = function (_subscribeHandler.syncErrorThrown) { | ^ 75 | var sourceObject = this.source; 76 | return sourceObject && sourceObject.subscribe(_subscribeHandler.syncErrorThrown); 77 | };
That new error you're running into seems to now be the same/similar to the errors being discussed in this issue:
You might like to try the potential solution I described in this comment, RE: cloning the humanify
repo and bumping the webcrack
version in package.json
to the latest version (2.14.1
):
For context/continuity, @j4k0xb has raised 2 bugfix PR's for the bugs identified in this issue:
I trade using chatgpt. I am getting the error shown in the screenshot. The file size is large (7mb on average) and the number of lines is high (134881 lines). What is the problem? Can you help me? This is very important to me