search

HashLips / hashlips_art_engine

HashLips Art Engine is a tool used to create multiple different instances of artworks based on provided layers.
MIT License
7.16k stars 4.3k forks source link

_metadata.json isnt being created #1594

Open dmlaj68 opened 4 months ago

dmlaj68 commented 4 months ago

Hey I'm having an issue whereby the image and individual json files are being created however at the end of the generation it isnt creating the _metadata.json file

I've read another thread that suggested to use their version of update_info.js but that didn't work It could be that my collection is large and its a size error as I generated 500,000 but im new to doing this so I thought I'd ask the experts

when I've ran a smaller test collection it has worked fine.

the error is as folllows using the original update_info.js;

Created edition: 171239, with DNA: b64d0c4b1d6cc18ca1c7cabf09a4851046f83221 C:\SBS\src\main.js:429 writeMetaData(JSON.stringify(metadataList, null, 2)); ^

RangeError: Invalid string length at JSON.stringify () at startCreating (C:\SBS\src\main.js:429:22) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

The error when using the update_info.js from @bolshoytoster gave this error;

C:\SBS\utils\update_info.js:55 JSON.stringify(data, null, 2) ^

RangeError: Invalid string length at JSON.stringify () at Object. (C:\SBS\utils\update_info.js:55:8) at Module._compile (node:internal/modules/cjs/loader:1369:14) at Module._extensions..js (node:internal/modules/cjs/loader:1427:10) at Module.load (node:internal/modules/cjs/loader:1206:32) at Module._load (node:internal/modules/cjs/loader:1022:12) at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:135:12) at node:internal/main/run_main_module:28:49

any help would be greatly appreciated

bolshoytoster commented 4 months ago

@dmlaj68 the RangeError: Invalid string length message isn't very accurate, it means that you're running out of memory creating the _metadata.json file. Make sure you have more memory when you create your collection. If you don't want to have to regenerate, you could probably create _metadata.json from the existing metadata files.

dmlaj68 commented 4 months ago

@dmlaj68 the RangeError: Invalid string length message isn't very accurate, it means that you're running out of memory creating the _metadata.json file. Make sure you have more memory when you create your collection. If you don't want to have to regenerate, you could probably create _metadata.json from the other metadata files.

how would i go about doing that? thanks for your reply

dmlaj68 commented 4 months ago

on this particular run the machine has 16gb I also ran a much slower generation on a VPS which had 120gb Ram lol and that threw the same error

dmlaj68 commented 4 months ago

@dmlaj68 the RangeError: Invalid string length message isn't very accurate, it means that you're running out of memory creating the _metadata.json file. Make sure you have more memory when you create your collection. If you don't want to have to regenerate, you could probably create _metadata.json from the existing metadata files.

Is there any other solution you can think of When running the operation the memory doesn't go above 72% usage and as above the VPS I tried on has 120gb ram so that definitely shouldn't run out

bolshoytoster commented 4 months ago

@dmlaj68 maybe your collection is just too big for node to stringify at once. You could try using my fork, which creates the json as a stream. (my fork is also much faster).

dmlaj68 commented 4 months ago

@dmlaj68 maybe your collection is just too big for node to stringify at once. You could try using my fork, which creates the json as a stream. (my fork is also much faster).

just tried your suggestion but im getting an error when running node index.js to run the generation

PS C:\hashlips_art_engine-main> node index.js node:internal/modules/cjs/loader:1146 throw err; ^

Error: Cannot find module 'JSONStream' Require stack:

its setup the same as with the OG files so not sure what the issue is, the files is says are missing are definitely there

bolshoytoster commented 4 months ago

@dmlaj68 have you tried running npm i again?