[10 minute read]
This repo used to be called jalagar/Generative_Gif_Engine but because it now supports GIF, MP4, it was renamed to jalagar/animated-art-engine. v3.0.0 is the beginning of the animated era.
Check out this Youtube Tutorial on how it works!
This python and node app generates layered-based gifs/MP4 to create animated NFT art! It is faster, simpler, and produces higher quality gifs/MP4s than any other open source animated generative tool out there. It also contains many more features including but not limited to stacking layers, if-then, ETH/Solana/Tezos, preview images, inserting legendaries, gifs/MP4, batching to support hundreds of layers, and multiprocessing.
Export your animation as a png image sequence, organize your layer folders with rarity, and the code does the rest! I plan to actively maintain this repo and enhance it with various tools for months to come so be sure to ask questions in the discussion and write issues.
There are three steps:
Checkout this Medium post and How does it work? for more information!
Here's an example final result (or you can download the code and run it and see more bouncing balls :)). It is also pushed to production on OpenSea.
EDIT tool now supports z-index/stacking, grouping, if-then statements, and incompatibilities. See this section for more information. Here is an example of having one layer that is both in front and behind the ball.
See below for some examples of other artists and devs using this tool for their own collections! Feel free to reach out to me or create a PR with your examples!
Dominic Overdrive NFT
Install an IDE of your preference. Recomended
Install Node.js v16.14.2. In order to install a specific version of node, you can use nvm
. On Mac you can brew install nvm
, on Windows you can follow these instructions https://www.freecodecamp.org/news/nvm-for-windows-how-to-download-and-install-node-version-manager-in-windows-10/. Then run nvm install 16.14.2
and nvm use 16.14.2
Run this command on your system terminal to check if node is installed:
node -v
Install the latest version of Python 3. I am currently using 3.8.1 but anything above 3.6 should work.
Run this command on your system terminal to check if node is installed:
python3 --version
If you want to output gifs then:
Install gifski. I recommend using brew brew install gifski
if you're on Mac OSX. If you don't have brew you can install it using brew on Mac OSX. Or if you're on Windows
you can install it using Chocolatey: choco install gifski
.
If you're on Linux, some people were having issues with gifski
so you can skip installing it. You will have to set
the gifTool
config to imageio
instead (see later instructions).
If none of those methods work, follow instructions on gifski gifski Github. Gifski is crucial for this tool because it provides the best gif generation out of all the tools I checked out (PIL, imageio, ImageMagic, js libraries).
If you want to output MP4s then:
Install ffmpeg. I recommend using brew brew install ffmpeg
if you're on Mac OSX. If you don't have brew you can install it using brew on Mac OSX. Or if you're on Windows
you can install it using Chocolatey: choco install ffmpeg
.
If you plan on developing on this repository, run pre-commit
to install pre-commit hooks.
If you're on Windows you can optionally install Make by running choco install make
. Make is already pre-installed on Mac.
Run this command on your root folder using the terminal:
make first_time_setup
If you have any issues with this command, try running each separate command:
python3 -m pip install --upgrade Pillow && pip3 install -r requirements.txt
cd step2_spritesheet_to_generative_sheet && npm i
Each environment can be different, so try Google your issues. I'll add a few known issues below:
Known issues:
cd
command might not work on Windows depending on what Terminal you are using. You may have to edit the Makefile
to use CHDIR
or the equivalent.choco install make
or follow these instructions. Otherwise you can copy and paste the instructions manually in Makefile
.Makefile
and replace python3 with python. Thank you!Load the png or gif files into the /layers
folder where each layer is a folder, and each folder contains
another attribute folder which contains the individual frames and a rarity percentage. For example if you wanted
a background layer you would have /layers/background/blue#20
and /layers/background/red#20
.
In each attribute folder, the frames should be named 0.png
-> X.png
or 0.gif
. See code or step 1 for folder structure. The code
will handle any number of layers, so you could have a layer with two frames, another layer with one frame, and another with 20 frames,
and as long as you pass numberOfFrames
= 20, then the layers will be repeated until they hit 20.
EDIT You can leave the frame names whatever you want, and set useFileNumbering
to false
. This makes it easier if you have hundreds of frames and don't want to rename each one.
Update global_config.json
with:
'totalSupply'
: total number of gifs/MP4 to generate.'height'
: height of one frame. This should be equal to width. Default is 350 (see [https://docs.opensea.io/docs/metadata-standards#:~:text=We%20recommend%20using%20a%20350%20x%20350%20image](OpenSea recommendation))'width'
: width of one frame. This should be equal to height. Default is 350 (see [https://docs.opensea.io/docs/metadata-standards#:~:text=We%20recommend%20using%20a%20350%20x%20350%20image](OpenSea recommendation))'framesPerSecond'
: number of frames per second. This will not be exact because PIL takes in integer milliseconds per frame
(so 12fps = 83.3ms per frame but rounded to an int = 83ms). This will not be recognizable by the human eye, but worth calling out.'numberOfFrames'
: number of total frames. For example you could have 24 frames, but you want to render it 12fps.'description'
: description to be put in the metadata.'baseUri'
: baseUri to be put in the metadata.'layersFolder'
: this is the folder that you want to use for the layers. The default is layers
, but this allows you to have multiple versions of layers and run them side by side. The current repo has four example folders, layers
, layers_grouping
, layers_if_then
, layers_z_index
which all demonstrate features from nftchef's repo.'quality'
: quality of the output, 1-100.'gifTool'
: pick which gif generation method to use, gifski
or imageio
. Gifski is better overall, but some people were having issues with it on Linux. Also imageio
will work for more pixel art, so if you don't want to download Gifski you can set this to imageio
.'MP4Tool'
: pick which MP4 generation method to use. Only supports ffmpeg
at the moment.'outputType'
: select gif
or mp4
.'useBatches'
: set to true
if you want to take advantage of batching. Otherwise does nothing.'numFramesPerBatch'
: number of frames for each batch. See batching for more information. Only does something if useBatches
is set to true
.'loopGif'
: true
if you want to loop the gif, false
if you don't want to loop it.'useMultiprocessing'
: true
if you want to use multi-processing which will speed up step1 and step3. You can configure how many processors to use with processorCount
. Use at your own discretion, I would recommend slowly increase processorCount
and monitor CPU usage, this could crash your computer.'processorCount'
: Number of processors to use with multi-processing. The cap is multiprocessing.cpu_count()
. Use at your own discretion.'useFileNumbering'
: Use 0.png -> X.png numbering or not. If you want to just use the render farm file names, set this to false.'enableAudio'
: BETA FEATURE. You can now add specific audio files per layer. See Add Specific Audio Trait Section for more info.'numLoopMP4'
: Number of times to loop mp4.'generateThumbnail'
: BETA - Flag to generate thumbnail images. thumbnailHeight
and thumbnailWidth
are the corresponding flags. ETH JSON supports image
and animation_url
, image
is thumbnail preview image on the feed, animation_url
is the image to pull from when the user clicks. thumbnailOutputType
is the output type for the thumbnail (you can have gif as thumbnail and mp4 as regular NFT).'generatePFP'
: BETA - Flag to save one individual frame as a PFP using pfpFrameNumber
.'pfpFrameNumber'
: BETA - Select which frame to use, starts at 0 (being the first frame).'animationUri'
: BETA - Animation URI used in the toggle HTML features HTML Animation Toggle.Update step2_spritesheet_to_generative_sheet/src/config.js
with your layerConfigurations
. If you want the basic
configuration, just edit layersOrder
, but if you want to take advantage of nftchef's repo, then scroll through the file for some examples and modify layerConfigurations
accordingly.
To run the process end to end run:
make all
Your output gifs will appear in build/gif
, and your output MP4 will appear in build/mp4
. The ETH JSON will appear in build/json
. Try it yourself with the default settings
and layers!
If you want to switch between generating GIFs vs. MP4, you can change the global_config.json
and just run make step3
.
In order to get nftchef's Generative Gif Engine to work, the input layers needs to be in Sprite Sheet. However this is tedious and unintuitive for many artists who use tools that export individual images.
Step 1 simply converts individual images to spritesheets with the rarity percentage. You provide the various layers in the
/layers
folder with the rarity in the folder name. Each image should be numbered from 0 -> X, and only accepts .png
.
If you do not include the rarity weight in the attribute folder name, that attribute will be ignored. These need to be integers. If you want decimal %s, multiple all the rarities by 10 or 100.
You can provide any number of frames in each layer folder, the code will repeat them up until it hits numberOfFrames
.
It will also trim any that have too many frames.
Example layers folder structure with four layers and two traits each layer:
layers
ββββBackground
β ββββGrey#50
β β 0.png
β ββββPink#50
β β 0.png
ββββBall
β ββββBlue#50
β β 0.png
β β 1.png
β β 2.png
β β ...
β ββββGreen#50
β β 0.png
β β 1.png
β β 2.png
β β ...
ββββHat
β ββββBirthday#50
β β 0.png
β β 1.png
β β 2.png
β β ...
β ββββCowboy#50
β β 0.png
β β 1.png
β β 2.png
β β ...
ββββLandscape
β ββββCupcake#50
β β 0.png
β ββββGreen Tower#50
β β 0.png
Example layer:
Background:
Grey:
Pink:
Ball:
Blue:
...
Green:
...
Hat:
Birthday:
...
Cowboy:
...
Landscape:
Cupcake:
Green Tower:
I am using python here instead of javascript libraries because I have found that image processing using PIL is much faster and without lossy quality than javascript. These benefits are much clearer in step 3.
You can run only step1 by running:
make step1
This will convert the pngs into spritesheets and the output will look something like this:
Output:
Background:
Grey#50.png:
Pink#50.png:
Ball:
Blue#50.png:
Green#50.png:
Hat:
Birthday#50.png:
Cowboy#50.png:
Landscape:
Cupcake#50.png:
Green Tower#50.png:
EDIT tool now supports z-index/stacking, grouping and if-then statements. See nftchef's docs for more information. The layers in this step will have to match the format expected in step 2. See the example layer folders for some more info.
EDIT tool now supports gif layers. You can provide layers as gifs and the code will split the gif into
frames. See layers_gif_example
. It will create a temp folder in step1_layers_to_spritesheet/temp with the
resulting separate frames, and then will parse through that folder to create the output. Make sure numberOfFrames
is set in global_config.json.
Step 2 takes the spritesheets from step 1 and generates all possible combinations based on rarity. This is where all the magic happens! The output is a bunch of spritesheets with all the layers layered on top of each other.
The original idea came from MichaPipo's Generative Gif Engine but now most of the code in this step is forked from nftchef's Generative Engine which is forked from HashLips Generative Art Engine. Please check out Hashlip's πΊ Youtube / π Discord / π¦ Twitter / βΉοΈ Website for a more in depth explanation on how the generative process works.
I recently modified this section to use the code from nftchef's Generative Engine which adds the following features:
layers_if_then
which has logic for if the ball is pink, wear a birthday or cowboy hat, or if the ball is purple, wear a mini ball hat. See nftchef's docs for more information. You only need to list the root folder names in config.js
.layers_grouping
we have common balls and hats, and rare balls and hats, and the first totalSupply - 1
balls are common, and the last one is rare. This will output in order, but you
can shuffle the layers by setting shuffleLayerConfigurations
in config.js
to true
.You will need to update global_config.json
and also update layerConfigurations
in step2_spritesheet_to_generative_sheet/src/config.js
.
You can run only step 2 by running:
make step2
Example output with the layers
folder (only first 4 displayed, but there are 16 total):
Example output with the layers_z_index
folder:
Step 3 takes the spritesheets from step 2 and creates gifs/MP4. Initially I used PIL, but found some issues with pixel quality.
In MichaPipo's original repo, they used javascript libraries to create the gifs. These copied pixel by pixel, and the logic was a bit complicated. Creating just 15 gifs would take 4 minutes, and I noticed some of the pixel hex colors were off. Also depending on CPU usage, the program would crash. I spent days debugging, when I just decided to start from scratch in another language.
I then tried imageio, and a few Python libraries, but they all had some issues generating gifs.
I spent weeks finding the best tool for this job, and came across gifski. This creates incredibly clean gifs and worked the best.
Now, generating 15 gifs takes < 30 seconds and renders with perfect pixel quality!
You can change the framesPerSecond
in global_config.json
and you can run only step 3 by running:
make step3
This allows you to not have to regenerate everything to play around with fps.
Example output with all 16 permutations (click on each gif for the 1000x1000 version):
Some metrics:
MichaPipo's Generative Gif Engine:
New Generative Gif Engine:
NOTE
imageio
was by far the best Python library, so I added it as an option in case you don't want
to download gifski
. imageio
will work for most pixel art and I know some people had issues
with gifski
on Linux (not Windows or Mac).
You can set which gif tool to use in global_config.json
by setting gifTool
to either gifski
(default) or imageio
.
If you want to switch between generating gif vs. MP4, you need to change outputType
to mp4
and only run make step3
.
Tool now supports z-index/stacking, grouping, if-then statements, and incompatibilities. See nftchef's docs for more information.
TLDR if you don't want to read the doc:
z_,
in front of the name, for example z1,
or z2,
. See layers_z_index
for an example, and try setting the layersFolder
to layers_z_index
to see this in action and checkout layerConfigurationsZIndex
in the config.js
for the configuration.layers_grouping
folder and layerConfigurationsGrouping
in the config.js
. - NOTE THIS DOES NOT WORK RIGHT NOW SEE https://github.com/jalagar/animated-art-engine/issues/175layers_if_then
folder and layerConfigurationsIfThen
in config.js
.layers_incompatible
and const incompatible
in config.js
. You can uncomment the line, and run the code with layersFolder
set to layers_incompatible
to see it working in action.π§ͺ BETA FEATURE
You can now add specific audio per trait. For example if you want wind noises with a wind background, and forest noises with a forest background.
Just put the audio file in the corresponding layer folder, and step3 will take that and put it on the mp4. You can see an example in the layers_audio
folder. Try it out by setting layersFolder
to layers_audio
and enableAudio
to true
, then run make all
. The mp4 will be the length of frames
and the audio will get truncated if it is too long.
The tool supports mp3
, wav
, and m4a
. If there are multiple audio files for the same NFT, it will
combine the audio files and overlap them.
π§ͺ BETA FEATURE
If you have existing metadata for an existing collection and want to either create a new collection with GIFs/MP4 or send GIF/MP4 version of the static image to holders, this feature is for you! OR if you want to export as a spritesheet that can be imported into a pixel metaverse, this feature is for you!
There are a few configurations to you can use the tool:
_dna.json
generated by NFT Chef's repo, and a _metadata.json
file which contains all the JSON files. Load the _dna.json
into the build
folder, and load the _metadata.json
into the build/json
folder. Setup your layers following the format above. Setup global_config.json
and config.js
and run make regenerate
. You can regenerate in parts by following instructions in Generate entire collection in parts except edit regenerate.py
instead of all.py
.
This is the most accurate and consistent way of generating GIFs based on existing layers and will work with NFT Chef's features._dna.json
. You will only have _metadata.json
which contains all the JSON files. Load this into the build/json
folder, setup layers, setup global_config.json
, config.js
and run make regenerate
. This under the hood attempts to regenerate
the DNA based on the JSON. This should work, but there may be features that are not backwards compatible so let me know if you come across such a case._metadata.json
file. Load all the individual .json
files into build/json
. Setup layers, setup global_config.json
, config.js
and run make regenerate
. This is more annoying to do (if you have a ton of files), but will regenerate the _metadata.json
, the _dna.json
, and then regenerate the collection.If you only want to regenerate spritesheets, you can set SKIP_STEP_ONE
to True
and SKIP_STEP_THREE
to True
in regenerate.py
. Then instead
of putting your layers in the layers
folder, you put them in step1_layers_to_spritesheet/output
as an entire layer, and then
run make regenerate
. The spritesheets will be in step2_spritesheet_to_generative_sheet/output
.
If you need more than 32 frames at 1000x1000, follow the batches configuration and then run make regenerate
. This will only work if you are doing
all the steps and not skipping any.
Please let me know if you have any issues or use cases I did not think of.
You can check the rarity stats of your collection with:
make rarity
If you want to have a layer ignored in the DNA uniqueness check, you can set bypassDNA: true
in the options
object. This has the effect of making sure the rest of the traits are unique while not considering the Background
Layers as traits, for example. The layers are included in the final image.
layersOrder: [
{ name: "Background" },
{ name: "Background" ,
options: {
bypassDNA: false;
}
},
If you need to generate a provenance hash (and, yes, you should, read about it here ),
run the following util
make provenance
This will add a imageHash
to each .json
file and then concatenate them
and hash the file value into one string which is the provenance
hash.
The Provenance information is saved to the build directory in _provenance.json
. This file contains the final provenance hash as well as the (long) concatenated hash string.
*Note, if you regenerate the gifs, You will also need to regenerate this hash.
If you need to remove a trait from the generated attributes for ALL the generated metadata .json files, you can use the removeTrait util command.
cd step2_spritesheet_to_generative_sheet && node utils/removeTrait.js "Trait Name"
If you would like to print additional logging, use the -d flag
cd step2_spritesheet_to_generative_sheet && node utils/removeTrait.js "Trait Name" -d
You can change the description and base Uri of your metadata even after running the code by updating global_config.json
and running:
make update_json
If you would like to manually add 'hand drawn' or unique versions into the pool of generated items, this utility takes a source folder (of your new artwork) and inserts it into the build
directory, assigning them to random id's.
example:
βββ ultraRares
β βββ gifs
β β βββ 0.gif
β β βββ 1.gif
β βββ json
β βββ 0.json
β βββ 1.json
You must have matching json files for each of your images.
Because this script randomizes which tokens to replace/place, it is important to update the metadata properly with the resulting tokenId #.
Everywhere you need the edition number in the metadata should use the ##
identifier.
"edition": "##",
Don't forget the image URI!
"name": "## super rare sunburn ",
"image": "ipfs://NewUriToReplace/##.png",
"edition": "##",
Run with make replace
. If you need to replace the folder name, you may have to edit the Makefile
directly with the folder.
Note this will not update _dna.json because these new JSONs don't have DNA. This will modify _metadata.json though.
π§ͺ BETA FEATURE
After running make all
you can run generate the Solana metadata in two steps:
step2_spritesheet_to_generative_sheet/Solana/solanaConfig.js
make solana
to generate the Solana metadata. This will create an output folder build/solana
with the gifs and the metadata.Most of the code comes from nftchef.
I have not tried this on any test net or production Solana chain, so please flag any issues or create a PR to fix them!
π§ͺ BETA FEATURE
I have not tried this on any test net or production Tezos chain, so please flag any issues or create a PR to fix them!
See Tezos README for more information.
Do you want higher resolution, more frames, and larger gifs/MP4? Batching is for you! Currently step2 is limited by 32000 pixel files, so in order to get around this we must batch the entire process into chunks and then combine them at the end.
Set useBatches
in global_config.json
to true
and then set numFramesPerBatch
to a smaller frame batch. NOTE try testing different frames per batch to see if rendering a smaller number of
editions is faster or slower. Some users have said smaller batches renders faster.
This works for odd number of frames as well, ex. 35 total frames, and 12 frames per batch, it will generate 2 batches of 12 frames, and 1 batch of 11 frames automatically.
Then run make all
which runs python3 all.py
. This under the hood generates the JSON metadata for the first
batch and then regenerates the next batches based on the existing JSON.
π§ͺ BETA FEATURE
Do you want someway for users to toggle between static and animated images (similar to Little Lemon Friends)?
Check out README_Assets/html/0.html
for an example (open it in any browser and click the icon on the top left). Markdown doesn't allow embedding all HTML tags and javascript so you'll have to open it separately.
Steps to generate HTML:
build/gifs
or build/mp4
folder. You can do this by running make all
or just copying them into the folders.build/pfps
folder. You can do this by running make all
with generatePFP
set to true
. You can pick which frame to be your PFPs in pfpFrameNumber
. OR you can just drag the PFPs if you already have them generated.generate_html/logo.png
. It has to be called logo.png
all lowercase.make html
. All the html files should be in build/html
. You can change styling by editing generate_html/template.html
.animationUri
in global_config.json
, then run make update_json
.π§ͺ BETA FEATURE
Let's say you want to generate a 10k collection with 100 frames each. Most computers don't have enough space to handle hundreds of GB of data, plus the whole process can take days and might get interrupted halfway.
Instead of paying for a remote server and having to pay for tons of storage, now you can run the whole generation process locally in parts! This is different than the "batching" mentioned in previous sections, which batches the frames into smaller batches. Generating in parts means you can generate only part of the collection at a time.
For example, let's say you have a 10k collection with 120 frames. Your global config might look like:
{
"totalSupply": 10000,
...
"startIndex": 0,
...
"useBatches": true,
"numFramesPerBatch": 20
}
Now instead of just running make all
which will most likely error, you can genereate only 1k editions at a time. You will need to edit all.py
.
Look for START_EDITION
and END_EDITION
. These are going to be which range of editions you want to generate. For example first we could generate 0 - 1000. Edit the file with START_EDITION = 0
, and END_EDITION = 1000
. This under the hood will generate all 10K JSON files, but only generate the first 1K. You can check rarity and other metadata info now. NOTE END_EDITION
is EXCLUSIVE, meaning this will only generate 0 - 999 (total of 1000).
After this finishes, move the build
folder to somewhere else on your computer, or an external hard
drive, and start generating START_EDITION = 1000
and END_EDITION = 2000
. This will generate 1000 - 1999. Repeat the process until (making sure to move the files out of the build folder), START_EDITION = 9000
to END_EDITION = 10000
.
If you want a preview gif/MP4 of a subset of gifs (like Hashlips), run
make preview
This will output preview.gif
/preview.mp4
in the build
folder. The default number of previews is 4 but you can change this in
step3_generative_sheet_to_output/preview.py
at the top NUM_PREVIEW_OUTPUT
. Currently it will randomly select the gifs/MP4,
if you want to output the first X, set SORT_ORDER
to OrderEnum.ASC
and if you want to output the last X,
set SORT_ORDER
to OrderEnum.DESC
.
Q: Why did you decide to use Python for step 1 and step 3?
A: I found that Python PIL work better and faster than JS libraries, and the code is simpler for me. Initially I tried PIL, imageio, and a few Python libraries, but they all had issues generating gifs. I spent weeks finding the best tool for this job, and came across gifski. This creates incredibly clean gifs and worked the best.
My philosophy is pick the right tool for the right job. If someone finds a better library for this specific job, then let me know!
Q: Why didn't you use Python for step 2?
A: The NFT dev community which writes the complicated logic for generative art mainly codes in javascript. I want to make it easy to update my code and incorporate the best features of other repos as easily as possible, and porting everything to Python would be a pain. You can imagine step 1 and step 3 are just helper tools in Python, and step 2 is where most of the business logic comes from.
Q: What file types do you support?
Input type: gif or png
Output type: gif or MP4
Q: What blockchains do you support?
Ethereum, Solana, Tezos.
Q: I have issues on Windows what should I do?
Check the discussion board, or ask Discord for more issues. Common problems can be found in the installation section.
Be sure to follow me for more updates on this project:
My ETH address is 0x4233EfcB109BF6618071759335a7b9ab84F2F4f3 if you feel like being generous :). I just quit my job to work on NFTs full time so anything is appreciated.
If you want to see this code in action, we used it for my fitness and mental health project Fit Friends (now shut down). Check it out here:
If you have any questions try asking the Hashlips' Discord. We have a separate channel just for this repo! Go to #animated-art-engine.