feature : core add-on ofxAssets

[elliotwoods]

hey all!

in a few of my addons i build an asset register to handle:

• images
• shaders
• fonts

The idea is common. That on program start (and whenever you call ofxAssets::rescan()), it scans the data/ folder for assets and builds the dictionaries.

e.g. if my folder structure is like

data/landscape.png
data/phong.vert
data/phong.frag
data/blur.frag
data/verdana.ttf

or sometimes i do...

data/images/landscape.png
data/shaders/phong.vert
data/shaders/phong.frag
data/shaders/blur.frag
data/fonts/verdana.ttf

I can then access these from anywhere in my app as long as i'm including the ofxAssets header:

#include "ofxAssets.h"
using namespace ofxAssets;

void testApp::draw() {
    shader["blur"].begin();
    image["landscape"].draw(10,10);
    shader["blur"].end();
    font["verdana"].drawString("is this API correct?", 10, 10);

    //or if i don't want to add the using namespace clause
    ofxAssets::shader["phong"].setUniform3f("position", myVec3);
}

I'd like to see this as a core add-on as I think it's rather universally useful, the code is beginning to be duplicated in different addons and the API is simple enough to avoid much deliberation. (spoke to soon)

[elliotwoods]

An example of where I've implemented this before: https://github.com/elliotwoods/ofxGrabScene/blob/master/src/ofxGrabScene/Assets.cpp https://github.com/elliotwoods/ofxGrabScene/blob/master/src/ofxGrabScene/Assets.h

I presume my use of statics needs some criticism from @arturoc before it's rock-steady (and beebop).

[julapy]

ive done something similar in ofxFlash but instead of scanning the data folder, it goes through the the XFL file and locates the assets based on their paths in the XFL. https://github.com/julapy/ofxFlash/blob/master/src/core/ofxFlashLibrary.cpp

all assets are loaded into OF when the app first launches, which is probably not the best approach as it eats up a lot of memory with a big asset library and sometimes you just want to load things when you need them.

having said that, maybe using a smart pointer or reference counting might be a good way of determining if an asset is being used and if not it can be automatically deleted until it is used again somewhere else in the app.

i like the ofAsset idea and it would be a relatively small / standalone class which would provide some life hacking shortcuts.

[elliotwoods]

nice! i think the issue with reference counting is that then we need people to make local instances to gather the intelligence we need regarding usage, which affects the API.

In terms of taking up too much memory in general, i think the idea would be to present a standard usage case whereby whenever you use ofxAssets, then you presume everything is loaded all the time.

I can see that we should probably use a definite assets folder like data/assets/ to avoid loading everything that might be in the data folder (for instance, if the user has some 1000 frame png sequences that they don't want loading by accident)

[bilderbuchi]

this would be a useful feature. One thing where I see this could be really practical, is an OF-global assets folder, and asset folders for addons. that way, we could avoid the duplication of certain files across all of OF (e.g., check how many identical .ttf files are in the repo). This would probably need some file-copying/binary-baking support, though, as far as I know. Is that realistic, also w.r.t cross-platformness?

[ofTheo]

Curious how fonts would work. How would it know what size font to load?

[underdoeg]

Maybe you could store the information that the font is there and as soon as you request it the first time, it'll load the size you want if not already loaded.

[elliotwoods]

Ah! For fonts I put a text file next door (e.g. 'verdana.txt' with a list of sizes I want to load) Admittedly there's maybe a more ideal approach there. Text files are good for being specific about how you want them to be used But perhaps aren't very standard/obvious

[bakercp]

Just for another reference, I have something similar (but way too much for an core addon) https://github.com/themaw/ofxLivedrawEngine/tree/master/src/assets and (see here for the main manager https://github.com/themaw/ofxLivedrawEngine/blob/master/src/assets/management/AssetManager.h). I have a fairly sophisticated asset management / reference counted system in there. For that project its all about sharing video frames via sources and sinks and caching videos with ofxVideoBuffers when needed. Anyway ofxLivedraw is almost up and running if you wanted to play with it.

Regarding fonts -- I'm working on something like this in ofxFont. ofxFont's font manager will keep the TTF in memory and add / cache sizes lazily as requested.

[elliotwoods]

Maybe

ofxAssets::font("verdana", 14).drawString(...);

With lazy loading.

Any objections to this becoming a core addon?

[ofTheo]

Hi Elliot!

I feel like we usually evaluate the addon inclusion possibilities once it exists and people can check it out / play with it etc. It would give us a chance to see how it works in practice ( try it with some projects ) and see if anything needs adjusting.

I think then it would be easier to figure out if this is something that could be useful to a wide range of people and an appropriate addon to come bundled with OF.

Cheers! Theo

ps: like your solution to the font size issue. nice as it would allow for whatever font you needed as you needed it.

[bakercp]

+1 for @ofTheo 's response. That said,@elliotwoods I believe that your concept of ofxAsset idea is solid.

[kylemcdonald]

+1 to Theo but I also just want to say I recognize the value of things being built at the core level in some cases. For example, ofFile wouldn't have made sense if it started as ofxFile. That said, there would have to be a more fleshed out proposal for ofxAssets to start development at that same level.

[elliotwoods]

here's a slightly tipsy rendition: https://github.com/elliotwoods/ofxAssets.git seems to work here

might not work on windows due to issues with directories (only tested osx here, but that's going to be fixed by the end of this week!)

please try and give feedback!

[yty]

I really like idea ofxAssets.. I hope you can join imagesSequene (CPU & GPU). imagesSprite ....

[ofZach]

@elliotwoods is this still something you'd like to have considered for a core addon? I've noticed you've made recent changes to the https://github.com/elliotwoods/ofxAssets repo... I personally like this kind of thing and I think it will be quite helpful, just want to check if this is something you still want to consider?

[danzeeeman]

+1 to this! We've all written something to do this and it would be awesome to have something in the core!

[elliotwoods]

hey! back in the house yes i'd love if this would be considered as a core addon. that would mean putting the current design up for discussion so it suits everybody equally i've rewritten this twice since we last talked, and i'm happy with the shape of it it now

Some updates:

Folders

Subfolders become namespaces, i.e.

bin/data/assets/images/icons/scissor.png can be accessed using ofxAssets::image("icons::scissor")

Addons

Right now if you call ofxAssets::addAddon("ofxAddonName");

then ../../../addons/ofxAddonName/data will be copied to bin/data/assts/ofxAddonName. Currently this happens during runtime in Debug mode (ideally it would be a copy/build script on all platforms, but this is a 'fool proof' approach to start with).

And at runtime, bin/data/assets/ofxMyAddon/images/icons/scissor.png can be accessed using ofxAssets::image("ofxMyAddon::icons::scissor")

Auto-reload

Uses Poco's file watcher and reloads if file changes (especially useful for shaders!).

Sounds

ofxAssets now supports sounds, which can be played with ofSoundPlayer, and if HAS_OFXAUDIODECODER is defined then you can also get the sound buffer (probably need to remove/clean this if it were to go core).

Possible future features:

• Other asset types (although i think we have the main bases covered)
• Copy into the app resources folder on OSX (e.g. for Release builds)
• Allow assets to be stored in an archive with password supports (e.g. for larger public projects)

[arturoc]

Some ideas:

• For autoloading we could have something more generic, a class that accepts a path or a set of paths and a loading function is easy to implement and it could load anything, then we can have predefined cases for the most common types, shader, image, sound... but won't be restricted to those and anyone can use it to reload anything if the file(s) changed, using it would be something like:

ofxAutoLoader<ofxAssimpModelLoader> autoloader("mymodel.dae", []{
    ofxAssimpModelLoader model;
    model.load("mypath.dae");
    return model;
});

which would query the modification time of "mymodel.dae" with a certain periodicity and call the passed function if it changed.

This class can also become useful on their own for people who only want to reload some resource quickly without the other functionality

• I think the way of specifying paths shouldn't use a different syntax, just use the original path. Android does something similar to what you point out by using shorter names derived from the file names and it becomes super limited at times.
• Poco is going away or at least it'll become an addon soon-ish so it would be good not to use it, you can just query the file date from time to time and load if it has changed