Asset icons from thumbnail image

[BigRoy]

Feature proposal

I'd like to implement support to allow custom asset thumbnails per asset in a project.

Example preview where asset sandbox > hero has a custom thumbnail image.

Implementation

However, regarding implementation on where to store the thumbnails I want to open up this issue get some feedback about how and where everyone would like to store these icon locations.

1. My original idea was to add it in project[config][template] as a new "thumbnails" template. Something like: "{root}/{project}/resources/.thumbnails/{asset[name]}.png". Whenever it finds the image it would load it up, otherwise it falls back to default icon.

Pros

• Set it once for a project and it would work for all assets
• Customizable to be outside of an asset's folder, or even inside of it. Project configuration can decide.

Cons

• It will require the model to check for each asset whether the file on disk exists, potentially slow for many assets. (But could be done in a separate thread however, and cached!)
• Can't easily (without symlinking) link two assets to the same icon on disk

2. Then I thought about setting it on the asset itself, e.g. in asset[data][icon]

Pros

• When the data is not set, we know there's no icon... so we can skip looking for it on disk.
• The same logic could also work for published content by storing in data[icon]
• Customizable to be outside of an asset's folder, or even inside of it.
• Feel like the more logical place to find the information. (very subjective!)

Cons

• Lots of duplication of "path/to/the/folder", plus potentially more string formatting for the full path.
• Might get studios to store full paths, even when not recommended by Avalon (because of cross-OS, different servers, etc.) because they could do it.
• Just more data to potentially manage.

---

Technical implementation of how to load the icons can be discussed, but my goal will be to support loading them threaded so the thumbnails are not intended to lag the UI or anything alike. My focus now is mostly to discuss how and where to arrange the storage of the thumbnails.

[mottosso]

Here's one more option I've often opted for in situations like this.

asset["data"]["icon"] = "flower"

Where flower is a keyword to a repository of icons, registered elsewhere. It's similar to how you get to pick an icon from AwesomeFont, where each character has a unique (unicode) ID, and the font itself is referenced elsewhere. It creates this nice separation between id and payload.

The assets could for example be..

config["data"]["icons"] = r"c:\path\to\icons"

Where that folder would contain e.g. flower.png or better yet, flower-16x16.png such that you would give an asset an id like flower, and the system could pick the right resolution based on use. 16 px for TreeView, 64 px for anything bigger etc.

[iLLiCiTiT]

There is also possibility to store thumbnail (.png, .jpeg, ...) directly to database and on disk store only set of default icons when thumbnail is not set.

[BigRoy]

https://github.com/getavalon/core/issues/390#issuecomment-499865054 Here's one more option I've often opted for in situations like this.

That's an interesting one too, like a relative search path to whatever is formatted as "image root" in the project config for thumbnails. Could be clever, yes. Still the project config could just say that the root is in the asset's folder. This could somehow combine best of both worlds.

There is also possibility to store thumbnail (.png, .jpeg, ...) directly to database.

I was thinking about that too - it would make the link more "direct" however I'm a bit afraid that in most cases the assets are queried in full across a wide range of locations in the code. Say per asset the icon is 3kb it would mean a project would suddenly start doing a lot of queries that would contain data of quite some size. Again, potentially premature optimization in my head - but that was what came up for me as first thought and I couldn't come up with direct benefits of storing it there as raw binary data.

[mottosso]

Say per asset the icon is 3kb it would mean a project would suddenly start doing a lot of queries that would contain data of quite some size.

You can solve this using the filter operator, to only ask for parts of a document.

Another thing to keep in mind when uploading this kind of data is that backups become more expensive, regardless of query performance.

[BigRoy]

You can solve this using the filter operator, to only ask for parts of a document.

True, however this would mean that all default queries in avalon tools, etc. should start doing so. ;) Quite a bit of management to ensure queries remain performant.

[iLLiCiTiT]

Also thumbnail can be stored as new entity in DB. Now are used types: project, asset, version, repre. "thumbnail" type can be add which should store binaries and on asset/representation will be stored "_id" of that entity. But true is that his may cause issues with Database latency...

[BigRoy]

This discussion unfortunately didn't lead into a clear decision path. Is there anyone else who can contribute ideas @tokejepsen @davidlatwe? Or maybe @mottosso would you happen to have new views/ideas on this for a first prototype?

[mottosso]

Start by storing it as a base64-encoded value in the database.

{
  "name": "MyAsset",
  "data": {"icon": "5gb154n2j4j25lkj46l4k5j34lkj..."}
}

base64 is plain-text you can later convert into a pixmap for Qt, and Python has a standard library for serialising/deserialising. Then if (probably not) there is a database hit on performance, memory or traffic, we can have a look at optimising things.

[mottosso]

Also set some max size for it in the serialiser, so it doesn't go out of hand. 1 overly sized icon won't get noticed, but once you have 100 icons it may, at which point it's a little late to re-convert.

imagedata = to_base64(QPixmap("c:\some\image"))
# Error! Too large

[iLLiCiTiT]

Qt is also able to resize images. But now, I'm not sure about loading from DB since #447 just solved slow loading of assets.

[mottosso]

Let's not worry about performance until we can prove it matters, like we did with that issue. We'll need the base64 functionality regardless, it's a good baseline and keeps things simple with multiple platforms, storage locations, remote access and keeps code from accessing both filesystem and database at once.

It's very likely we can introduce caching at some point as well, to continue reaping those benefits.

1. Either in-memory, for the lifetime of a e.g. Maya session
2. Remote, such as in a memcached or redis instance

Those are hoops potentially worth jumping through to avoid another link from database to filesystem, but we won't know until we've tried. Odds are the impact is nil.

[iLLiCiTiT]

Just an idea: Cashing can be based on datetime of thumbnail creation. Cashed file name can be based on template "{asset_id}.{datetime_of_creation}". That would require to store also that information to data:

{
  "name": "MyAsset",
  "data": {
        "icon": "5gb154n2j4j25lkj46l4k5j34lkj...",
        "thumbnail_creation": 2019-09-13 12:34:05.819066
    }
}

[mottosso]

Yes, that's a good idea. You could in fact store this along with the thumbnail itself. base64 is a binary encoder, meaning you could say..

class MyIcon(object):
  def __init__(self):
    self.fname = "c:/some/file.png"
    self.image = QPixmap(fname)

import pickle
import base64

icon = MyIcon()
encoded = base64.b64encode(pickle.dumps(icon))
icon = base64.b64decode(pickle.loads(encoded))
print(icon.fname)

Untested, but you get the gist.

[mottosso]

For a start though, to keep the image portable, I would suggest storing the PNG bytes themselves, e.g...

with open("image.png", "rb") as f:  # binary
  image = f.read()
bimage = b64encode(image)

I'm pretty sure Qt is able to take a filepath to a PNG, or the data as-is. By storing the PNG instead of a class instance, the data will be portable across platforms and versions of Python (e.g. 2 and 3) and Qt (4 and 5) too.

[mkolar]

We have resumed the conversation in pype club due to a lot of clients asking for thumbnails across the board in loader for example. I'll sum up our thoughts thus far.

1. We feel the best way is to store the thumbnails on disc so their storage can be moved around and backed up separately from the main db if need be, also to prevent performance hits on the db calls.

2. We'd make a new type in the db called thumbnail with a very simple schema. It would hold path to the image file. In optional data there could be more of them, so we could point it to different resolutions, sequence to allow playing on mouse hover, or even id of a quicktime representation that corresponds to it, so user could invoke the moving version when he clicks on the thumbnail.

3. Each asset and version would then have optional data thumbnail that would point to an id of the thumbnail to use. So we could do things like assigning the same thumbnail to multiple assets if we wanted. For example, a studio might want an image of a film reel to be a thumbnail for all sequences.

Now whether the image itself is stored somewhere on the disk (point 1. ) or directly in the db, is a secondary problem I think, and it could easily work with both. If the thumbnail entity contains the actual data, use it, otherwise look for it on the disk. The key question I believe is whether we should make a new entity for it so that we don't have to store string paths, or binaries directly on assets, versions and so on.

What to do with them once they exist is then a question for another issue :)

what do you guys think?

[BigRoy]

1. My gut feeling is that saving on disk will be most trivial and serves the purpose we need. :+1:
2. I am not sure that's needed. Can't we just support a filepath either with a frame identifier like %04d in the case you'd need a sequence. I wouldn't want to link a thumbnail a quicktime - seems like that is defeating the purpose of the 'thumbnail'.

What's the reason an embedded path wouldn't suffice?

Say project config has thumbnailTemplate that can define where the thumbnails are saved using the full context. E.g. one could decide to store it globally in the project in {root}/resources/thumbnails/{thumbnail} or per asset {root}/{asset}/{thumbnail} (simplified example).

Then the asset (or version or representation?) would just provide a thumbnail data that would be what is added to the template? Would that be good enough?

Again, we should avoid hardcoding full paths in the database (cross-OS, multiple offices, etc.)

[mottosso]

I remember suggesting this before, but what about:

1. Studio/workstation defines an environment variable, e.g. ICON_ROOT=/some/path
2. DB provides suffix, e.g. spiderman/heroCharacterIcon.png

That way, you get cross-platform and flexibility on where icons are at. The suffix could even be interpreted as an ID of sorts, if the icon is stored in the/another DB.

also to prevent performance hits on the db calls.

Have you profiled it? Because my guess is fetching anything from the DB is going to be a lot faster than dealing with files. The DB is essentially a big Python dict, handing out data straight from memory, especially from frequent reads.

Try not to prematurely optimise this. I would give serialising it into a db entry like we mentioned above a try. You may just find it's quicker than you'd expect and I can't think of any reason you would want thumbnails separated from their database entry, unless it's for some (potentially misplaced) optimisation reason.

For sequences/video, that's a separate problem domain altogether. I would solve one problem at a time. Thumbnails are useful in their own right is likely to have a very different access pattern and usecase.

[mkolar]

Should have been more clear about the path :)

Studio/workstation defines an environment variable, e.g. ICON_ROOT=/some/path DB provides suffix, e.g. spiderman/heroCharacterIcon.png

Say project config has thumbnailTemplate that can define where the thumbnails are saved using the full context. E.g. one could decide to store it globally in the project in {root}/resources/thumbnails/{thumbnail} or per asset {root}/{asset}/{thumbnail} (simplified example).

Absolutely. (well some combination of the two :) ). We were thinking along similar lines. So that studio can potentially move the thumbnail path around different storage if needed. Keeping in mind that in cross-platform scenarios, it would be up to the studio setup to make sure this root is resolved to the correct platform.

Maybe simply {THUMBNAIL_ROOT}/P01_superProject/{thumbnailID}.png

The reason I don't use asset here is that thumbnails are generated from asset/substet/version and can then be used on asset itself for example, but also on the version. Storing just the id doesn't bake in the link to the original created context, thus making it easier to reuse. Client might want to display the latest published thumbnail on an asset for example. We could do that dynamically.

Have you profiled it? Because my guess is fetching anything from the DB is going to be a lot faster >than dealing with files.

We haven't, however, here are our thoughts.

• When serving DB over vpn to freelancers, we could cache the thumbnails to his local drive to avoid getting images across the vpn all the time,
• They are non-essentiall project data that could be backed up separately.
• NDAs for some of our client occasionally don't allow access to any images outside of the studio even via VPN. With avalon they would be able to give freelancers working on a particular asset ability to connect to a mongo DB to publish data, but prevent them from seeing the rest of the project. As soon as we put image data in the DB, we'll have pushback.

That being said, it would be trivial to have multiple option for studio to choose from. If they want them in the DB, so be it, if on the drive, no problem. It's just about pointing to that data.

am not sure that's needed. Can't we just support a filepath either with a frame identifier like %04d in the case you'd need a sequence.

Yeah we discussed the same point a lot. But what if you want to store multiple resolutions in the future? What happens when you want to assign the same thumbnail to multiple entities? We want to for example use a generic thumbnail or icon for any entity that doesn't have it's own yet. These will need to be stored somewhere. We just feel that adding a new entity is the cleanest way to work with this data.

When it comes to the animation, we think that thumbnail should by default always be single image. If you store a sequence (to show turnaround in gui for instance) it should be extra data that only loads when you need it.