proposal: a new package for memory-efficient asset management

[hajimehoshi]

Operating System

• [ ] Windows
• [ ] macOS
• [ ] Linux
• [ ] FreeBSD
• [ ] OpenBSD
• [ ] Android
• [ ] iOS
• [ ] Nintendo Switch
• [ ] PlayStation 5
• [ ] Xbox
• [ ] Web Browsers

What feature would you like to be added?

Overview

From our experiment in our games (Odencat Inc.) and other games, we found we often had to implement an asset manager for memory efficiency. Memory efficiency is important for some low-end platforms like mobiles, browsers, and consoles. However, it is annoying to implement such a manager for each game.

I propose to have an official asset manager for such use cases.

API

package assetmanager

type AssetManager struct {}

// New creates a new asset manager.
func New(fs fs.FS) *AssetManager

// SetImageCacheLifetime sets the lifetime of image cache entries in ticks.
// The default is (TBD).
// TODO: Should this be time.Duration?
// TODO: Is this a lifetime for *ebiten.Image, or decoded data?
func (a *AssetManager) SetImageCacheLifetime(ticks int)

// SetAudioCacheLifetime sets the lifetime of audio cache entries in ticks.
// The default is (TBD).
// TODO: Should this be time.Duration?
// TODO: Is this a lifetime for players, or decoded data?
func (a *AssetManager) SetAudioCacheLifetime(ticks int)

// Image returns an image for the given path.
// A returned image is cached.
// A cached entry is discard a while after its last usage.
//
// Image might return the same object for the same path.
//
// This package doesn't import any image-decoding packages like image/png,
// so you have to import appropriate packages.
func (a *AssetManager) Image(path string) (*ebiten.Image, error)

// ShortAudioPlayer returns an audio player for the given path.
// ShortAudioPlayer is suitable for one-shot SE players.
//
// ShortAudioPlayer always creates a new player for the given path.
//
// The decoded data is cached.
// A cached entry is discarded a while after its last usage.
// 
// The audio file type is detected based on the magic number of the first 4 bytes.
func (a *AssetManager) ShortAudioPlayer(path string) (*audio.Player, error)

// LongAudioPlayer returns and audio player for the given path.
// LongAudioPlayer is suitable for long BGM players.
//
// LongAudioPlayer might the same player for the same path.
// Thus, you cannot use multiple players for the same path at the same time.
//
// The decoded data is not cached, but the player is cached.
// A cached entry is discarded a while after its last usage.
//
// The audio file type is detected based on the magic number of the first 4 bytes.
func (a *AssetManager) LongAudioPlayer(path string, options *LongAudioPlayerOptions) (*audio.Player, error)

type LongAudioPlayerPositions struct {
    Loop bool
    IntroLength int64
    LoopLength int64
}

Rationale

Why not an independent module?

AssetManager has to know the last usage of an image, thus AssetManager has to access the internal of Ebitengine.

[hajimehoshi]

Q. What if we want to use CDN as a backend? A. You can implement a fs.FS with the CDN backed, but in this case, an asset manager have to consider asynchronous loading. Hmm. For example, Image can return a channel instead of an image object directly.

[hajimehoshi]

The backend doesn't have to be a fs.FS. What about this?

type Backend interface {
    Open(path string) (io.ReadSeekCloser, error)
}

func NewBackendFromFS(fs fs.FS) Backend

func New(backend Backend) *AssetManager

[hajimehoshi]

I realized there are two memory regions: CPU and GPU. The aim is to minimize GPU memory usages compared to CPU memory usages. For Audio, only CPU memory matters. Thus, the APIs to set lifetime should be like this:

• SetImageCPUCacheLifetime // e.g. a decoded RGBA binary data. The default is like 1 minutes?
• SetImageGPUCacheLifetime // e.g. *ebiten.Image's internal texture data. The default is like 1 seconds?
• SetAudioCPUCacheLifetime // e.g. a decoded audio PCM data. The default is like 1 minutes?

[kettek]

To note, merged filesystems can be good/important for modding. e.g., you load your base assets, then load other implementations on top of the base to override. See https://github.com/kettek/go-multipath -- it's basically just a virtualized FS that aggregates multiple FS sources based upon a hierarchy.

[hajimehoshi]

To note, merged filesystems can be good/important for modding

I think having Backend interface (https://github.com/hajimehoshi/ebiten/issues/3100#issuecomment-2351080032) is enough since you can implement Backend as you like. Is my understanding correct?

[kettek]

Yes, that should actually be sufficient.