Convert avatars to WEBP during upload

[silverwind]

Feature Description

As discussed in https://github.com/go-gitea/gitea/pull/24248#issuecomment-1518131032 and below, we should evaluate converting and optimizing all avatars to WEBP. Currently all input formats (WEBP,PNG,JPG,GIF) end up as PNG during the avatar processing which squares them.

Notably, animation via APNG or WEBP input would be nice to have as well because WEBP can do animation, so even APNG should be convertable. Currently this avatar convertion strips APNG animation, would be nice if it can be preserved.

Related old issue: https://github.com/go-gitea/gitea/issues/18907

[silverwind]

Also, trying to upload a animated WEBP image currently gives this error, likely because the conversion code can not deal with animated WEBP correctly:

[zeripath]

Squaring

hmm... as far as I can see the squaring process is unrelated to the choice of format.

If you take a look at the code for Prepare:

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/modules/avatar/avatar.go#L47-L86

The squaring happens because of:

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/modules/avatar/avatar.go#L64-L84

Scaling up

Similarly the increase in size occurs because of the call to resize.

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/modules/avatar/avatar.go#L84

Exporting as png

Exporting as png appears to occur in two places:

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/services/repository/avatar.go#L50

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/services/user/user.go#L265

These would both need to change to use webp.

We could probably change the code in generateRandomAvatar to use webp too:

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/models/repo/avatar.go#L51

(It does not look like:

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/tests/integration/user_avatar_test.go#L55

would need to change)

Animation

Unfortunately the golang x/image/png library does not support animation. There has been a feature request:

https://github.com/golang/go/issues/53364

but no action has been taken.

There are other libraries that use cgo to bind to libwebp and should provide animation support but we would need to change from using image.Decode to some other function and it would need to detect if there is animation present.

Gitea at present will probably just take the first frame of an animated gif. So I think animation is going to take a bit of work.

Issues with Scaling and Resize

As far as I can see resize.Resize (and in some cases cutter.Crop) will convert the avatar into image.RGBA which decompresses the image and may make it quite large - taking up memory. This does not seem quite ideal.

So I think what we need to do is to rethink about what the avatar code is supposed to do.

1. Should we be doing any of this resizing and cropping at all? Should we simply be enforcing our max height and width and storing like that?
2. If we do resizing/cropping should we only resize/crop when the image is bigger than the avatar size?
3. Should we store as webp?
4. Should we look at being able to store animations?

[silverwind]

Should we be doing any of this resizing and cropping at all? Should we simply be enforcing our max height and width and storing like that?

I don't think we need to square or crop. Browser rendering can and should take care of letterboxing the image. We can certainly ensure this in our rendering. If other sites that embed gitea avatars assume our avatars are always square, they will need to fix it themselves.

Should we store as webp?

Yes, browser support is good enough that we can store a well-compressed webp.

Should we look at being able to store animations?

Animations are nice to have, but not a requirement for me. Happy to defer this to https://github.com/golang/go/issues/53364.

[sosasees]

Should we look at being able to store animations?

this is not needed. animated avatars can be fun on a casual website, but on a serious website like Gitea they would be just annoying.

[silverwind]

Should we be doing any of this resizing

I guess resizing could be considered above a certain pixel count. Let's say a lazy user directly uploads a photo taken from a digital camera which is let's say 50 Megapixels. We'd definitely want to downsize that to not destroy rendering performance and bandwidth consumption for other users.

I would say we could use a Megapixel constant above which to downsize the image to something more suitable for web consumption. Maybe make it 1 Megapixel, which is a 1000x1000 image that is suitable for fine rendering on 4dppx devices down to 250px width/height, which I think is close to avatars rendered on user pages.

[zeripath]

OK looking at encoding directly to webp we can't use x/image/webp and would need to use something that wraps the libwebp C library or write a PR to x/image/webp to do the encoding.

The first option I came across: https://github.com/kolesa-team/go-webp requires that I install libwebp-dev on my system - this would likely cause a significant issue with compiling on Windows.

PATCH for this option - would need changes to builders to install libwebp-dev though

```patch diff --git a/go.mod b/go.mod index 24765df18..e9e71a27e 100644 --- a/go.mod +++ b/go.mod @@ -218,6 +218,7 @@ require ( github.com/josharian/intern v1.0.0 // indirect github.com/kevinburke/ssh_config v1.2.0 // indirect github.com/klauspost/pgzip v1.2.6 // indirect + github.com/kolesa-team/go-webp v1.0.4 // indirect github.com/kr/pretty v0.3.1 // indirect github.com/kr/text v0.2.0 // indirect github.com/libdns/libdns v0.2.1 // indirect diff --git a/go.sum b/go.sum index 62c01c690..269d46d55 100644 --- a/go.sum +++ b/go.sum @@ -792,6 +792,8 @@ github.com/klauspost/pgzip v1.2.5/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQ github.com/klauspost/pgzip v1.2.6 h1:8RXeL5crjEUFnR2/Sn6GJNWtSQ3Dk8pq4CL3jvdDyjU= github.com/klauspost/pgzip v1.2.6/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs= github.com/kljensen/snowball v0.6.0/go.mod h1:27N7E8fVU5H68RlUmnWwZCfxgt4POBJfENGMvNRhldw= +github.com/kolesa-team/go-webp v1.0.4 h1:wQvU4PLG/X7RS0vAeyhiivhLRoxfLVRlDq4I3frdxIQ= +github.com/kolesa-team/go-webp v1.0.4/go.mod h1:oMvdivD6K+Q5qIIkVC2w4k2ZUnI1H+MyP7inwgWq9aA= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg= @@ -1334,6 +1336,7 @@ golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EH golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= +golang.org/x/image v0.0.0-20210628002857-a66eb6448b8d/go.mod h1:023OzeP/+EPmXeapQh35lcL3II3LrY8Ic+EFFKVhULM= golang.org/x/image v0.7.0 h1:gzS29xtG1J5ybQlv0PuyfE3nmc6R4qB73m6LUUmvFuw= golang.org/x/image v0.7.0/go.mod h1:nd/q4ef1AKKYl/4kft7g+6UyGbdiqWqTP1ZAbRoV7Rg= golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= diff --git a/models/repo/avatar.go b/models/repo/avatar.go index a76a94926..ac961989a 100644 --- a/models/repo/avatar.go +++ b/models/repo/avatar.go @@ -6,7 +6,6 @@ package repo import ( "context" "fmt" - "image/png" "io" "net/url" "strings" @@ -16,6 +15,9 @@ import ( "code.gitea.io/gitea/modules/log" "code.gitea.io/gitea/modules/setting" "code.gitea.io/gitea/modules/storage" + + "github.com/kolesa-team/go-webp/encoder" + "github.com/kolesa-team/go-webp/webp" ) // CustomAvatarRelativePath returns repository custom avatar file path. @@ -48,7 +50,12 @@ func generateRandomAvatar(ctx context.Context, repo *Repository) error { repo.Avatar = idToString if err := storage.SaveFrom(storage.RepoAvatars, repo.CustomAvatarRelativePath(), func(w io.Writer) error { - if err := png.Encode(w, img); err != nil { + options, err := encoder.NewLossyEncoderOptions(encoder.PresetDefault, 75) + if err != nil { + log.Error("Unable to create encoder options: %v", err) + return err + } + if err := webp.Encode(w, img, options); err != nil { log.Error("Encode: %v", err) } return err diff --git a/services/repository/avatar.go b/services/repository/avatar.go index 74e5de877..9c6dd777a 100644 --- a/services/repository/avatar.go +++ b/services/repository/avatar.go @@ -6,7 +6,6 @@ package repository import ( "context" "fmt" - "image/png" "io" "strconv" "strings" @@ -16,6 +15,9 @@ import ( "code.gitea.io/gitea/modules/avatar" "code.gitea.io/gitea/modules/log" "code.gitea.io/gitea/modules/storage" + + "github.com/kolesa-team/go-webp/encoder" + "github.com/kolesa-team/go-webp/webp" ) // UploadAvatar saves custom avatar for repository. @@ -47,7 +49,12 @@ func UploadAvatar(ctx context.Context, repo *repo_model.Repository, data []byte) } if err := storage.SaveFrom(storage.RepoAvatars, repo.CustomAvatarRelativePath(), func(w io.Writer) error { - if err := png.Encode(w, *m); err != nil { + options, err := encoder.NewLossyEncoderOptions(encoder.PresetDefault, 75) + if err != nil { + log.Error("Unable to create encoder options: %v", err) + return err + } + if err := webp.Encode(w, *m, options); err != nil { log.Error("Encode: %v", err) } return err diff --git a/services/user/user.go b/services/user/user.go index d52a2f404..1b379c025 100644 --- a/services/user/user.go +++ b/services/user/user.go @@ -6,7 +6,6 @@ package user import ( "context" "fmt" - "image/png" "io" "time" @@ -25,6 +24,9 @@ import ( "code.gitea.io/gitea/modules/storage" "code.gitea.io/gitea/modules/util" "code.gitea.io/gitea/services/packages" + + "github.com/kolesa-team/go-webp/encoder" + "github.com/kolesa-team/go-webp/webp" ) // RenameUser renames a user @@ -262,7 +264,12 @@ func UploadAvatar(u *user_model.User, data []byte) error { } if err := storage.SaveFrom(storage.Avatars, u.CustomAvatarRelativePath(), func(w io.Writer) error { - if err := png.Encode(w, *m); err != nil { + options, err := encoder.NewLossyEncoderOptions(encoder.PresetDefault, 75) + if err != nil { + log.Error("Unable to create encoder options: %v", err) + return err + } + if err := webp.Encode(w, *m, options); err != nil { log.Error("Encode: %v", err) } return err ```

https://github.com/chai2010/webp appears to wholesale import the libwebp source - which should work but we'd have to keep on top of ensuring that it's up to date.

https://git.sr.ht/~jackmordaunt/go-libwebp is a Ccgo port of libwebp to go - which could work without cgo - however, the library is marked experimental so is questionable.

[silverwind]

https://github.com/chai2010/webp sounds like the best bet to me, but it still needs CGO, right? I thought we could get rid of CGO eventually, so I doubt that adding a new CGO dependency will be worth it.

I guess as a first step, we can go and alter the resizing/cropping mechanism as discussed while still outputting well compressed PNGs, which in turn would solve https://github.com/go-gitea/gitea/issues/8972.

[wxiaoguang]

I guess introducing a full featured webp/png converter would bloat Gitea not just a little ....

[silverwind]

I guess introducing a full featured webp/png converter would bloat Gitea not just a little

If I recall correctly, gitea already converts all uploaded avatars to PNG, so we do already feature the PNG encoder. Problem is just that the encoding is not doing any compression I think, resulting in unnecessarly huge avatar image sizes.

[wxiaoguang]

IMO CGO is somewhat better than CCGO at the moment, CCGO is not that popular and users will be locked in it (there is a stale PR: "Pure Go SQLIite", IIRC it also uses CCGO ....)

Of course CGO is not ideal either, it affects generated executable.

A new brief idea: introduce sidecar/external-plugin mechanism: provide a "GiteaImageConvert" server, if users need it, they could deploy it with Gitea together, then Gitea will use it to convert images. If such mechanism can be made general and stable enough, the CustomAvatarServer/PagesServer and more features could be integrated with Gitea while keeping Gitea itself as simple as possible.

[silverwind]

A new brief idea: introduce sidecar/external-plugin mechanism: provide a "GiteaImageConvert" server, if users need it, they could deploy it with Gitea together, then Gitea will use it to convert images. If such mechanism can be made general and stable enough, the CustomAvatarServer/PagesServer and more features could be integrated with Gitea while keeping Gitea itself as simple as possible.

I'd certainly not see this necessary for the avatar encoder. png encode is in the stdlib so will be forever stable.

[wxiaoguang]

The problem is webp ......

[silverwind]

Yes, webp is another topic, but i don't think it's one to invest too much time into currently as the size benefits over PNG aren't that great if both are compressed well. Sooner or later, a proper pure golang webp encoder will emerge and then we will use it.

[wxiaoguang]

Problem is just that the encoding is not doing any compression I think, resulting in unnecessarly huge avatar image sizes.

After reading the code, I think png.Encode already does default compression.

png.Encode -> writeIDATs -> e.writeImage(e.bw, e.m, e.cb, levelToZlib(e.enc.CompressionLevel))

The CompressionLevel defaults to 0, aka DefaultCompression -> zlib.DefaultCompression

But, the affect is not good?

[silverwind]

I didn't test this, but if https://github.com/go-gitea/gitea/issues/8972#issuecomment-1542513735 is to be believed, something may be wrong, either the upsizing (that we should remove and only downsize above 1 megapixel) or the compression would be my guess.

[sosasees]

we should downsize to 1024px² instead of 1000px² so that the downsized images could be rendered slightly faster.

citation: Make Better Textures, The 'Power Of Two' Rule & Proper Image Dimensions (this is for games but i think it applies to computer images everywhere)

[silverwind]

we should downsize to 1024px² instead of 1000px² so that the downsized images could be rendered slightly faster.

Good idea, that makes the downsize threshold 1048576 pixels then.

[zeripath]

OK now I have a PR that will make avatars webp we can look at the resize/crop code.

Its helpful first to understand what we already have:

func Prepare(data []byte) (*image.Image, error) { ... }

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/modules/avatar/avatar.go#L47-L86

1. We reject if the image is greater than MaxWidth, MaxHeight:

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/modules/avatar/avatar.go#L52-L57

Default values are:

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/modules/setting/picture.go#L17-L18

This rejection is important because resize and crop will create RGBA images - so those could be huge byte slices.

2. If we are not a square image we crop to a square at the centre of the image:

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/modules/avatar/avatar.go#L64-L82

3. We resize the avatar to the constant AvatarSize:

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/modules/avatar/avatar.go#L84

https://github.com/go-gitea/gitea/blob/23ae939ef3ef03848038372de21490725855b5f9/modules/avatar/avatar.go#L26

So we are currently downsizing/upsizing everything to 290 x 290 pixels.

---

Proposal

1. Drop the crop code portion
2. Only resize if imgCfg.Width > 290 || imgCfg.Height > 290
3. Let maxDim = max(imgCfg.Width, imgCfg.Height)
4. Resize to imgCfg.Width/maxDim*290, imgCfg.Height / maxDim * 290

[silverwind]

2. Only resize if imgCfg.Width > 290 || imgCfg.Height > 290
3. Resize to imgCfg.Width/maxDim290, imgCfg.Height / maxDim 290

I see 290 was chosen because it is the size of the biggest rendered avatar on the user page. But we should actually take into account Avatar.RenderedSizeFactor for this resize, so that would with the default value of 3 make the size 870 above which to resize and to which resize to.

Other points sound good.

[wxiaoguang]

-> Improve avatar uploading / resizing / compressing #24653

JPG/PNG/APNG/WEBP all works (for most cases)