[Localhost Dev server] Images uploaded via File Adapter are broken

[DeadSuperHero]

Hey, Ben!

I'm in the process of testing out Bandwagon. So far, it's great, but it seems I have an old outstanding issue with my setup. Long story short: I'm using the file adapter for media attachments, and this is just a very simple localdev setup.

When I try to submit image attachments, Emissary just don't seem to push them through, and I end up with a lot of broken image links. I'm not totally sure what causes this, but I'm happy to share some context.

When I try to submit an image attachment, this comes up in the console:

Uncaught (in promise) no-response: no-response :: [{"url":"http://localhost/667946575fde7e7dc3a72407/attachments/66794bf63acc0bea7ff163c0"}]

at b.U (http://localhost:8080/workbox-1b51a4a5.js:1:22977)
at async b.q (http://localhost:8080/workbox-1b51a4a5.js:1:7612)

A subsequent message then states: GET http://localhost/667946575fde7e7dc3a72407/attachments/66794bf63acc0bea7ff163c0 net::ERR_FAILED

What's also interesting is that when I look at the GET requests, it seems that the image attachments served via the FILE adapter aren't themselves being served on port 8080? Instead, they just point to localhost...maybe that's a problem? They come up as (failed)net::ERR_FAILED and (failed)net::ERR_CONNECTION_REFUSED.

I haven't dug TOO much deeper yet...I'm not totally sure what causes this.

One really crazy thing, though, is that audio uploads work perfectly fine with Bandwagon?! Maybe the file adapter is just aggressive about what media types get uploaded? Not totally sure here.

[benpate]

It's strange that images would fail but audio works fine. I can definitely dig in deeper with you on this, but let's try something really basic first.

I recently updated Emissary to use FFmpeg for all of the media processing. There's just no way to do everything it does natively in Go, so that has to be installed on your machine to process images. So, can you confirm that FFmpeg is installed? Alternatively, if you're working with the pre-built Docker image, that should already include FFmpeg.

We'll work it out. I promise :)

[benpate]

Btw.. I may have a lead on this: if you have FFmpeg installed, it may still depend on the specific libraries included with your FFMpeg installation.

I have one server where FFmpeg is having trouble determining the original file type. I'm still investigating a fix, but am hopeful that I'll be able to post an update for this soon.

[benpate]

A little more progress on this. I'm willing to bet 15 "Imaginary Internet Points" that it's because of the VERSION of FFmpeg. On my machine, homebrew installs version 7.0, but the Docker apt-get is installing version 5.1.5.

I'm currently looking into how to get around this with Docker. It may be because of the Linux version we're using, but I'm not sure about that just yet. Docker is still pretty new to me, so I'm open to any advice or suggestions.

This is likely the same issue that's blocking my Bandwagon.fm rollout, so I'm committed to getting this fixed ASAP.

[dolanor]

I started playing with emissary finally, so I played with bandwagon as well to get a grasp of how it works. For now, I confirm having these issues with the docker compose version.

I'm quite ok with docker, so I'm gonna investigate as well right now.

[dolanor]

Ok,

So you use debian:bookworm-slim, which limits you to ffmpeg v5. If we add https://www.deb-multimedia.org for debian bookworm, we can get ffmpeg v6. Otherwise, we might have to just install a v7 into the image. It might be a PITA if the dependencies are too much of a hassle. I'll try locally with a ffmpeg v6, to see if it fixes things.

[dolanor]

With a ffmpeg v6, it still doesn't seem to work. In the logs, when I try to look at a photo album, it logs:

{
emissary_demo_server    |   "code": 404,
emissary_demo_server    |   "location": "handler.getBundle",
emissary_demo_server    |   "message": "Bundle not found",
emissary_demo_server    |   "timestamp": 1719786813,
emissary_demo_server    |   "details": [
emissary_demo_server    |       "stylesheet"
emissary_demo_server    |   ],
emissary_demo_server    |   "innerError": null

And I get the broken file icon instead of the images.

[dolanor]

I went too deep. It's just a broken link because the port is not in the URL. Adding :8080 in my case fixed it…

[dolanor]

So it seems .IconURL is set with the wrong host somehow. I don't know the codebase enough to know where is the mistake. I'll stop now. Hope you know where the problem @benpate :)

[benpate]

Both issues are true.

I originally installed FFmpeg using homebrew, and things worked great. But it looks like it's very difficult to get FFmpeg onto a Docker instance, and it does really want version 7 to work. I've made a workaround by starting with an FFmpeg Docker image, then installing Emissary onto that. It doesn't spark joy, but it seems to work for now. I'm posting that to "main" here in a bit -- again as a stopgap until someone with more Docker experience can create a "right way" to do this.

Ports are also an issue. By default, values like a person's profile photo use an absolute URL, but the port number isn't getting carried through. I think I can take care of this one, too, relatively quickly.

And, just a quick "thank you" to @dolanor who was able to speedrun the Docker troubleshooting (that took me a couple days) in just a few hours. You're awesome, and I really value your contributions. Hopefully you can provide some guidance on how to use Docker best in this situation.

I'll get my updates posted and then let's see how they work.

[dolanor]

You're very welcome :) Thanks for the kind words.

For docker, I think your solution is as good as any. We need a container with ffmpeg and emissary in it. Emissary is easy to deploy, because of 0 library dependencies. Which is not the case of ffmpeg.

Getting an image with ffmpeg v7 will be either:

• install an empty image, copy emissary on it and ffmpeg v7 with its dependencies. Not so easy
• install a slim debian stable image, add external debian repositories to get up to date v7. They won't be on debian stable, so it is gonna be dirty as well
• install a slim debian testing image, add external debian repositories to get up to date v7. It's a debian testing, but the attack surface is mostly emissary via the setup and normal port. And if there is an exploit on ffmpeg v7. I would say it's maybe the cleanest solution.
• or your solution

I'll send a PR once I've tested it has the correct ffmpeg version.

[benpate]

That's awesome. Thank you.

I've just confirmed that the Docker images are working correctly for me on production (along with fixes for the HTTP port on "local" domains).

You can pull the latest from the master branch to see this working. It should "just work" for user profile images, but you might have to re-upload thumbnail images for other content types.

Please let me know that this is working for you all, and I'll close this issue :)

[benpate]

An update on this. My new Docker builds are working well in production. I think it's safe to close this, but I'm still open to using another/better Docker image as a base in the future. When I get a chance I'll experiment with the ChainGuard images, which look very promising.

I really appreciate the insights from all of you on this thread. Please be loud about any other issues you have getting Emissary (or Bandwagon) running on your machines.