[Bug?]: Assets with names containing international characters don't work in prod on cloudflare-pages

rvlzzr commented 3 months ago

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Current behavior 😯

Assets named with international characters do not load in prod, at least when using the cloudflare-pages preset.

Expected behavior 🤔

Assets named with international characters work consistently in dev and prod using any preset.

Steps to reproduce 🕹

Steps:

Image files src/assets/서아.webp and src/assets/seoa.webp exist
Import them and use them in img tags
Note that both images show up in dev, but when deployed to prod, at least on cloudflare, the seoa.webp image loads correctly while the 서아.webp image is broken.

I haven't yet tested this using other presets so I'm not sure whether or not this issue is exclusive to cloudflare-pages.

Context 🔦

I am trying to import images such as 서아.webp named with international characters and use them in img tags.

Your environment 🌎

No response

pheki commented 3 months ago

I think this is actually an issue with cloudflare pages. I'm not using SolidJS, but I managed to reproduce it yesterday when deploying a pages app with different wrangler versions, a directory I have was getting uploaded with a unicode character on its filename (ú) changed to 0xFFFD (�).

Both deployment and GET requests with url-encoding were working fine for months until this last deployment (which was the first deployment in a few weeks). Luckily when I downgraded wrangler to 3.67.1 it worked for me, but I think it was just a coincidence as it later failed with 3.67.1 and now it's working even with 3.73.0.

rvlzzr commented 2 months ago

It works correctly using Astro. https://anydrop.com/ is just a basic astro app deployed with wrangler pages deploy dist. The images with and without international characters both load correctly as expected. I don't see why it should be possible for Astro but not for SolidStart.

export default defineConfig({
  output: 'server',
  adapter: cloudflare()
});

---
import Seoa from '../assets/geurimjaseoa.webp';
import 서아 from '../assets/서아.webp';
---

<html lang="en">
  <head>
    <meta charset="utf-8"/>
    <link rel="icon" type="image/svg+xml" href="/favicon.svg"/>
    <meta name="viewport" content="width=device-width"/>
    <meta name="generator" content={Astro.generator}/>
    <title>Astro</title>
  </head>
  <body>
    <h1>Seoa</h1>
    <img src={Seoa.src} alt="Seoa"/>
    <h1>서아</h1>
    <img src={서아.src} alt="서아"/>
  </body>
</html>

pheki commented 2 months ago

Edit: It turns out that the issue I'm pointing in this comment is completely unrelated to this one. Sorry for the confusion.

Hey, I probably wasn't clear enough, but from my experience this is an issue that happens sometimes when uploading files to Cloudflare Pages. I'm not using solid, but I have an app that worked correctly for months, except that on some of my last deployments my file names got incorrectly changed. My file path changed from something like assets/próprio/cropped.png to assets/pr�prio/cropped.png`, however after retrying the deployment some times it eventually worked as expected.

My guess is that you got unlucky with your Solid deployment, but lucky with Astro.

To confirm that, I suggest that you try verifying the files that got uploaded to Cloudflare Pages.

Open the Cloudflare Dashboard on https://dash.cloudflare.com
Click on Workers & pages
Select your project
Find the deployment you did with Cloudflare pages that was failing to load the images and click on "View details"
On the "Assets uploaded" tab, expand the assets directory (or wherever is the image) and check the filename. If my guess is correct, you should find ��.webp
If your dist/build directory is still there, compare to the image name there to see if SolidStart generated the correct image name.

Another thing you can try is deploying again now and see if it happens to work (you can deploy to a preview branch by adding e.g. --branch development)

My guess is that there's some Cloudflare server that's misconfigured or running different software/version that's screwing up the encoding.

ryansolid commented 2 months ago

Hmm.. this is lower level than we are handling since we don't serve the assets. Nitro is where I'd probably look to see if there is a similar issue for cloudflare-pages.

rvlzzr commented 2 months ago

I just tried Nuxt and it works correctly there as well. I don't see Nitro doing anything wrong here.

rvlzzr commented 2 months ago

I notice that in dist/_routes.json Nuxt does wildcard excludes:

{
  "version": 1,
  "include": [
    "/*"
  ],
  "exclude": [
    "/_nuxt/*",
    "/favicon.ico",
    "/robots.txt"
  ]
}

Whereas the _routes.json generated by Solid-Start lists each excluded file individually. I'm not sure what piece is responsible for generating _routes.json but could it be modified to use wildcard excludes like Nuxt? Perhaps the individual excludes for files containing these characters aren't encoded the way Cloudflare expects here?

ryansolid commented 2 months ago

That's a good hint. Vinxi uses Nitropack directly which probably differs from how Nuxt does.

pheki commented 2 months ago

It seems like the issue I hit and pointed out above with corrupted uploads is unrelated to this one, sorry for the confusion.

I managed to reproduce this issue and I can confirm that in this case the assets were uploaded correctly, have the same encoding when comparing the asset in "assets uploaded", the exclude in the "invocation routes" and the <img> tag, but the function is still called when trying to get that image.

Perhaps the individual excludes for files containing these characters aren't encoded the way Cloudflare expects here?

Yeah, it does seem like the issue happens when matching the request path with the exclude, either because of a different encoding or because Cloudflare doesn't handle international characters as expected in their matching system. I haven't been able to find any related issue.

solidjs / solid-start