"If no extension is specified assume html"

[d4h0]

Hi,

Thanks for creating devserver! :)

The code contains the following:

    let extension = path.extension().and_then(OsStr::to_str);

    // If no extension is specified assume html
    let path = if extension == None {
        path.with_extension("html")
    } else {
        path.to_owned()
    };
    let extension = extension.unwrap_or("html");

I think that is a pretty unexpected behavior (at least for me ;)), because – as far as I know – URLs don't need to have an extension.

I'm working on a static-site-generator and I'd prefer it if my URLs don't contain an unnecessary ".html" extension. Would it, therefore, be possible to remove this code?

[kettle11]

Hi @d4h0!

The reason the ".html" extension is appended is to enable the scenario you described, if I'm understanding the scenario correctly.

That code lets you navigate to http://localhost:8080/test and devserver will look for the "test.html" file.

If the code you mentioned were removed devserver would return a 404 unless you navigated to exactly http://localhost:8080/test.html with the .html added.

That code should only cause a problem if you actually want to load a file that has no file extension, but probably that is an uncommon use case.

Was devserver not working as you expected?

[d4h0]

That code should only cause a problem if you actually want to load a file that has no file extension, but probably that is an uncommon use case.

That was what I actually tried to archive... ;)

However, it seems this approach isn't recommended in my situation. I'm creating a static-site-generator and URLs without ".html" extension lead to pages that have the wrong content type (which I found out after testing with several servers).

With Apache or other full web servers, it's possible to rewrite the URL so it doesn't contain any extension. But in this case, the original file has an extension from which the server can derive the right content type.

That code lets you navigate to http://localhost:8080/test and devserver will look for the "test.html" file.

I still think devserver should keep the original filename and just return an error because changing the filename is very unexpected :)

[kettle11]

After thinking about this and doing a little research I think I agree that appending .html is not always appropriate. It may be that an image (or other file) is expected to be served instead of .html when no file extension is specified.

The current approach of defaulting to appending .html is not unprecedented. The very popular http-server for NodeJS does the same thing, but it has a flag to change the default file extension.

Other servers seem to provide configuration files for what to do when no extension is specified but I'd like devserver to do generally what a user would expect by default. Many static website generators generate URLs without an extension, so devserver should just work for them without requiring additional flags. That was the original motivation for appending ".html".

Perhaps a better solution is if no file extension is specified to search the directory for a file with that name and serve the first one found regardless of extension. This would likely be correct in most cases. If that default behavior would cause issue for users of devserver then a flag could be introduced to disable it.

What do you think?

[d4h0]

I would do the following:

No file extension is added (because this is very unexpected and the user most likely links to a URL without a file extension).

Because no file extension can be used to derive a content type, devserver should default to 'text/plain' (which should be displayed in the browser as normal text).

Ideally, a warning is printed to the terminal (in case the user is wondering why the content isn't rendered properly).

Because the behavior of web servers with files that don't have a file extension seems pretty undefined (I tested 3 servers, and all used a different content type), ideally there would be an option to set the content type of files that don't have a file extension.

Then, if the server the user deploys to, defaults to 'text/html' (and which is what this user wants), the user can configure devserver like this, and everything works.

This seems the best approach to me.

(Unfortunately, the server I'm deploying to (Netlify) defaults to 'text/plain', so now I'm just using a folder with an 'index.html' file in it. Then I can uses '/foo/' as URL which is okay, I guess).

[kettle11]

There are two separate but connected issues here:

1. What path should devserver look for when receiving a request without an extension?

Often people use URLs like this if they're trying to make a website with "pretty URLs" without the trailing slash like: www.example.com/my-blog-post.

Typically assuming the extension is .html is appropriate here, as devserver already does. I do not think it is common that servers accomplish this by looking for a file on the disk without .html and serving it as an HTML file.

The current default assumption of .html seems fine for now. But if there were a common use case, or a use case without a workaround, adding a flag would be OK.

Maybe if someone had a directory of generated files they didn't want to add extensions to but they wanted to serve locally for some reason? But I think it's best to wait until someone actually raises a concern instead of trying to anticipate their need.

2. What to do when the file actually has no extension?

I think your proposal to serve the file as text/plain and log a warning is appropriate. If a file without an extension exists that should be the file served. I can see how it'd be frustrating to have devserver silently fail to find the file because it's searching for a file with the .html extension.

---

Based on the points you've raised what I'm inclined to do is the following:

1. If a file with no extension exists serve that as text/plain and log a warning.
2. Otherwise look for a file with a .html extension and serve that if found. If it's not found 404

This approach by default handles the most common cases, but it will no longer silently fail as it did for you.

If a scenario comes up later that requires a flag then that can be added then.

[d4h0]

Typically assuming the extension is .html is appropriate here, as devserver already does. I do not think it is common that servers accomplish this by looking for a file on the disk without .html and serving it as an HTML file.

To be honest, I'm pretty sure that is exactly what they would do (if they receive a request for "/foo" they will look for "/foo" on the filesystem, not for "/foo.html").

If there is "/foo/index.html" and a client requests "/foo" many web servers will serve "index.html" from "/foo" (which is incorrect (the correct URL is "/foo/") and in the past could lead to problems with search engines because of duplicate content). But that's different from serving "/foo.html" for "/foo".

However, I guess many people configure their web server like this (which is easy).

So, after some thinking, I came to the conclusion that your current approach makes sense (for a web development server). Files without extensions lead to many problems (for example syntax highlighting), so having "*.html" files locally is useful.

What I would suggest, however, is to document this behavior. Because, at least to me, this was unexpected, and many who could benefit from this feature probably will not try it, or read the source code.

Based on the points you've raised what I'm inclined to do is the following:

Yes, this seems like a good approach :)

[kettle11]

@d4h0 You're completely right that it should be properly documented.

Once I implement the above changes I will add proper documentation and at that point I'll close this issue.

Thank you again for raising this issue! It wasn't something I had given proper thought and this discussion helped a lot.

[d4h0]

Sounds good :)

[kettle11]

This commit implements the behavior we discussed: https://github.com/kettle11/devserver/commit/f94c8d2b1074364b853dd23b4b7ce922e3552255

A warning is logged if a file is served without an extension. If the file is served without an extension it's served with a media (MIME) type of application/octet-stream. The server will also serve a .html file if there's a file with the correct name and .html extension.

Thanks for talking through this one!

[d4h0]

That's great, thank you!