Javascript is served as text/plain instead of text/javascript

morgunl2 commented 4 years ago

Describe the bug The web-server waitress which in the default configuration is used in development mode returns local static *.js with content type text/plain. In addition to response header X-Content-Type-Options: nosniff this behaviour breaks the local javascript totally (browsers refuse to execute such js code due to wrong MIME type).

To Reproduce Just try to serve the static javascript files

Expected behavior The javascript files should be served with content type text/javascript

mmerickel commented 4 years ago

What code is serving the static files? Can you provide a reproducible example? This is almost certainly not pyramid or waitress' fault but something else configured in your app.

morgunl2 commented 4 years ago

Running under Windows, without venv. The fresh installation reproduces the same problem.

Please find the working project below.

source.zip

stevepiercy commented 4 years ago

Please paste the relevant code. I will not download a .zip from a stranger.

morgunl2 commented 4 years ago

I cannot tell you what code is relevant. This is zipped fresh cookiecutter project with the only addtion in template -- link to the static js file.

Below is my layout.mako The only addition is the links to chosen.js The javascript file you can find here: https://harvesthq.github.io/chosen/ But I believe, the same problem reproduces with any javascript static file.

<!DOCTYPE html>
<html lang="${request.locale_name}">
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="description" content="pyramid web application">
    <meta name="author" content="Pylons Project">
    <link rel="shortcut icon" href="${request.static_url('pyramid_test:static/pyramid-16x16.png')}">

    <title>Cookiecutter Starter project for the Pyramid Web Framework</title>

    <!-- Bootstrap core CSS -->
    <link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">

    <!-- Custom styles for this scaffold -->
    <link href="${request.static_url('pyramid_test:static/theme.css')}" rel="stylesheet">

    <!-- HTML5 shiv and Respond.js IE8 support of HTML5 elements and media queries -->
    <!--[if lt IE 9]>
      <script src="//oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js" integrity="sha384-0s5Pv64cNZJieYFkXYOTId2HMA2Lfb6q2nAcx2n0RTLUnCAoTTsS0nKEO27XyKcY" crossorigin="anonymous"></script>
      <script src="//oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js" integrity="sha384-ZoaMbDF+4LeFxg6WdScQ9nnR1QC2MIRxA1O9KWEXQwns1G8UNyIEZIQidzb0T1fo" crossorigin="anonymous"></script>
    <![endif]-->

    <!--MY ADDITION TO TEST CONTENT-TYPE-->
    <script src="${request.static_url('pyramid_test:static/chosen/chosen.jquery.min.js')}"></script>
    <script src="${request.static_url('pyramid_test:static/chosen.jquery.min.js')}"></script>
    <!--MY ADDITION TO TEST CONTENT-TYPE-->
  </head>

  <body>

    <div class="starter-template">
      <div class="container">
        <div class="row">
          <div class="col-md-2">
            <img class="logo img-responsive" src="${request.static_url('pyramid_test:static/pyramid.png') }" alt="pyramid web framework">
          </div>
          <div class="col-md-10">
            ${ next.body() }
          </div>
        </div>
        <div class="row">
          <div class="links">
            <ul>
              <li><i class="glyphicon glyphicon-cog icon-muted"></i><a href="https://github.com/Pylons/pyramid">Github Project</a></li>
              <li><i class="glyphicon glyphicon-globe icon-muted"></i><a href="https://webchat.freenode.net/?channels=pyramid">IRC Channel</a></li>
              <li><i class="glyphicon glyphicon-home icon-muted"></i><a href="https://pylonsproject.org">Pylons Project</a></li>
            </ul>
          </div>
        </div>
        <div class="row">
          <div class="copyright">
            Copyright &copy; Pylons Project
          </div>
        </div>
      </div>
    </div>

    <!-- Bootstrap core JavaScript
    ================================================== -->
    <!-- Placed at the end of the document so the pages load faster -->
    <script src="//code.jquery.com/jquery-1.12.4.min.js" integrity="sha256-ZosEbRLbNQzLpnKIkEdrPv7lOy9C27hHQ+Xp8a4MxAQ=" crossorigin="anonymous"></script>
    <script src="//maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
  </body>
</html>

stevepiercy commented 4 years ago

    <!--MY ADDITION TO TEST CONTENT-TYPE-->
    <script src="${request.static_url('pyramid_test:static/chosen/chosen.jquery.min.js')}"></script>
    <script src="${request.static_url('pyramid_test:static/chosen.jquery.min.js')}"></script>
    <!--MY ADDITION TO TEST CONTENT-TYPE-->

Note that you have the same filename in two locations. That's one issue.

Did you save the file with a BOM? If so, don't do that.

I'm running on macOS. What OS are you using?

I tried to reproduce the issue with using a fresh pyramid-cookiecutter-starter using mako with no backend.
I saved this file https://raw.githubusercontent.com/harvesthq/chosen-package/master/chosen.jquery.min.js into my project's static directory.
I added <script src="${request.static_url('mymako:static/chosen.jquery.min.js')}"></script> just before the closing </body> tag. See why this is a good idea. Additionally jQuery must load before any jQuery plugins.
It returns Content-Type: application/javascript

The only differences are where you placed the <javascript> tag in your template and the project name. However I would not name anything with test in it because that is a reserved word for running tests via pytest.

digitalresistor commented 4 years ago

X-Content-Type-Options is not something that is set by Pyramid/WebOb/Waitress, the cookie cutter also does not set this option at all.

Do you have a Github repo with the full source code please? The default cookie cutter also does not use a session and thus would not be sending a session cookie in the response either. So this is not just a cookie cutter.

Now for the JavaScript files being served with the wrong content-type:

What version of Windows are you running? What version of Python?

in a python interpreter (the same one you use to run pyramid), please run the following code:

import mimetypes

# Default that is returned after importing mimetypes
print(mimetypes.guess_type("chosen.jquery.min.js", strict=False))

# Reset to defaults
mimetypes.init(files=None)
print(mimetypes.guess_type("chosen.jquery.min.js", strict=False))

# Completely reset and don't load anything but built-in map
mimetypes.init(files=None)
mimetypes.init(files=[])
print(mimetypes.guess_type("chosen.jquery.min.js", strict=False))

And provide that output.

morgunl2 commented 4 years ago

Here is the code: https://github.com/morgunl2/pyramid_test

Sorry for session cookies and other headers, it seems, that browser cache was not clean enough, didn't mean to confuse.

Thank all of you for your answers. It seems, I have some local issues with my python installation. Windows 7, x64, Python 3.6. I'll try to localize the problem and send the solution.

Python 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]

Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import mimetypes
   ...:
   ...: # Default that is returned after importing mimetypes
   ...: print(mimetypes.guess_type("chosen.jquery.min.js", strict=False))
   ...:
   ...: # Reset to defaults
   ...: mimetypes.init(files=None)
   ...: print(mimetypes.guess_type("chosen.jquery.min.js", strict=False))
   ...:
   ...: # Completely reset and don't load anything but built-in map
   ...: mimetypes.init(files=None)
   ...: mimetypes.init(files=[])
   ...: print(mimetypes.guess_type("chosen.jquery.min.js", strict=False))
   ...:
('text/plain', None)
('text/plain', None)
('text/plain', None)

digitalresistor commented 4 years ago

It'd be interesting to see:

import mimetypes
import pprint

pprint.pprint(mimetypes.types_map)

since it looks like your mimetypes registry is missing a bunch of entries that would be expected.

stevepiercy commented 4 years ago

I cannot replicate the issue locally on macOS using your repo.

platform darwin -- Python 3.7.3, pytest-5.4.1, py-1.8.1, pluggy-0.13.1

I think you're right that something funky is going on with your Python install. Did you create a virtual environment and install your project into it?

digitalresistor commented 4 years ago

@stevepiercy the issue is that his mimetypes are broken. It's not something that a virtualenv would help or fix.

morgunl2 commented 4 years ago

It's very weird.

Windows 7, 64 bit. Clean Python 3.7.7 installation.

In [3]: import pprint
In [4]: pprint.pprint(mimetypes.types_map)
...
'.jpeg': 'image/jpeg',
'.jpg': 'image/jpeg',
'.js': 'application/javascript',
'.json': 'application/json',
'.ksh': 'text/plain',
...

But:

In [5]: mimetypes.guess_type("chosen.js")
Out[5]: ('text/plain', None)
In [6]: mimetypes.guess_type("http://example.com/chosen.js")
Out[6]: ('text/plain', None)

The next experiment. Another PC, Windows 10, 64 bit. Clean Python 3.7.7 installation.

pprint.pprint(mimetypes.types_map)
...
 '.jpeg': 'image/jpeg',
 '.jpg': 'image/jpeg',
 '.js': 'application/javascript',
 '.json': 'application/json',
 '.ksh': 'text/plain',
...

But:

In [5]: mimetypes.guess_type("chosen.js")
Out[5]: ('application/javascript', None)
In [6]: mimetypes.guess_type("http://example.com/chosen.js")
Out[6]: ('application/javascript', None)

morgunl2 commented 4 years ago

Problem localized and solved. mimetypes.py read data from Windows registry during the initialization.

def init(files=None):
    global suffix_map, types_map, encodings_map, common_types
    global inited, _db
    inited = True    # so that MimeTypes.__init__() doesn't call us again

    if files is None or _db is None:
        db = MimeTypes()
        if _winreg:
            db.read_windows_registry()

        if files is None:
            files = knownfiles
        else:
            files = knownfiles + list(files)
    else:
        db = _db

    for file in files:
        if os.path.isfile(file):
            db.read(file)
    encodings_map = db.encodings_map
    suffix_map = db.suffix_map
    types_map = db.types_map[True]
    common_types = db.types_map[False]
    # Make the DB a global variable now that it is fully initialized
    _db = db

Also, my registry looks like:

Deleting this key solved the problem:

C:\Python37\python.exe -m mimetypes chosen.min.js
type: application/javascript encoding: None

Thank you for your help! I hope it will help someone someday.

digitalresistor commented 4 years ago

I wonder if there is an application that you installed or something that changed that from the default. I am glad it is solved though! Thanks for following up!

Pylons / pyramid

Javascript is served as text/plain instead of text/javascript #3570