search

googlearchive / simian

Simian is an enterprise-class Mac OS X software deployment solution. Google App Engine hosted server, with a client powered by the Munki open-source project.
Apache License 2.0
332 stars 51 forks source link

Unicode characters in pkginfo description may cause package modification notification emails to fail #31

Open mholttech opened 8 years ago

mholttech commented 8 years ago

I'm getting an error 500 when I copy/paste a pkginfo file. Uploading the file works perfectly

Here is the log:

2015-10-15 20:34:48.592 /admin/package 500 261ms 0kb instance=0 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36 module=default version=a56a3dc98d87
100.43.220.122 - mholt [15/Oct/2015:20:34:48 -0700] "POST /admin/package HTTP/1.1" 500 225 - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36" "rlc-simian.appspot.com" ms=261 cpu_ms=53 cpm_usd=0.000025 instance=0 app_engine_release=1.9.27 
W 2015-10-15 20:34:48.418
No admins defined! Configure admins in Admin Tools -> ACL Groups.
W 2015-10-15 20:34:48.446
No admins defined! Configure admins in Admin Tools -> ACL Groups.
E 2015-10-15 20:34:48.590
'ascii' codec can't encode character u'\xae' in position 726: ordinal not in range(128)
Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/apps/s~rlc-simian/a56a3dc98d87.387476931731189739/simian/mac/admin/__init__.py", line 153, in handle_exception
    super(AdminHandler, self).handle_exception(exception, debug_mode)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~rlc-simian/a56a3dc98d87.387476931731189739/simian/mac/admin/package.py", line 170, in post
    self.UpdatePackageInfoFromPlist(create_new=True)
  File "/base/data/home/apps/s~rlc-simian/a56a3dc98d87.387476931731189739/simian/mac/admin/package.py", line 380, in UpdatePackageInfoFromPlist
    self.NotifyAdminsOfPackageChangeFromPlist(plist_xml)
  File "/base/data/home/apps/s~rlc-simian/a56a3dc98d87.387476931731189739/simian/mac/admin/package.py", line 283, in NotifyAdminsOfPackageChangeFromPlist
    main_body = str(plist.GetXml(indent_num=2))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xae' in position 726: ordinal not in range(128)

Here is the plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>_metadata</key>
    <dict>
        <key>created_by</key>
        <string>admin</string>
        <key>creation_date</key>
        <date>2015-10-16T02:58:46Z</date>
        <key>munki_version</key>
        <string>2.3.1.2535</string>
        <key>os_version</key>
        <string>10.10.3</string>
    </dict>
    <key>autoremove</key>
    <false/>
    <key>catalogs</key>
    <array>
        <string>unstable</string>
    </array>
    <key>description</key>
    <string>Adobe® Flash® Player is a cross-platform browser-based application runtime that delivers uncompromised viewing of expressive applications, content, and videos across screens and browsers.</string>
    <key>display_name</key>
    <string>Adobe Flash Player</string>
    <key>installed_size</key>
    <integer>17140</integer>
    <key>installer_item_hash</key>
    <string>b5eca33f72b1b419a7f9754b53cc8780b03e351520f189a43986a9e1f2aa22fa</string>
    <key>installer_item_location</key>
    <string>AdobeFlashPlayer-19.0.0.207.dmg</string>
    <key>installer_item_size</key>
    <integer>16364</integer>
    <key>minimum_os_version</key>
    <string>10.5.0</string>
    <key>name</key>
    <string>AdobeFlashPlayer</string>
    <key>package_path</key>
    <string>Install Adobe Flash Player.app/Contents/Resources/Adobe Flash Player.pkg</string>
    <key>receipts</key>
    <array>
        <dict>
            <key>installed_size</key>
            <integer>17140</integer>
            <key>packageid</key>
            <string>com.adobe.pkg.FlashPlayer</string>
            <key>version</key>
            <string>19.0.0.207</string>
        </dict>
    </array>
    <key>unattended_install</key>
    <true/>
    <key>uninstall_method</key>
    <string>removepackages</string>
    <key>uninstallable</key>
    <true/>
    <key>version</key>
    <string>19.0.0.207</string>
</dict>
</plist>
mholttech commented 8 years ago

forgot to mention as well that even though I get the error 500, the new package is created in Simian

juofmgoblue commented 8 years ago

Thanks for the report, stacktrace, and offending plist!

The package is still uploaded because all that is failing is the method call to email admins about the new upload: https://github.com/google/simian/blob/master/src/simian/mac/admin/package.py#L380

If I had to quickly guess, I'd say the issue is the unicode registered trademark symbols in: Adobe® Flash®

Can you try removing those to see if that uploads as expected?

johnrandolph commented 8 years ago

yeah, I think we'll need a bug fix there to either handle unicode properly when sending the email, or do a str.encode('ascii', 'ignore') from the input package to drop the unicode.

obviously the latter is a bad solution to varying degrees, losing unicode (R) is one thing, losing an entire non-latin alphabet description or something isn't really an option.

On Fri, Oct 16, 2015 at 8:53 AM, Justin McWilliams <notifications@github.com

wrote:

Thanks for the report, stacktrace, and offending plist!

The package is still uploaded because all that is failing is the method call to email admins about the new upload: https://github.com/google/simian/blob/master/src/simian/mac/admin/package.py#L380

If I had to quickly guess, I'd say the issue is the unicode registered trademark symbols in: Adobe® Flash®

Can you try removing those to see if that uploads as expected?

— Reply to this email directly or view it on GitHub https://github.com/google/simian/issues/31#issuecomment-148708688.

mholttech commented 8 years ago

I just tested without the ® and it did resolve the 500 error.

juofmgoblue commented 8 years ago

Thanks for the confirmation, this seems like a relatively straightforward fix then.