Unicode errors when using external PostgreSQL database

kmo@… originally submitted this as ticket:923

Keywords: unicode, pg8000

When testing the standard development build of CalendarServer-7.0 with the only config change that it connects to an external PostgreSQL database, an error message is logged from twext.enterprise.jobs.jobitem regarding a value that must not be of type unicode. Error message attached.

The problem seems to be located in the function calendarserver.tap.util.getDBPool where connectionFactory is instantiated from pgServiceFromConfig in case of an internal database and from DBAPIConnector for external databases. pgServiceFromConfig takes care of translating utf8 strings fetched from the database into a byte stream (type str), however DBAPIConnector seems to be missing that translation?

How to reproduce:

Get a clean build of CalendarServer-7.0

Modify the standard configuration file by adding a section similar to this:

   <key>DBType</key>
    <string>postgres</string>
    <key>DatabaseConnection</key>
    <dict>
      <key>endpoint</key>
      <string>tcp:localhost</string>
      <key>database</key>
      <string>caldav</string>
      <key>user</key>
      <string>caldav</string>
      <key>password</key>
      <string>password</string>
    </dict>

Start the server, add a single event to a calendar.

This problem is probably related to the newly introduced use of the pg8000 module.

apple / ccs-calendarserver

Unicode errors when using external PostgreSQL database #401