some characters not displaying right on new site

[jywarren]

encoding issues

[btbonval]

Jeff added some special characters to the following wiki page: http://publiclab.org/wiki/sandbox

The node table and the node_revisions table, which houses the wiki test page, both use UTF-8. They should support the test characters.

mysql> SHOW CREATE TABLE node;
...
) ENGINE=MyISAM AUTO_INCREMENT=7964 DEFAULT CHARSET=utf8 |
...
mysql> SHOW CREATE TABLE node_revisions;
...
) ENGINE=MyISAM AUTO_INCREMENT=13904 DEFAULT CHARSET=utf8 |
...

It looks like the experimental text did not input into the database, as they are shown with literal ? in the latest revision.

mysql> SELECT * FROM node_revisions WHERE nid = 721 ORDER BY timestamp DESC LIMIT 1;
+-----+-------+-----+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------+-----+------------+--------+
| nid | vid   | uid | title   | body                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | teaser | log | timestamp  | format |
+-----+-------+-----+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------+-----+------------+--------+
| 721 | 12203 |   1 | Sandbox | A page for experimentation, etc.

##'Dashboard' "And this"

????????????????

<a onClick="$('#text').toggle()">there can only be one of these toggles on a page</a>
<div id="text">Hello World!</div>

<a class="toggle" data-toggle="target-area">other Toggle</a> 

<div id="target-area"> 
How about this? 
</div> 

<div class="btn-group">
  <a href="#" class="btn">Main button</a>
  <a class="btn dropdown-toggle" data-toggle="dropdown" href="#"><span class="caret"></span></a>
  <ul class="dropdown-menu">
    <li>First item</li>
    <li class="divider"></li>
    <li>Second item</li>
  </ul>
</div>

<a class="toggle" data-toggle="target-area">Toggle</a>

<div id="target-area">
How about this?
</div> |        |     | 1366209482 |      1 |
+-----+-------+-----+---------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------+-----+------------+--------+
1 row in set (0.00 sec)

[btbonval]

On the real server, there are 16 ?'s in the database. On my cloned database, I replaced 4 ?s with 5 random characters:

UPDATE node_revisions SET body=REPLACE(body,'????','ÇéñðƔ') WHERE vid=12203;

In my database, I see:

mysql> SELECT body FROM node_revisions WHERE vid=12203;
...
##'Dashboard' "And this"

ÇéñðƔÇéñðƔÇéñðƔÇéñðƔ

<a onClick="$('#text').toggle()">
...

Yet the web page shows 25 ?'s, even in the view source.

The problem is not into or out of the database, but into and out of Rails.

[btbonval]

In my cloned environment, database.yml had database encodings of "ascii".

production:
  adapter: mysql2
  username:
  password:
  encoding: ascii
  database: publiclaboratory

development:
  adapter: mysql2
  username: bryan
  password:
  encoding: ascii
  database: publiclaboratory

I tried "utf-8" which raised an invalid type error. Then I tried "utf8" and the wiki now renders the appropriate characters on my local clone. I was able to edit the wiki, insert additional characters throughout the page, and submit that successfully: the resulting page renders as expected.

The database.yml file on the plots2 system is configuring for "ascii" encoding on all three database entries.

database.yml.example updated in 54fb9d7123099db662123e8c90cd17c78f2128f1

The production server will need the changes made to database.yml. This might not be a complete solution, but it will help.