cambridge-cares / TheWorldAvatar

A knowledge-graph-based digital twin of the world.
https://theworldavatar.io/
MIT License
84 stars 25 forks source link

Encoding issues with non-ascii characters in units #513

Open markushofmeister opened 1 year ago

markushofmeister commented 1 year ago

There are reported issues with encoding of special characters using Blazegraph, i.e. Blazegraph claims to use utf-8 encoding while actually using iso-8859-1 (details: https://github.com/blazegraph/database/issues/224).

This causes issues when instantiating non-ascii strings, e.g. symbols for units. This is relevant for the AverageQuareMetrePriceEstimation as well as PropertyValueEstimation agents where we use £ as symbol for poundSterling; however, there are also unit symbols defined within the ontology of units of measure, which are affected, e.g. °, °C, µg

Proper encoding (potential solution see here) shall be ensured when 1) uploading data into Blazegraph (@markushofmeister) as well as when 2) retrieving data for visualisation (@mdhillman)

This should also fix the issue observed earlier, e.g. AirQuality agent #227

mdhillman commented 1 year ago

This should now be resolved in v3.3.4 of the DTVF (already hosted online). Please note however, that you will need to add a charset="UTF-8" tag to the import statement of the DTVF script in your index.html file (the examples have been updated to reflect this).

markushofmeister commented 1 year ago

I just observed the following behaviour for the latest version of the DTVF (but also applicable to previous versions):

When referencing the online JS and CSS resources in the index.html file, the visualisation renders special characters nicely (figure 1):

<script src="https://kg.cmclinnovations.com/cdn/dtvf/3.7.0/dtvf.min.js" charset="UTF-8"></script>
<link href="https://kg.cmclinnovations.com/cdn/dtvf/3.7.0/dtvf.min.css" rel="stylesheet" />

However, when copying the JS and CSS file content from https://kg.cmclinnovations.com/cdn/dtvf/ into local files and referencing them, again styling issues exist (figure 2)

<script src="./dtvf.min.js" charset="UTF-8"></script>
<link href="./dtvf.min.css" rel="stylesheet" />

image image