search

hassanakbar4 / foobar-git

tractive test
0 stars 0 forks source link

Make the datatracker timezone fields timezone aware #3329

Open hassanakbar4 opened 3 years ago

hassanakbar4 commented 3 years ago

component_Datatracker: Infrastructure type_defect | by rjsparks@nostrum.com

(This has been tracked outside the ticketing system for some time - moving it here for consistency)

From Henrik:

The MySQL databaase on which the datatracker currently runs does not support timezone-aware datetime fields. A workaround for this now exists within the Django framework, permitting the code to use timezone-aware datetimes to some extent, but this came in several years after the original transition to Django, and until now the codebase has continued to use timezone-naive datetime values.

Looking at moving to a database (PostgreSQL) with timezone support for datetime fields, it turned out that the database engine transition would be substantially easier if the change to use timezone-aware datetimes was done first.

This turned out to be a quite substantial bit of work; the count of changes to the codebase is now above 2100. The test suite now passes cleanly, and a page-by-page A-B comparison is in progress. This is turning up additional places where there are discrepancies that need code or template changes.

In addition to the code and template changes, the datetime fields in the database, most of which currently are saved according to the server local time (PST8PDT, or America/Los_Angeles), while some are in meeting local time for various IETF meetings. These must be converted to UTC, to be in line with the assumptions made by Django when timezone-aware field support on MySQL is activated. This involves value-by-value conversion of about 3.95 million fields, in a safe manner, i.e., insuring that we don’t double convert any fields in case the conversion run is interrupted and has to be re-run. Where models are subclassed, fields in the parent class also must be protected from double conversion during processing of the child models. All in all, this also has required a number of iterations and validations of the conversion process.

At the beginning of the year, the migration took about 2 hours on a high-end macbook pro. We had scheduled downtime for conversion early in the year, but cancelled it when we found more issues to address. Since then, the branch for this project has been languishing, with trunk moving along at a rapid pace. We will need to bring it back up to trunk, making any necessary additional adjustments given the meetings that have taken place, and the new fields that have been introduced in the interim.

Issue migrated from trac:3329 at 2021-09-27 12:29:12 +0500

hassanakbar4 commented 3 years ago

@hassanakbar4 commented

See https://trac.ietf.org/trac/ietfdb/browser/branch/tzaware, and in particular these diffs .

hassanakbar4 commented 3 years ago

@hassanakbar4 changed status from new to accepted