search

grnet / djnro

DjNRO hits the decks of eduroam database management
http://djnro.grnet.gr/
Other
10 stars 21 forks source link

eduroam database version 2 adaptation #62

Closed zmousm closed 4 years ago

zmousm commented 5 years ago

eduroam db v2 schema changes

[realm]

[institution]

[service_loc]

eduroam db v2 schema caveats

coordinates database modeling

djnro models changes

Contact

Address_i18n (new model)

Realm

Institution

InstitutionDetails

ServiceLoc

RealmData

djnro data migrations

djnro models cleanup

Realm

InstitutionDetails

ServiceLoc

TBA: forms, views, templates

ghalse commented 5 years ago

From mobility day, reporting on the recent GeGC meeting:

From September 2019, the OT will start naming and shaming people who've not upgraded to version 2 of the database format. From Q1 2020, version 1 will no longer be supported.

dpenezic commented 5 years ago

Short notice : a) mobile service_loc is for example bus, GPS coordinate cover bus line b) institution type change way of need to be print in finale file, but i dont see reason to be change in program solution at all c) collecting system know when you provide v1 or v2 in few ways , so no need to be cover via provided files

zmousm commented 5 years ago

a) mobile service_loc is for example bus, GPS coordinate cover bus line

Thanks, I never considered that.

c) collecting system know when you provide v1 or v2 in few ways , so no need to be cover via provided files

Except that the server has no way of knowing what version is requested.

vladimir-mencl-eresearch commented 4 years ago

Hi @zmousm , have you already started looking at implementing this?

Wondering whether to dive into it myself. Would you know whether this would also need changes to the underlying data model, or whether we have all the information required to construct the right format in a view?

Cheers, Vlad

zmousm commented 4 years ago

Hi @vladimir-mencl-eresearch. Yes I have (had), more than once. My patch set is half-baked so it is not pushed yet. I currently need to rebase on top of the final PY3 changes.

Changes to the models are necessary and unavoidable, as I tried to elaborate previously.

It would really help if you could cast your preference, providing arguments from a Django perspective, how to deal with the open questions and caveats, namely modeling coordinates.

zmousm commented 4 years ago

I had second thoughts about modeling coordinates, triggered after realizing that ServiceLoc deletion does not cascade to Coordinates deletion (this is how the FKs in the M2M through table work). I finally used a signal to work around that original annoyance, however I noticed how the reverse relations between Coordinates and ServiceLoc vs. Realm and InstitutionDetails are asymmetric and wondered how this could be simplified, without ab-using signals. I realized that a generic FK together with a OneToOneField pointing to itself (a linked list of sorts) would be enough to represent either a chain (for ServiceLoc) or a single set (for the other models) of coordinates. Something like this:

class Coordinates(models.Model):
    previous = models.OneToOneField("self", null=True, blank=True,
                                    related_name="next")
    content_type = models.ForeignKey(ContentType, blank=True, null=True)
    object_id = models.PositiveIntegerField(blank=True, null=True)
    content_object = fields.GenericForeignKey('content_type', 'object_id')

    class Meta:
        unique_together = ((..., 'previous', 'content_type', 'object_id'),)

The first instance in a chain (or the single instance otherwise) would get previous=None. Meta.unique_together would enforce the same constraints as the respective signals. The single set restriction, both initially/temporarily for ServiceLoc as well as for the other models, for would be enforced through a signal requiring previous=None. However such a linked list would be notoriously ineffective in terms of SQL queries, as it would typically require as many round-trips as the members of a chain (even if this might not be a big concern initially) and it would be complicated to handle with Django ORM. The better way to handle this is with a modified pre-order traversal tree with django-mptt. However the latter seemed somewhat too involved (at this stage) for this simple use case, especially considering that eventually we should be using GeoDjango (with PostGIS) to effectively handle such data.