Extend RIPE import with route information

For the tuency project we want to extend the RIPE importer with routing information so that IP addresses can be mapped to related ASNs. In particular, we want to import the ripe route files (ripe.db.route.gz and ripe.db.route6.gz) which contain a mapping from network addresses to ASNs.

Overview

My plan for implementing this in intelmq-certbund-contact is

Start a new branch for this: wip-import-routes
Work out the datamodel and implement the import
Importing the route information is optional as for now it's nonly needed by tuency.

Data model

The table for this will likely be the following:

CREATE TABLE route_automatic (
    route_automatic_id SERIAL PRIMARY KEY,
    address CIDR NOT NULL,
    asn BIGINT NOT NULL,
    LIKE automatic_templ INCLUDING ALL,

    UNIQUE (address, asn, import_source)
);

The routing data from RIPE does have cases where an address is associated with multiple ASNs, so the UNIQUE constraint cannot be just (address, import_source) as one might perhaps expect.

Optional route import

Since not all users need the routing information, the actual import is optional and has to be explicitly activated by a command line flag.

The changes to the data model and e.g. the ripe_download script do not have top be optional, though. The ripe.db.inetnum.gz alone is much bigger than the route files so downloading the route files even if they're not needed should not be much of a problem in practice.

Also, having the route_automatic table in all databases makes dealing with future updates easier as there are fewer differences between the installiations that need to be taken into account.

Intevation / intelmq-certbund-contact