subnet.allocated_ips is incorrect

[asadoughi]

In my local sqlite in-memory setup subnet.allocated_ips returns 0 or 1 and not the expected count of allocated ips.

[asadoughi]

Specifically, looking at it's use in ipam.py:QuarkIpam._choose_available_subnet. Try creating a subnet of 192.168.1.1/29 and more than 8 ports. It fails, but for the wrong reason (exception raised in allocate_ip_address instead of _choose_available_subnet).

[asadoughi]

Reproduction steps

1. Create network
2. Create subnet (192.168.1.1/29)
3. Create mac_address_range (quark specific)
4. Create port
5. Repeat step 4 8 more times (9 ports)

You'll have to do debugging in ipam.py:QuarkIpam._choose_available_subnet.

[asadoughi]

This if statement is removed in the ip_policies pull request since the _choose_available_subnet function should not choose a subnet that does not have IPs available, assuming subnet.allocated_ips had been the correct value.

https://github.com/jkoelker/quark/blob/master/quark/ipam.py#L148

[asadoughi]

It might be that sqlite cannot perform this kind of query:

SELECT quark_subnets.tenant_id AS quark_subnets_tenant_id,
       quark_subnets.id AS quark_subnets_id,
       quark_subnets.created_at AS quark_subnets_created_at,
       quark_subnets.name AS quark_subnets_name,
       quark_subnets.network_id AS quark_subnets_network_id,
       quark_subnets._cidr AS quark_subnets__cidr,
       quark_subnets.first_ip AS quark_subnets_first_ip,
       quark_subnets.last_ip AS quark_subnets_last_ip,
       quark_subnets.ip_version AS quark_subnets_ip_version,
       quark_subnets.next_auto_assign_ip AS quark_subnets_next_auto_assign_ip,
       quark_subnets.enable_dhcp AS quark_subnets_enable_dhcp,
       quark_subnets.tag_association_uuid AS quark_subnets_tag_association_uuid,
       count(quark_ip_addresses.subnet_id) AS count
FROM quark_subnets
LEFT OUTER JOIN quark_ip_addresses ON quark_subnets.id = quark_ip_addresses.subnet_id AND quark_ip_addresses._deallocated = 0
WHERE quark_subnets.network_id = '4dec86ab-657b-40bf-b828-1dfa6f2c7b98'                                                                                                        
GROUP BY quark_ip_addresses.tenant_id, quark_ip_addresses.id, quark_ip_addresses.created_at, quark_ip_addresses.address_readable, quark_ip_addresses.address,
      quark_ip_addresses.subnet_id, quark_ip_addresses.network_id, quark_ip_addresses.version, quark_ip_addresses._deallocated, quark_ip_addresses.deallocated_at
ORDER BY count DESC;

I'm switching over to MySQL to test that theory.

[asadoughi]

Query performs similarly on MySQL: 0 with no IPs allocated, 1 if 1 (or more!?) IPs are allocated.

[asadoughi]

I forgot repoze in my api-paste.ini, that might be part of it -- still debugging.

[asadoughi]

Ah, the main issue is the GROUP BY part of the SELECT is on elements that are distinct to an IP and that defeats what we are trying to accomplish with the subnet IP allocation count.

Have to re-do the following function:

def subnet_find_allocation_counts(context, net_id, **filters):
    query = context.session.query(models.Subnet,
                                  sql_func.count(models.IPAddress.subnet_id).
                                  label('count')).\
        outerjoin(models.Subnet.allocated_ips).\
        filter(models.Subnet.network_id == net_id)
    if "version" in filters:
        query = query.filter(models.Subnet.ip_version == filters["version"])
    query = query.group_by(models.IPAddress)
    query = query.order_by("count DESC")
    return query