BUG: coords_from_node_geodata with "ignore_zero_length" causes index mismatch

[rbolgaryn]

if ignore_zero_length (default True), the coords array skips the entries with zero length lines but the array with indices elements_with_geo still includes such lines. If from and to bus of a line have the same coordinates, the lengths of coords and elements_with_geo do not match, and an error is raised elsewhere.

It seems like the variable elements_with_geo should be updated to consider the fact that coords skips a line. Alternatively, the parameter ignore_zero_length should be dropped outright, and the user should be warned if there are any zero-lengths entries in coords.

[dlohmeier]

I remember that problems could arise with these zero-length branches. It should of course be considered for the elements_with_geo, so I suggest the following code:

def coords_from_node_geodata(element_indices, from_nodes, to_nodes, node_geodata, table_name,
                             node_name="Bus", ignore_zero_length=True):
    """
    Auxiliary function to get the node coordinates for a number of branches with respective from
    and to nodes. The branch elements for which there is no geodata available are not included in
    the final list of coordinates.

    :param element_indices: Indices of the branch elements for which to find node geodata
    :type element_indices: iterable
    :param from_nodes: Indices of the starting nodes
    :type from_nodes: iterable
    :param to_nodes: Indices of the ending nodes
    :type to_nodes: iterable
    :param node_geodata: Dataframe containing x and y coordinates of the nodes
    :type node_geodata: pd.DataFrame
    :param table_name: Name of the table that the branches belong to (only for logging)
    :type table_name: str
    :param node_name: Name of the node type (only for logging)
    :type node_name: str, default "Bus"
    :param ignore_zero_length: States if branches should be left out, if their length is zero, i.e.\
        from_node_coords = to_node_coords
    :type ignore_zero_length: bool, default True
    :return: Return values are:\
        - coords (list) - list of branch coordinates of shape (N, (2, 2))\
        - elements_with_geo (set) - the indices of branch elements for which coordinates wer found\
            in the node geodata table
    """
    have_geo = np.isin(from_nodes, node_geodata.index.values) \
        & np.isin(to_nodes, node_geodata.index.values)
    elements_with_geo = element_indices[have_geo]
    fb_with_geo, tb_with_geo = from_nodes[have_geo], to_nodes[have_geo]
    x_from_values = node_geodata.x[fb_with_geo].values
    y_from_values = node_geodata.y[fb_with_geo].values
    x_to_values = node_geodata.x[tb_with_geo].values
    y_to_values = node_geodata.y[tb_with_geo].values
    ignored_elem = np.full(len(x_from_values), False)
    if ignore_zero_length:
        ignored_elem = (x_from_values == x_to_values) & (y_from_values == y_to_values)
        if np.any(ignored_elem):
            logger.info("%s %s will be dropped, as their from and to %s coordinates are identical."
                        % (table_name + "s", elements_with_geo.index[ignored_elem], node_name))
    coords = [[(x_from, y_from), (x_to, y_to)] for x_from, y_from, x_to, y_to
              in zip([x_from_values[~ignored_elem], y_from_values[~ignored_elem],
                      x_to_values[~ignored_elem], y_to_values[~ignored_elem]])]
    elements_without_geo = set(element_indices) - set(elements_with_geo)
    if len(elements_without_geo) > 0:
        logger.warning("No coords found for %s %s. %s geodata is missing for those %s!"
                       % (table_name + "s", elements_without_geo, node_name, table_name + "s"))
    elements_with_geo = elements_with_geo[~ignored_elem]
    return coords, elements_with_geo

Do you have any test dataset? Could you test the code snippet? Unfortunately, there are no tests implemented in pandapower for this.

[robcalon]

I encountered two small issues when running this code:

1. elements_with_geo is already an index
2. The zipped items should not be in list

This results in the following snippet worked for my case, it would be nice to see this issue closed.

def coords_from_node_geodata(element_indices, from_nodes, to_nodes, node_geodata, table_name,
                             node_name="Bus", ignore_zero_length=True):
    """
    Auxiliary function to get the node coordinates for a number of branches with respective from
    and to nodes. The branch elements for which there is no geodata available are not included in
    the final list of coordinates.

    :param element_indices: Indices of the branch elements for which to find node geodata
    :type element_indices: iterable
    :param from_nodes: Indices of the starting nodes
    :type from_nodes: iterable
    :param to_nodes: Indices of the ending nodes
    :type to_nodes: iterable
    :param node_geodata: Dataframe containing x and y coordinates of the nodes
    :type node_geodata: pd.DataFrame
    :param table_name: Name of the table that the branches belong to (only for logging)
    :type table_name: str
    :param node_name: Name of the node type (only for logging)
    :type node_name: str, default "Bus"
    :param ignore_zero_length: States if branches should be left out, if their length is zero, i.e.\
        from_node_coords = to_node_coords
    :type ignore_zero_length: bool, default True
    :return: Return values are:\
        - coords (list) - list of branch coordinates of shape (N, (2, 2))\
        - elements_with_geo (set) - the indices of branch elements for which coordinates wer found\
            in the node geodata table
    """
    have_geo = np.isin(from_nodes, node_geodata.index.values) \
        & np.isin(to_nodes, node_geodata.index.values)
    elements_with_geo = element_indices[have_geo]
    fb_with_geo, tb_with_geo = from_nodes[have_geo], to_nodes[have_geo]

    x_from_values = node_geodata.x[fb_with_geo].values
    y_from_values = node_geodata.y[fb_with_geo].values
    x_to_values = node_geodata.x[tb_with_geo].values
    y_to_values = node_geodata.y[tb_with_geo].values

    ignored_elem = np.full(len(x_from_values), False)

    if ignore_zero_length:
        ignored_elem = (x_from_values == x_to_values) & (y_from_values == y_to_values)
        if np.any(ignored_elem):
            logger.info("%s %s will be dropped, as their from and to %s coordinates are identical."
                        % (table_name + "s", elements_with_geo[ignored_elem], node_name))

    coords = [[(x_from, y_from), (x_to, y_to)] for x_from, y_from, x_to, y_to
              in zip(x_from_values[~ignored_elem], y_from_values[~ignored_elem],
                      x_to_values[~ignored_elem], y_to_values[~ignored_elem])]

    elements_without_geo = set(element_indices) - set(elements_with_geo)

    if len(elements_without_geo) > 0:
        logger.warning("No coords found for %s %s. %s geodata is missing for those %s!"
                       % (table_name + "s", elements_without_geo, node_name, table_name + "s"))
    elements_with_geo = elements_with_geo[~ignored_elem]
    return coords, elements_with_geo