Sending Voltage Phasor Values as received in an UDP socket: Huge Delay

[Pratyush-das]

Hi all, I am essentially using the RandomPMU code to send values to a PDC. The values the PMU sends are not random, but are coming from another source through UDP. The UDP socket receives the values (24 bytes/packet) at a rate of 1packet per ms. i.e. 1000 FPS. However, the pyPMU instance is supposed to send whatever value it has at that moment and then move on. I am seeing that even when I stop sending data, the PMU continues to send data to the PDC for minutes. Why is that? What am I doing wrong here?

## Import necessary libraries
import math
import numpy as np
import socket
import struct
from synchrophasor.frame import ConfigFrame2
from synchrophasor.pmu import Pmu
import time
# from synchrophasor.frame import ConfigFrame2
# from time import time
from synchrophasor.frame import CommandFrame
from datetime import datetime

##
"""
randomPMU will listen on ip:port for incoming connections.
After request to start sending measurements - random
values and values from OPAL RT model. It will also receive RR from OPAL RT 
and configure frame rate accordingly.
"""

if __name__ == "__main__":

    ## Configure socket for Phasor data ##

    UDP_IP = "10.10.114.22"
    UDP_PORT = 9201
    sock_ph = socket.socket(socket.AF_INET,  # Internet
                         socket.SOCK_DGRAM)  # UDP
    sock_ph.bind((UDP_IP, UDP_PORT))
    print("socket bound, waiting for data...")

    ## Configure socket for RR Value ##

    UDP_PORT2 = 600  # OPAL sends RR to this port
    sock_rr = socket.socket(socket.AF_INET,  # Internet
                            socket.SOCK_DGRAM)  # UDP
    sock_rr.bind((UDP_IP, UDP_PORT2))  # binding is required for receiving not sending
    sock_rr.settimeout(1 / 1000)  # you dont want to wait forever a data that might not come
    flag = False

    ## Configure socket for RR Value send to CA algorithm ##
    UDP_PORT3 = 9901
    sock_rr_send = socket.socket(socket.AF_INET,  # Internet
                                  socket.SOCK_DGRAM)  # UDP
    ## PMU configuration ##

    pmu = Pmu(ip="10.10.114.22", port=1410) #original port config
    ## connect to another VM 10.10.114.23
    #pmu = Pmu(ip="10.10.114.22", port=1410)
    pmu.logger.setLevel("DEBUG")

    cfg = ConfigFrame2(1,  # PMU_ID
                       1000000,  # TIME_BASE
                       1,  # Number of PMUs included in data frame
                       "PDAS_bus1",  # Station name
                       1410,  # Data-stream ID(s)
                       (True, True, True, True),  # Data format - POLAR; PH - REAL; AN - REAL; FREQ - REAL;
                       3,  # Number of phasors(was 3)
                       1,  # Number of analog values
                       1,  # Number of digital status words
                       ["VA", "VB", "VC", "ANALOG1", "BREAKER 1 STATUS",
                        "BREAKER 2 STATUS", "BREAKER 3 STATUS", "BREAKER 4 STATUS", "BREAKER 5 STATUS",
                        "BREAKER 6 STATUS", "BREAKER 7 STATUS", "BREAKER 8 STATUS", "BREAKER 9 STATUS",
                        "BREAKER A STATUS", "BREAKER B STATUS", "BREAKER C STATUS", "BREAKER D STATUS",
                        "BREAKER E STATUS", "BREAKER F STATUS", "BREAKER G STATUS"],  # Channel Names
                       [(0, "v"), (0, "v"),
                        (0, "v")],  # Conversion factor for phasor channels - (float representation, not important)
                       [(1, "pow")],  # Conversion factor for analog channels
                       [(0x0000, 0xffff)],  # Mask words for digital status words
                       50,  # Nominal frequency
                       1,  # Configuration change count
                       25)  # Rate of phasor data transmission) 50 default here

    pmu.set_configuration(cfg)
    pmu.set_header("Random PMU: Modified by Pratyush")

    pmu.run()

    ############## C37.118 Reporting Rates #########################

    rr_list = [5, 10, 25, 50, 100]  # list of available RR as per standard

    flag = False
    while True:
        #print(pmu.cfg2,type(pmu.cfg2))
        cframe = pmu.cfg2
        pmu_rr = cframe.get_data_rate()
        pmu_rr = str(pmu_rr).encode("ascii") # int to bytes e.g. 50 to b'50'
        sock_rr_send.sendto(pmu_rr, (UDP_IP, UDP_PORT3))
        #print(pmu_rr)
        try:
            aware_rr = sock_rr.recv(1024)
            #drr = int(struct.unpack('d', aware_rr[8:16])[0]) # if coming from OPAL
            del_rr = int(aware_rr.decode("utf-8")) # the received value is +1 or -1
            #drr = crr + del_rr
            drr=del_rr
            flag = True
        except socket.timeout:  # to catch the error essentially it will only store drr if there is something to store and set the flag
            pass
        if pmu.clients:
            ## Socket Values
            ## Listen to the socket

            raw = sock_ph.recv(1024)
            #print(raw)
            mag = struct.unpack('d', raw[8:16])[0]
            # print("mag =",mag,type(mag))
            angle = struct.unpack('d', raw[16:24])[0]
            # print("angle =",angle,type(angle))
            header = struct.unpack('d', raw[0:8])[0]
            # print("header =",header,type(header))
            phasor = (mag, angle)
            #print(phasor)
            #print(Vol_A, type(Vol_A))
            #socket.send_string("Received ")
            #print("Random Integer Received from %s" % Vol_A)  # display received string
            #  Send reply back to client
            # socket.send_string("Received from client 1 at :" + str(datetime.now()))
            Vol_A=raw
            VA = float(mag)
            phi_A = float(angle)
            VB = VA
            phi_B = phi_A+(math.pi) * 2 / 3
            VC = VA
            phi_C = phi_A-(math.pi) * 2 / 3
            pmu.send_data(phasors=[(VA,phi_A),(VB,phi_B),(VC,phi_C)],analog=[9.91],digital=[0x0001])
            #print([(VA,phi_A),(VB,phi_B),(VC,phi_C)])
        if flag: # if it received something it changes the RR
            pmu.set_data_rate(drr)
            time.sleep(1/ 1000)
            flag=False

    pmu.join()

[sstevan]

Hey @Pratyush-das, we have not managed to dig into the UDP support. Contributor @pledna might help with this.

[poledna]

Hello, sorry i'm late to the party, This isn't as much an issue in the UDP, looking at the code we can clearly see that it is utilized a TCP PMU and not UDP as shown below

## PMU configuration ##

pmu = Pmu(ip="10.10.114.22", port=1410) #original port config

it is using a TCP socket and not UDP, now that i got my UDP parenting out of the way :) , as it is using a TCP socket with the default data_rate of 30 FPS I'd recommend changing the DataRate to 1000FPS because of the delay present in here:

https://github.com/iicsys/pypmu/blob/66e6c495a91efb8a6018061ff7955a6654b1404d/synchrophasor/pmu.py#L347-L348

it MIGHT be it, you're receiving data at ONE THOUSAND FRAMES and sending it at :

rr_list = [5, 10, 25, 50, 100] # list of available RR as per standard

at a maximum of 100 FPS and then there is another sleep on that last if, so it seems that you're receiving 1000FPS and sending only 100 so increasing the queue A LOT. but I'm not sure I'd check all sleeps (and if you commented out that sleep i commented in another issue) and the ReportingRates. Also I'd try making a minimal example with receiving all the 1000 FPS and sending them at 1000FPS without sleeps/delay.

So to reminisce queue theory, your faucet is producing 1000fps and your sink is sinking 100FPS so when you stop after 1s you'd 900 frames in the queue unsent, and that would be sent (in this case) until there is nothing in the queue anymore. and 900 frames at 100FPS is 9 s so you can see that if you sample a minute where this is going, a looooong time posting data after closing.

Also as a question why are you designing a pmu with a 1000fps frame rate? That's A LOT of data to be transmitted, it doesn't feel post-processing by your code you already receive all data processed, then why such a high frame-rate? (as food for thought) I know that the Recursive algorithm can produce a lot of data but should you send EVERY single frame it produces? Did you consider a post processing filter?