Output stream initialization erase outputstreammeasurement content

[korunekosama]

Hello ! I'm trying to create an output stream (IEE C37.118-2005) containing a virtual device. This virtual device has no phasors but several measurements produced by a custom action adapter. When adding the device to the output stream, all the related measurements are well include in the stream (I can see the records in the outputstreammeasurement table).

With OpenPDC 2.7.4 : when I try to initialize the output stream, I got the following errors in the console : Failed to execute startup data operation "PhasorProtocolAdapters.dll [PhasorProtocolAdapters.CommonPhasorServices::PhasorDataSourceValidation()]" due to exception: Input string was not in a correct format. and [Action Adapter Collection] Failed to initialize adapter 100.1234-RESULTS-TEST: Input string was not in a correct format. Of course the output stream doesn't work properly (no data).

With OpenPDC 2.8.196 doing the same way : when I initialize the output stream, there's no error message ; but all the measurements are removed from the stream. There is no more records remaining in the outputstreammeasurement table.

Could you tell me if it's a bug ; or if an output stream can't be use to send measurements from a virtual device ?

Regards

[ritchiecarroll]

If you have defined the "structure" of the virtual device in the output stream, such as desired phasors and analogs, then you must also manually "map" measurements to the fields. You do this using a "signal reference" format. For example, for a virtual device called "SHELBY", the first analog value would have a signal reference of "SHELBY-AV1". You map measurements to a signal reference in the "Measurements" link associated with the output stream.

See here for more information on signal references: https://github.com/GridProtectionAlliance/openPDC/blob/master/Source/Documentation/wiki/Developers_About_the_Signal_Reference.md

[korunekosama]

Hello, Thank you for your reply.

This virtual device do not 'contain' somme phasors or analogs/digitals but calculated measurement type (which are timeseries of course) from a custom action adapter. These calculated values (which are present in the Measurement table and have their own dedicated Signal Reference value, like 'CALCULATEDITEM.PMU1.VALUE:I.MAG' for example) are well associated to the virtual device (they are well transmit when using the GEP or STTP protocol with an internal subscription from another server/PDC) ; but not with an output stream as I explain. I can had the virtual device to the output stream and measurements associated to these calculated values are automatically well inserted in the outputstreammeasurement table ; with the correct AdapterID refering to the device declared in the outputstreamdevice table. Tables outputstreamdeviceanalog / outputstreamdevicedigital and outputstreamdevicephasor are empty as I haven't any of this type of measurements associated to this virtual device.

The issue is they disapear from this table when I initialize the output stream for the first time.

Perhaps another way to describe the problem would be to know whether it is possible to transmit measurements that are calculated type from one PDC to another with an output stream ? Or the only way to do this would be the use of internal subscription (with STTP or GEP) ? This last solution has the major drawback that all the measurements from the PDC are then transferred to the remote one, including some that I do not wish to share with.

Regards

[StephenCWills]

The following code prepares the list of OutputStreamMeasurement records that will be removed from the OutputStream.

// Validate measurements associated with this output stream
foreach (DataRow outputStreamMeasurement in database.Connection.RetrieveData(database.AdapterType, $"SELECT * FROM OutputStreamMeasurement WHERE AdapterID = {adapterID} AND NodeID = {nodeIDQueryString}").Rows)
{
    // Parse output stream measurement signal reference
    deviceSignalReference = new SignalReference(outputStreamMeasurement.Field<string>("SignalReference"));

    // Validate that the signal reference is associated with one of the output stream's devices
    if (deviceAcronyms.BinarySearch(deviceSignalReference.Acronym, StringComparer.OrdinalIgnoreCase) < 0)
    {
        // This measurement has a signal reference for a device that is not part of the associated output stream, so we mark it for deletion
        measurementIDsToDelete.Add(outputStreamMeasurement.ConvertField<int>("ID"));
    }

    // Validate that the signal reference type is valid for an output stream
    if (validOutputSignalKinds.All(validSignalKind => deviceSignalReference.Kind != validSignalKind))
    {
        // This measurement has a signal reference type that is not valid for an output stream, so we mark it for deletion
        measurementIDsToDelete.Add(outputStreamMeasurement.ConvertField<int>("ID"));
    }
}

This is entirely based on the signal reference of the OutputStreamMeasurements. It's important to note that the OutputStreamMeasurement table has its own SignalReference field that overrides the signal reference of the corresponding input stream measurement. So you can use the openPDC Manager pages to update the signal reference on your output stream's measurements to make sure your measurements don't get removed. All you need to do is format your signal reference in a way that places it somewhere meaningful in the data frame.

The format of the signal reference is ACRONYM-ST#, where ACRONYM refers to the acronym of the virtual output stream device, ST is a valid signal type from the list below, and # is a one-based integer index of that measurement's position in the data frame.

Valid signal types:

• Angle: PA
• Magnitude: PM
• Frequency: FQ
• DfDt: DF
• Status: SF
• Analog: AV
• Digital: DV
• Quality: QF

[StephenCWills]

As for STTP/GEP, there are ways to restrict the number of measurements transmitted to the remote side, both from the publisher configuration and the subscriber configuration. Therefore, oversharing isn't necessarily a deal-breaker for those protocols.

[ritchiecarroll]

You can also setup a TLS-based subscription with STTP (or GEP) - this, unlike the internal subscription, has screens which "control" exactly which which points you would like to share from a publisher and of those points, which ones the subscriber would like to receive.

[korunekosama]

Hello, First of all thank you for your informative answers and your support !

Stephen, I understand that if I want my measurements not to be deleted by an internal validation process I have to rename all the SignalReference value in the outputstreamMeasurement table in order to include the name of the virtual device that I have associated to this output stream ; despite this association is already done with the AdapterID reference in this table. Because I have 3000+ measurements to include in this stream I would have to do this with an SQL UPDATE request ; not beeing sure what would be the 'name' of each measurement that would be received by OpenHistorian on the other side of the stream (it's important that these measurements keep the same original name and not the modified SignalReference one). Because it's a significant work to do to test this solution, I first tried the one mentionned by Ritchie.

I so made a TLS-based subscription first with STTP protocol, struggling a bit with the local and remote certificates configuration ; finally achieved it by copying / pasting the 'default' certificates openpdc.cer and openhistorian.cer on server where they were not originally. Then making the configuration off the measurements groups. When I initiated the subscriber part (OpenHistorian), an error message mentionned that the connection failed because the other part (OpenPDC Publisher) did not respond. I bypassed this step by changing the default connection port 6177 suggested by the create Autorization Request process to 6167 as it is declared in the openPDC.exe.Config ( / ConfigurationString) of the publisher server. I then had the new following error message, this time in the openPDCConsole of the OpenPDC publisher : [TLS!DATAPUBLISHER] Data publisher encountered an exception while receiving comm and channel data from client connection: Returned by WSARecv or WSARecvFrom to i ndicate the remote party has initiated a graceful shutdown sequence Indeed, on the side of the subscriber the console show the following informations : [10.100-OPENHISTORIAN] Attempting connection to tcp://10.100.100.10:6167... [10.100-OPENHISTORIAN] Attempting command channel connection to publisher... [10.100-OPENHISTORIAN] Connection established. [10.100-OPENHISTORIAN] Data subscriber command channel connection to publish er was established. [10.100-OPENHISTORIAN] Data subscriber command channel connection to publish er was terminated. No errors were shown in the subscriber OpenHISTORIAN console, but no data were received.

I finally changed the protocol from STTP to GEP in the subscribe device of OpenHistorian ; and it finally worked : the measurements (and only those that I want) are received. Well, not simple :o) I wonder why the STTP protocol didn't work, as the GEP does. I haven't found any parameter on the 'authorized subscribers' panel to specify the incoming requests type : only TLS or gateway are mentionned, but not GEP or STTP .... Maybe it is specify in the first subscribers frames, when they initiate contact with the publisher ?

As additional informations : OpenHistorian version (subscriber) : 2.7.278 OpenPDC version (publisher) : 2.8.196

Anyway, I managed to get the desired result; and would like to thank you again for your precious help.

Regards

[ritchiecarroll]

Might have been related to the port numbers - STTP uses its own ports - but congrats!