New feature - centralise logging to TPP Log engine

[Saadi6]

Hi @gdbarron. Nice to see this project is still very much active!

Anyone using your libraries with multiple TPP servers may appreciate a method to Post informational/error messages to TPP's Log interface. TPP does log exit response from PowerShell scripts that are triggered by TPP engine itself but I find it useful to feed other pertinent information back into the logging engine while the script is executing, especially when interacting with other systems. If logged locally to disk, one has to hunt for the server which executed the script. This also has the added benefit of centralising enterprise-wide logging when TPP logs are being forwarded to a SIEM solution.

The method is POST Log. If an invalid value for ID is passed then the Event in the logs will not be matched to a component. For me, the params Component, Text1, Text2, SourceIp, Severity are plenty. Documentation is not entirely accurate - ID is required in my experience.

Thanks and appreciate your work!

[gdbarron]

Hi @Saadi6, good to see you again and thanks for your feature request. I've started looking into this, but no matter what I pass into this function, I am not able to get it working. I am passing in GroupID as a string, eg. 0029, and EventID (ID) as a hex equivalent of the event id 00290001 which is 46cd1. I've tried padding the hex string. I've even tried the example on the docs site. What am I missing here?

One interesting thing I notice is a difference in the error. When providing the GroupID and ID, I get 'Unable to parse input document.'. When providing GroupID, ID, and Component, I get a 400 Bad Request.

[WhyNotMore]

Hi @gdbarron , Try to convert the Event ID like this:
[convert]::toint32(00290001, 16) The result can be passed into the POST Log function

[gdbarron]

Hey there @WhyNotMore. I tried that and still was not successful.

I've tried it as shown in the screenshot with GroupID as an integer and also with it as a string so the preceeding zeros are kept. Same error. Adding Component changes the error to a 400.

[gdbarron]

Sometimes, you overlook the smallest detail. I was passing Id instead of ID for the event id. Working fine now. But, the docs are definitely incorrect. I have to pass Group ID, Event ID, and Component otherwise it fails. @BeardedPrincess, could you confirm the required fields for POST /log?

[Saadi6]

@gdbarron, yes those params are correct. Documentation is not. I will let the appropriate people know. Some params with IDs have a specific meaning in TPP so I use a particular ID when I know how I want TPP to treat this log message. Passing a made up value will not result in an error but best to use same made up ID each time for that param.

[gdbarron]

Thanks, @Saadi6. Any chance you could get some more specifics on the other values? I know Component, according to the docs, is meant to be a DN, but I've seen other uses/values. What IDs does Component ID correlate to? I'm thinking of enforcing Event ID to begin with the Group ID, any issues with that? Not really clear on what Grouping is at all :)

I'm creating an enum for severity to make that a bit easier to digest. The default group ids aren't integers so I can't create an enum, but I've created a hash which can be used, eg. $TppEventGroup.'WebSDK REST API'. Let me know if you have a better approach for this.

[BeardedPrincess]

@gdbarron and @Saadi6 That definitely looks like a documentation bug (at least)... also, it would be better if those parameters were not case sensitive (I've found some are, and others are not). I'll get the documentation feedback submitted, and see if I can get a bug report on the case insensitivity bit and see where it goes.

The component should be tied to the DN of the 'object' that you're interacting with while writing events. If there is no such 'component', the common practice is to associate to the DN of an engine (like \VED\Engines\TPP01). Obviously this gets a little weird if you're going through a load-balancer and don't have visibility about which TPP server you're connected to.

Grouping is an idea that a single task would all have the same group.. it's a number (int) I believe. For example, if you were moving a bunch of objects around, and logging some logic around each move, they might all have the same grouping ID representing they all happened as part of the same script or effort.

Somewhere I've seen a reservation of EventIDs (groups) documented internally that can be used for custom events from scripting as well. This is so that you can define your own set of Events and have proper translation when viewing the logs. I'll try and find that tomorrow and post back here. (Probably could use a published KB on how to actually use that functionality.. I'll look into getting that written and published on the support site)

[gdbarron]

Good info, thanks much.

Grouping makes sense after that explanation. I could see GUID being a good type for that. If it is an int, a string, from an api perspective, might be better.

For custom events, is this what you were talking about?

[gdbarron]

@BeardedPrincess, could you add something else to your inquiry? A few of the group IDs aren't working and it looks like this is when the converted event id is negative. My conversion is what @WhyNotMore suggested, [System.Convert]::ToInt32($GroupID$EventID, 16).

• Group Name: Venafi Tools, Group ID: FFFE, Event ID: 0001, Converted: -131071
• Group Name: Tracing, Group ID: FFFF, Event ID: 0001, Converted: -65535
• Group Name: CyberArk, Group ID: 8001, Event ID: 0001, Converted: -2147418111

[WhyNotMore]

@gdbarron you could try changing the Int to 64. like this: [System.Convert]::ToInt64($GroupID$EventID, 16) it should give you a positive.

[gdbarron]

@WhyNotMore, is that for all or just the negative ones?

[WhyNotMore]

@gdbarron it should work for all values. The number on the end of each function represents the bit count, therefore, represents the maximum Integer value that can be returned.

Int16 = 16bit (short) Int32 = 32bit (normal) Int64 = 64bit (long)

[BeardedPrincess]

So, here's what I found... event IDs 0x02990000 - 0x0299FFFF are reserved for customers to use for custom WebSDK events. This means that Venafi will never create new event IDs in this range to prevent overwriting custom event IDs that customers may define.

There are no 'negative' event IDs. They are all 64-bit integer values usually displayed as hex to make it easier to read but they're actually stored & defined - and passed to this function - as integer values. (and to keep the uninitiated guessing ;-) ). The first 32-bits (the first 4 hex characters) are a logical grouping. For example, all Aperture related UI events (when a user does something in Aperture) are within the range (group) of 001B. This means that all EventIDs between 0x001B0000 and 0x001BFFFF are reserved for Aperture UI related events. The event IDs are always documented in a KB by version (since each version brings new events). For the 18.x versions, you can see all of the defined event IDs here: https://support.venafi.com/hc/en-us/articles/360003460191.

I need to do a bit of research and testing so that I can publish a KB on how to create the definitions for custom events... I did it once a long time ago. But in the meantime, you should use Event IDs within the 0x02990000 - 0x0299FFFF range. You as the developer get to choose what information goes into Value1, Value2, Text1, Text2, etc. The log will still show, but it won't have any 'translation'. Just the raw data will show for now in the logs when viewed until I can document for you how to define the translation.

Update: Documentation feedback has been submitted (2019-03-08)

[WhyNotMore]

@BeardedPrincess I just want to clarify on the event IDs. can we use event IDs 0x02000000 - 0x0299FFFF? Document below states that the Add-on application event definitions 0200-0299 reserved for us to use: https://docs.venafi.com/Docs/18.4SDK/TopNav/Content/Logging/cco-Logging-EventStructure-LogSchema.php?Highlight=event%20definition Am I interpreting correctly?

[gdbarron]

Implemented in #71