Export Data application development

[edgarbelda]

Hi @DougSchmidt-AI! I'm developing a application quite complex for export data from Aquarius. I attach an outline of the application.

For monitoring the time series I used your example (TimeSeriesChangeMonitor) But now I wonder if there is an efficient way to get approval periods (both time series and rating models) to monitor changes. If you have any further advice, you are also welcome. Thank you.

Regards.

[DougSchmidt-AI]

Hi Edgar.

While there aren't any APIs dedicated to monitoring only approval changes, there are two APIs which can be used as a trigger to check a time-series or rating model for approval changes.

Both APIs follow the "changes since" pattern, where your integration is responsible for:

• storing a token outside of AQTS
• passing that token into the next API request
• processing the list of changed items in the response
• storing the response's NextToken for use in the next API request

Monitoring a time-series for approval changes

This is the /GetTimeSeriesUniqueIdList?ChangesSinceToken={yourSavedToken} request that the TimeSeriesChangeMonitor.exe uses.

The ChangeEventType is an optional request parameter, which can be:

• Data - Only report changes to points in the time-series
• Attribute - Only report changes to the attributes of the time-series.
• Empty (default value) Report changes to either the points or attributes.

Since your integration needs to monitor both point values and approval changes, you should just leave the ChangeEventType request parameter at its default value.

In the API response, you'll see that each time-series that has changed has three properties: UniqueId, FirstPointChanged, and HasAttributeChange.

When HasAttributeChange is true, that is the signal for your integration to request /GetApprovalsTransactionList?TimeSeriesUniqueId={UniqueId} to fetch the current approval list for the time-series.

Your integration will then need to compare that list to some previously saved copy, in order to determine if an approval change really occurred.

Monitoring a rating-model for approval changes

Use the /GetRatingModelDescriptionList?ChangesSinceToken={yourSavedToken} request to periodically poll for changes to rating models.

When a rating model is reported as having changed, use the /GetRatingCurveList?RatingModelIdentifier={changeRatingModelIdentifier} to get the detailed information of the rating curve, which includes an Approvals response property.

Then your integration will need to comparing the rating model's current approval list to some previously saved copy, in order to determine if an approval change really occured.

Summary of what your integration will need to persist

• The TimeSeries.NextToken value, to detect time-series changes.
• The RatingModels.NextToken value, to detect rating-model changes. (ie. The "changes since" tokens are different for different APIs, so you need to save both)
• A list of approval regions per time-series, so you can detect an approval change.
• A list of approval regions per rating-model, so you can detect an approval change.

I hope this helps.

[edgarbelda]

Thank you very much Doug. It helps me a lot. I'll try to make it as efficient as possible.

Cheers!

[edgarbelda]

Hi @DougSchmidt-AI! I'm monitoring another AQUARIUS server with the developed application and it always shows me the same error with the server token. I'm attaching a screenshot:

Do you know how can I fix it? or Why is this happening?

Thanks a lot.

[DougSchmidt-AI]

Hi Edgar.

The error you are seeing is not related to the REST API authentication (it isn't related to the "session" token set in the X-Authentication-Token header of each API request).

Instead, it is much more serious database error, something which should never occur during normal operation. That DB is seriously broken at this point, and is unusable as a production system.

The most likely cause is that one of the internal views (TIMESERIES_EVENT_TOKEN_VIEW) no longer exists in the DB. This view is used to provide the ChangesSinceToken value in the response to a few Publish API requests. I can see you are running the TimeSeriesChangeMonitor program, which definitely uses theses "ChangesSince" API requests to efficiently monitor for changes.

So the real question is "why does that view no longer exist" and my answer is "it should always exist", so I don't really know.

However, recall the previous Oracle DB problem on missing sequences, where the KV database administrator did not properly restore your test DB with all the required DB parts (just the tables and rows, but no sequences). This feels like that same situation to me. It seems like someone at KV is not following proper DB techniques for restoring databases.

You'll need to get in touch with KV and ask them how they are managing their Oracle databases so strangely. A database is much, much more than just tables and rows.

Let me know what you find out.

[edgarbelda]

Thank you very much for your explanations. I think the error was because they were backing up the database. Now, my application seems to be working fine.

I used your application (TimeSeriesChangeMonitor) because at first I thought it was a problem with my development. And that's when I tried your application and realised that it also failed.

If I find anything else, I'll keep you posted.

Cheers