search

Chaffelson / nipyapi

A convenient Python wrapper for Apache NiFi
Other
243 stars 76 forks source link

Having issue creating a LogMessage processor with values set for the default properties #289

Closed instance-id closed 2 years ago

instance-id commented 2 years ago

Description

I am trying to deploy a LogMessage node, and while most everything is fine, the issue is that upon creation of the node, the properties are just creating a new separate set of properties after the original ones and not setting those values. Seen below is the code section in which I am using, followed by an example of the outcome. I tried the properties field using both single and double quotes, but it didn't seem to make a difference. Also, the L in Level for the Log Level property is capitalized in Nifi, but the second word of each of the other properties is not, I don't know if that has anything to do with it?

    log_message_name = F"{numeric}.{str(start_num + 2)}_{title}_Log"
    filename_string = '${filename}'
    prefix_string = F"__Log_PutFile_{put_file_name}_{title.lower()}"
    message_string = F"Unable to deliver {filename_string} to {put_file_path}"

    # -- LogMessage Processor --------------
    log_message = nipyapi.canvas.create_processor(
        parent_pg=process_group,
        processor=nipyapi.canvas.get_processor_type('LogMessage'),
        location=(400.0, 500.0),
        name=log_message_name,
        config=nipyapi.nifi.ProcessorConfigDTO(
            scheduling_period='1s',
            auto_terminated_relationships=['success', 'failure'],
            properties={
                "Log Level": "error",
                "Log prefix": prefix_string,
                "Log message": message_string
            },
        )
    )

Urgency

While it is technically blocking from being able to bulk deploy a large number of processors that are needed to be tested sooner rather than later, it really isn't a huge deal at this point, especially since I can just copy/paste the text if needed.

I am not sure if there is something that I am just not doing correctly, but after scouring this repo and more generic searches for 'nipyapi LogMessage' or log with keywords such as prefix and message, I can't seem to find any examples of it not only being done properly but being done at all.

If someone might be able to point me in the right direction, I would greatly appreciate it.

Update, I tried adjusting how I went about this based on a code example I found for another processor, but it, unfortunately, didn't seem to help any. Still had the same result of duplicating the properties. This was attempted by removing the initial properties I was trying to set during the creation of the processor so that there were only just the default property fields, trying to update the existing ones.

    log_properties_dict = log_message.to_dict()
    properties = log_properties_dict['component']['config']['properties']
    properties['Log Level'] = 'error'
    properties['Log prefix'] = prefix_string
    properties['Log message'] = message_string
    newConfig = nipyapi.nifi.ProcessorConfigDTO(properties=properties)
    nipyapi.canvas.update_processor(log_message, newConfig)
instance-id commented 2 years ago

Never mind, my apologies. Upon dumping out the data that was within properties = log_properties_dict['component']['config']['properties'] I discovered that the properties were under the names 'log-level', 'log-prefix', and 'log-message'. By using those values instead, I was able to update them properly.

Chaffelson commented 2 years ago

Glad you found the solution! This is not uncommon, as the codegen for the base level of NiPyAPI has a format translation from Java standards to python, so inspecting the names is most effective usually.

On Thu, 30 Sep 2021, 18:09 instance.id, @.***> wrote:

Never mind, my apologies. Upon dumping out the data that was within properties = log_properties_dict['component']['config']['properties'] I discovered that the properties were under the names 'log-level', 'log-prefix', and 'log-message'. By using those values instead, I was able to update them properly.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Chaffelson/nipyapi/issues/289#issuecomment-931507826, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZAZOF7BN2XGZTAZKH3KVDUESKVHANCNFSM5FCWNBKA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

instance-id commented 2 years ago

Something of a side question/recommendation request, when using nipyapi.canvas.create_connection(), I wanted to specify bends in the connection, but there doesn't seem to be a way to define them with the function available.

What would be the best way to go about doing so? Creating a while new create_connection function, or getting the return from the create function, then modifying it and sending another update, or something else altogether?

edit: Perhaps it's not even counted as a bend, but basically the position of the connector node in the middle between two processors.

double edit: Nevermind again, sorry, I got it figured out, lol.