Export data from MQTT device using mqtt-export

[Erickrk]

📚 Docs or Wiki Bug Report

Description [REQUIRED]

I've been reading through the docs and trying to piece together how to export data being published to my local broker, edgex-mqtt-broker, to a broker on the internet, specifically to HiveMQ. To achieve this, I'm attempting to mimic my device using the MQTT simulated device and using the App Service Configurable tutorial to export data, but with no luck so far.

First, the MQTT simulated device doesn't post its test messages to the default "edgex/events". Instead, it publishes to an MQTT topic named "incoming/data/my-custom-device/values", which I believe does not trigger the "edgex/events" topic by default. Following the Application Service Configurable docs, I managed to create a configuration.yaml file that can subscribe to other topics—unfortunately, always with "edgex/" as a prefix. Is there a way to change that? Here is my current configuration:

Writable:
  LogLevel: "INFO"

  StoreAndForward:
    Enabled: false

  Pipeline:
    TargetType: "metric"
    # FilterByDeviceName
    ExecutionOrder: "ToLineProtocol, MQTTExport"

    Functions:
      ToLineProtocol:
        Parameters:
          Tags: ""

      MQTTExport:
        Parameters:
          BrokerAddress: "broker.hivemq.com:1883"
          Topic: "edgex-export"
          SecretName: "mqtt"
          ClientId: "MQTT-Export"
          QOS: "0"
          AutoReconnect: "true"
          MaxReconnectInterval: "" # Empty value means use default setting
          KeepAlive: "" # Empty indicates to use default value
          ConnectTimeout: "" # Empty indicates to use default value
          Retain: "false"
          SkipVerify: "false"
          AuthMode: "none"
          PersistOnError: "false"
          WillEnabled: "false"
          WillPayload: ""
          WillQos: "0"
          WillRetained: "false"
          WillTopic: ""
          PreConnect: "true"
          PreConnectRetryCount: "" # Empty value means use default setting
          PreConnectRetryInterval: "" # Empty value means use default setting

Service:
  Host: "localhost"
  Port: 59703
  StartupMsg: "app-mqtt-export has Started"

MessageBus:
  Optional:
    ClientId: "app-mqtt-export"

Trigger:
  SubscribeTopics: "incoming/data/#"

I've modified the simulated device code to publish data to "edgex/incoming/data/" and confirmed that messages are being published by subscribing to this topic using another script.

If I use 'events/#' as the value for SubscribeTopics, it successfully publishes some incorrect data but with the same frequency as the Virtual Device used to, so I assume it's only a parsing error. On the other hand, when using the default configurations, the Virtual Device is exported to HiveMQ without issues. The only difference I see with its implementation is the 'FilterByDeviceName' tag and the topic where events are posted—the default 'edgex/events'.

Am I on the right path to export data from an MQTT device, and what am I missing in the configuration?

🔬 Minimal Reproduction

What's the affected URL? [REQUIRED]

Following the steps above should lead to the same outcome.

Expected vs Actual Behavior

I expected that the MQTT device would be able to post something to 'edgex/events' by default, but in practice, I have to change both the JavaScript code used to simulate the device and the app-service-configurable YAML file to get it exported.

🔥 Exception or Error

n/a

🌍 Your Environment

I'm running Edgex 3.1 and trying to follow the latest tutorials.

[cloudxxx8]

The MQTT simulator should work with MQTT Device Service to publish the events. Or, you can write your own simulator to publish data the the topic you want.

The App Service trigger subscription will always add the prefix by default, but you can use the External MQTT as the trigger, not the internal message bus. https://docs.edgexfoundry.org/3.1/microservices/application/details/Triggers/#external-mqtt-broker-configuration

[Erickrk]

Thank you for the external mqtt tip. I wonder how would the MQTT simulator do that since it's code doesn't publish to edgex/events. Is it through the device/profile definition?

[cloudxxx8]

the mock-device.js source code is in the docs, and it's just an example. https://docs.edgexfoundry.org/3.1/examples/Ch-ExamplesAddingMQTTDevice/#creating-and-running-a-mqtt-device-simulator

[Erickrk]

Hm..ok. I guess I will try again to change the publish topic. In theory if I publish to 'edgex/data/my-custom-device/values' we should get a valid output to cloud, right?