programming-the-iot / book-exercise-tasks

This repo is for issues / tasks ONLY. All programming and related exercises for each chapter of 'Programming the Internet of Things' are listed here.
Other
11 stars 12 forks source link

PIOT-CDA-03-006: Create / edit module - SensorAdapterManager #51

Open labbenchstudios opened 4 years ago

labbenchstudios commented 4 years ago

Description

NOTE: This class is a bit involved, although for the short term, its focus should be on managing simulators only. Chapter 04 will add new functionality to work with emulators.

Review the README

Estimated effort may vary greatly

Actions

NOTE: The implementation examples depicted here are only one way to implement the requirements listed. Your own implementation may vary of course.

def startManager(self) -> bool: logging.info("Started SensorAdapterManager.")

if not self.scheduler.running:
    self.scheduler.start()
    return True
else:
    logging.info("SensorAdapterManager scheduler already started. Ignoring.")
    return False

def stopManager(self) -> bool: logging.info("Stopped SensorAdapterManager.")

try:
    self.scheduler.shutdown()
    return True
except:
    logging.info("SensorAdapterManager scheduler already stopped. Ignoring.")
    return False
- Implement the `handleTelemetry()` method:
```python
def handleTelemetry(self):
    humidityData = self.humidityAdapter.generateTelemetry()
    pressureData = self.pressureAdapter.generateTelemetry()
    tempData     = self.tempAdapter.generateTelemetry()

    humidityData.setLocationID(self.locationID)
    pressureData.setLocationID(self.locationID)
    tempData.setLocationID(self.locationID)

    logging.debug('Generated humidity data: ' + str(humidityData))
    logging.debug('Generated pressure data: ' + str(pressureData))
    logging.debug('Generated temp data: ' + str(tempData))

    if self.dataMsgListener:
        self.dataMsgListener.handleSensorMessage(humidityData)
        self.dataMsgListener.handleSensorMessage(pressureData)
        self.dataMsgListener.handleSensorMessage(tempData)

Optional

Estimate

Tests

2020-09-07 20:25:47,916:SensorSimAdapterManagerTest:INFO:Testing SensorAdapterManager class [using simulators]... 2020-09-07 20:25:48,073:base:INFO:Adding job tentatively -- it will be properly scheduled when the scheduler starts 2020-09-07 20:25:48,073:ConfigUtil:INFO:Loading config: ../../../../../../../config/PiotConfig.props . . . 2020-09-07 20:25:48,074:SensorDataGenerator:INFO:Current time set to: Mon Sep 7 20:25:48 2020 2020-09-07 20:25:48,074:SensorDataGenerator:INFO:timeEntries tuple. Array Size: 1440 ND Size: 1440 Dimensions: 1 Shap e: (1440,) Type: float64 2020-09-07 20:25:48,074:SensorDataGenerator:DEBUG:Noise=10.000000; Noise Scale=0.100000; Mean Magnitude=1.000000 2020-09-07 20:25:48,074:SensorDataGenerator:INFO:dataEntries tuple. Array Size: 1440 ND Size: 1440 Dimensions: 1 Shap e: (1440,) Type: float64 2020-09-07 20:25:48,075:SensorDataGenerator:INFO:Current time set to: Mon Sep 7 20:25:48 2020 2020-09-07 20:25:48,075:SensorDataGenerator:INFO:timeEntries tuple. Array Size: 1440 ND Size: 1440 Dimensions: 1 Shap e: (1440,) Type: float64 2020-09-07 20:25:48,075:SensorDataGenerator:DEBUG:Noise=10.000000; Noise Scale=10.000000; Mean Magnitude=3.000000 2020-09-07 20:25:48,075:SensorDataGenerator:INFO:dataEntries tuple. Array Size: 1440 ND Size: 1440 Dimensions: 1 Shap e: (1440,) Type: float64 2020-09-07 20:25:48,075:SensorDataGenerator:INFO:Current time set to: Mon Sep 7 20:25:48 2020 2020-09-07 20:25:48,075:SensorDataGenerator:INFO:timeEntries tuple. Array Size: 1440 ND Size: 1440 Dimensions: 1 Shap e: (1440,) Type: float64 2020-09-07 20:25:48,075:SensorDataGenerator:DEBUG:Noise=10.000000; Noise Scale=0.100000; Mean Magnitude=1.000000 2020-09-07 20:25:48,076:SensorDataGenerator:INFO:dataEntries tuple. Array Size: 1440 ND Size: 1440 Dimensions: 1 Shap e: (1440,) Type: float64 2020-09-07 20:25:48,076:SensorAdapterManager:INFO:Started SensorAdapterManager. 2020-09-07 20:25:48,077:base:INFO:Added job "SensorAdapterManager.handleTelemetry" to job store "default" 2020-09-07 20:25:48,077:base:INFO:Scheduler started 2020-09-07 20:25:48,078:base:DEBUG:Looking for jobs to run 2020-09-07 20:25:48,078:base:DEBUG:Next wakeup is due at 2020-09-07 20:25:53.072280-04:00 (in 4.994015 seconds) 2020-09-07 20:25:53,073:base:DEBUG:Looking for jobs to run 2020-09-07 20:25:53,073:base:INFO:Running job "SensorAdapterManager.handleTelemetry (trigger: interval[0:00:05], next ru n at: 2020-09-07 20:25:53 EDT)" (scheduled at 2020-09-07 20:25:53.072280-04:00) 2020-09-07 20:25:53,074:base:DEBUG:Next wakeup is due at 2020-09-07 20:25:58.072280-04:00 (in 4.998481 seconds) 2020-09-07 20:25:53,074:SensorAdapterManager:INFO:Simulated humidity data: name=Not Set,timeStamp=2020-09-07 20:25:53.07 4702,curValue=34.83071784173395 2020-09-07 20:25:53,074:SensorAdapterManager:INFO:Simulated pressure data: name=Not Set,timeStamp=2020-09-07 20:25:53.07 4702,curValue=995.1129911788777 2020-09-07 20:25:53,074:SensorAdapterManager:INFO:Simulated temp data: name=Not Set,timeStamp=2020-09-07 20:25:53.074702 ,curValue=20.032501789888606 2020-09-07 20:25:53,074:DefaultDataMessageListener:INFO:Sensor Message: name=Not Set,timeStamp=2020-09-07 20:25:53.07470 2,curValue=34.83071784173395 2020-09-07 20:25:53,074:DefaultDataMessageListener:INFO:Sensor Message: name=Not Set,timeStamp=2020-09-07 20:25:53.07470 2,curValue=995.1129911788777 2020-09-07 20:25:53,074:DefaultDataMessageListener:INFO:Sensor Message: name=Not Set,timeStamp=2020-09-07 20:25:53.07470 2,curValue=20.032501789888606 2020-09-07 20:25:53,074:base:INFO:Job "SensorAdapterManager.handleTelemetry (trigger: interval[0:00:05], next run at: 20 20-09-07 20:25:58 EDT)" executed successfully 2020-09-07 20:25:58,074:base:DEBUG:Looking for jobs to run 2020-09-07 20:25:58,074:base:INFO:Running job "SensorAdapterManager.handleTelemetry (trigger: interval[0:00:05], next ru n at: 2020-09-07 20:25:58 EDT)" (scheduled at 2020-09-07 20:25:58.072280-04:00) . . . 2020-09-07 20:26:48,073:base:DEBUG:Looking for jobs to run 2020-09-07 20:26:48,073:base:DEBUG:Next wakeup is due at 2020-09-07 20:26:53.072280-04:00 (in 4.998391 seconds) 2020-09-07 20:26:48,073:base:INFO:Running job "SensorAdapterManager.handleTelemetry (trigger: interval[0:00:05], next ru n at: 2020-09-07 20:26:53 EDT)" (scheduled at 2020-09-07 20:26:48.072280-04:00) 2020-09-07 20:26:48,074:SensorAdapterManager:INFO:Simulated humidity data: name=Not Set,timeStamp=2020-09-07 20:26:48.07 4792,curValue=35.203348572031835 2020-09-07 20:26:48,074:SensorAdapterManager:INFO:Simulated pressure data: name=Not Set,timeStamp=2020-09-07 20:26:48.07 4792,curValue=997.7317060337219 2020-09-07 20:26:48,074:SensorAdapterManager:INFO:Simulated temp data: name=Not Set,timeStamp=2020-09-07 20:26:48.074792 ,curValue=20.203318298162788 2020-09-07 20:26:48,074:DefaultDataMessageListener:INFO:Sensor Message: name=Not Set,timeStamp=2020-09-07 20:26:48.07479 2,curValue=35.203348572031835 2020-09-07 20:26:48,074:DefaultDataMessageListener:INFO:Sensor Message: name=Not Set,timeStamp=2020-09-07 20:26:48.07479 2,curValue=997.7317060337219 2020-09-07 20:26:48,074:DefaultDataMessageListener:INFO:Sensor Message: name=Not Set,timeStamp=2020-09-07 20:26:48.07479 2,curValue=20.203318298162788 2020-09-07 20:26:48,074:base:INFO:Job "SensorAdapterManager.handleTelemetry (trigger: interval[0:00:05], next run at: 20 20-09-07 20:26:53 EDT)" executed successfully 2020-09-07 20:26:48,078:SensorAdapterManager:INFO:Stopped SensorAdapterManager. 2020-09-07 20:26:48,078:base:INFO:Scheduler has been shut down 2020-09-07 20:26:48,078:base:DEBUG:Looking for jobs to run 2020-09-07 20:26:48,079:base:DEBUG:No jobs; waiting until a job is added

Ran 1 test in 60.163s

OK

MAbuShattal commented 2 years ago

I'm running the following code for SensorDataManager.py and testing using SensorDataManagerTest.py:

#####
# 
# This class is part of the Programming the Internet of Things project.
# 
# It is provided as a simple shell to guide the student and assist with
# implementation for the Programming the Internet of Things exercises,
# and designed to be modified by the student as needed.
#

import logging

from importlib import import_module

from apscheduler.schedulers.background import BackgroundScheduler

import programmingtheiot.common.ConfigConst as ConfigConst

from programmingtheiot.common.ConfigUtil import ConfigUtil
from programmingtheiot.common.IDataMessageListener import IDataMessageListener

from programmingtheiot.cda.sim.SensorDataGenerator import SensorDataGenerator
#from programmingtheiot.cda.sim.HumiditySensorSimTask import HumiditySensorSimTask
from programmingtheiot.cda.sim.TemperatureSensorSimTask import TemperatureSensorSimTask
#from programmingtheiot.cda.sim.PressureSensorSimTask import PressureSensorSimTask

class SensorAdapterManager(object):
    """
    Shell representation of class for student implementation.

    """

    def __init__(self):
        self.dataGenerator = SensorDataGenerator()

        configUtil = ConfigUtil()

        tempFloor = configUtil.getFloat(section = ConfigConst.CONSTRAINED_DEVICE, 
                                    key = ConfigConst.TEMP_SIM_FLOOR_KEY,
                                    defaultVal = SensorDataGenerator.LOW_NORMAL_INDOOR_TEMP)
        tempCeiling = configUtil.getFloat(section = ConfigConst.CONSTRAINED_DEVICE,
                                    key = ConfigConst.TEMP_SIM_CEILING_KEY, 
                                    defaultVal = SensorDataGenerator.HI_NORMAL_INDOOR_TEMP)

        self.pollRate = configUtil.getInteger( 
            section = ConfigConst.CONSTRAINED_DEVICE, 
            key = ConfigConst.POLL_CYCLES_KEY, 
            defaultVal = ConfigConst.DEFAULT_POLL_CYCLES)
        self.useEmulator = configUtil.getBoolean(
            section = ConfigConst.CONSTRAINED_DEVICE,
            key = ConfigConst.ENABLE_EMULATOR_KEY)
        self.locationID = configUtil.getProperty(
            section = ConfigConst.CONSTRAINED_DEVICE,
            key = ConfigConst.DEVICE_LOCATION_ID_KEY, 
            defaultVal = ConfigConst.NOT_SET)
        if self.pollRate <= 0: 
            self.pollRate = ConfigConst.DEFAULT_POLL_CYCLES
        self.scheduler = BackgroundScheduler()

        self.scheduler.add_job( self.handleTelemetry, 
                            'interval', seconds = self.pollRate)
        self.dataMsgListener = None

        if not self.useEmulator:
            tempFloor = configUtil.getFloat( section = ConfigConst.CONSTRAINED_DEVICE,
                            key = ConfigConst.TEMP_SIM_FLOOR_KEY,
                            defaultVal = SensorDataGenerator.LOW_NORMAL_INDOOR_TEMP)

        tempCeiling = configUtil.getFloat( section = ConfigConst.CONSTRAINED_DEVICE,
                            key = ConfigConst.TEMP_SIM_CEILING_KEY, 
                            defaultVal = SensorDataGenerator.HI_NORMAL_INDOOR_TEMP)
        tempData = self.dataGenerator.generateDailyIndoorTemperatureDataSet(
                     minValue = tempFloor, maxValue = tempCeiling, useSeconds = False)

        self.tempAdapter = TemperatureSensorSimTask(dataSet = tempData)

    def handleTelemetry(self):
        tempData = self.tempAdapter.generateTelemetry()
        tempData.setLocationID(self.locationID)
        logging.info('Generated temp data: ' + str(tempData))
        if self.dataMsgListener:
            self.dataMsgListener.handleSensorMessage(tempData)

    def setDataMessageListener(self, listener: IDataMessageListener) -> bool:
        if listener: 
            self.dataMsgListener = listener

    def startManager(self):
        logging.info('Started SensorAdapterManager.')
        if not self.scheduler.running:
            self.scheduler.start() 
        else: 
            logging.warning( 'SensorAdapterManager scheduler already started.')

    def stopManager(self):
        logging.info('Stopped SensorAdapterManager.') 
        try: 
            self.scheduler.shutdown() 
        except: 
            logging.warning('SensorAdapterManager scheduler already stopped.')

It seems to work for the first iteration but fails in the 2nd iteration and generates the following error:

Finding files... done.
Importing test modules ... done.

2021-12-15 04:33:47,033:SensorAdapterManagerTest:INFO:Testing SensorAdapterManager class...
2021-12-15 04:33:47,034:ConfigUtil:INFO:Loading config: ../../../../../../../config/PiotConfig.props
2021-12-15 04:33:47,034:ConfigUtil:DEBUG:Config: ['Mqtt.GatewayService', 'Coap.GatewayService', 'ConstrainedDevice']
2021-12-15 04:33:47,034:ConfigUtil:INFO:Created instance of ConfigUtil: <programmingtheiot.common.ConfigUtil.ConfigUtil object at 0x7fb13800c4f0>
/usr/local/lib/python3.8/dist-packages/apscheduler/util.py:95: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new time zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
  if obj.zone == 'local':
2021-12-15 04:33:47,098:base:INFO:Adding job tentatively -- it will be properly scheduled when the scheduler starts
2021-12-15 04:33:47,099:SensorDataGenerator:INFO:Current time set to: Wed Dec 15 04:33:47 2021
2021-12-15 04:33:47,099:SensorDataGenerator:INFO:timeEntries tuple. Array Size: 1440  ND Size: 1440  Dimensions: 1  Shape: (1440,)  Type: float64
2021-12-15 04:33:47,099:SensorDataGenerator:DEBUG:Noise=10.000000; Noise Scale=0.100000; Mean Magnitude=1.000000
2021-12-15 04:33:47,099:SensorDataGenerator:INFO:dataEntries tuple. Array Size: 1440  ND Size: 1440  Dimensions: 1  Shape: (1440,)  Type: float64
======================================================================
ERROR: setUpClass (src.test.python.programmingtheiot.part02.integration.system.SensorAdapterManagerTest.SensorAdapterManagerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mas/programmingtheiot/python-components/src/test/python/programmingtheiot/part02/integration/system/SensorAdapterManagerTest.py", line 34, in setUpClass
    self.sensorAdapterMgr = SensorAdapterManager()
  File "/home/mas/programmingtheiot/python-components/src/main/python/programmingtheiot/cda/system/SensorAdapterManager.py", line 76, in __init__
    self.tempAdapter = TemperatureSensorSimTask(dataSet = tempData)
TypeError: __init__() got an unexpected keyword argument 'dataSet'

----------------------------------------------------------------------
Ran 0 tests in 0.070s

FAILED (errors=1)

Any suggestions?

Side note: There is another issue about circular import in the classes SensorDataManager.py and TemperatureSensorSimTask.py. Modifying TemperatureSensorSimTask.py to solve it create the test for TemperatureSensorSimTaskTest.py. invalid. Modifying SensorDataManager.py to solve the issue results in the SensorDataManager.py being useless since it needs the definition of TemperatureSensorSimTask.py. Now, preferring to solve the issue to keep SensorDataManager.py running, I have the issue above.

Best regards, Mohammad Abu Shattal

labbenchstudios commented 2 years ago

Thanks for the input and detailed description. Here are some initial thoughts -

Exception issue

The exception being thrown appears to indicate that TemperatureSensorSimTask does not have the required dataSet parameter specified in its constructor. The parameter is used to contain the data set that the task will iterate upon during each call when in simulation mode (which is the case for this specific exercise). Can you verify whether or not this is the case with your implementation? See PIOT-CDA-03-003 for details on the constructor implementation for TemperatureSensorSimTask.

Import issue

Although the naming may imply a circular reference, they are not designed / declared as such in the sample code. The import declaration first references the module (the Python file) and then the class, which has the same name as the module. The template code provided includes the named modules, with like-named classes. Can you verify this in your naming conventions?

Considerations regarding naming

The Python style guide indicates that module names are lowercase and class names are CamelCase. I've taken some liberty here, as it is my opinion that the CDA's CamelCase module names (file names) are easier to read when considering the GDA code base (written in Java) also follows this naming convention. My interpretation of the style guide is that in the case of readability, this is acceptable.

Hope this helps!

labbenchstudios commented 2 years ago

Updated this card (and many others) with sample solutions based on my own implementations. Hope this helps!

Gau17 commented 8 months ago

Dear Professor Andy,

I noticed a minor issue in the provided code, just thought I'd point it out.

The pressure ceiling default value should reference the SensorDataGenerator.HI_NORMAL_ENV_PRESSURE value from ConfigConst. But in the code it references SensorDataGenerator.LOW_NORMAL_ENV_PRESSURE.

I've mentioned the corrected piece of code below:

pressureCeiling = \
    self.configUtil.getFloat( \
        section = ConfigConst.CONSTRAINED_DEVICE, key = ConfigConst.PRESSURE_SIM_CEILING_KEY, defaultVal = SensorDataGenerator.HI_NORMAL_ENV_PRESSURE)

Best Regards, Gautam Bidari

labbenchstudios commented 7 months ago

Thanks for the input - good catch. The code sample is now updated to reflect this correction. Much appreciated!