search

kizniche / Mycodo

An environmental monitoring and regulation system
http://kylegabriel.com/projects/
GNU General Public License v3.0
2.94k stars 492 forks source link

Error with function regulate_ph_ec: TypeError: unsupported operand type(s) for +: 'NoneType' and 'float' when running it #1244

Closed VitorFrost closed 1 year ago

VitorFrost commented 1 year ago

Describe the problem/bug

I am trying to use pH control, and I have found the function regulate_ph_ec on the mycodo files, and added it as a custom function. I was trying to mod this function to limit the maximum added volume, but I was getting error, then I tried it without mods, and also got errors.

Versions:

Reproducibility

Install the function regulate_ph_ec.py and configure it to fix pH only (I don't know about EC, didn't tried it) When it detect it needs to fix pH, it doesn't works and gives the log error: 2022-11-25 15:04:11,806 - DEBUG - mycodo.function.regulate_ph_ec_7bbf4a29 - Measurements: EC: 4.927158827938737, pH: 4.927158827938737 2022-11-25 15:04:11,806 - DEBUG - mycodo.function.regulate_ph_ec_7bbf4a29 - pH is dangerously low: 4.93. Should be > 5.00. Dispensing 10.0 ml base 2022-11-25 15:04:11,806 - ERROR - mycodo.controllers.controller_function_7bbf4a29 - Exception while running loop() Traceback (most recent call last): File "/var/mycodo-root/mycodo/controllers/controller_function.py", line 76, in loop self.run_function.loop() File "/home/pi/Mycodo/mycodo/functions/custom_functions/regulate_ph_ec.py", line 771, in loop "ml_ph_raise", self.get_custom_option("ml_ph_raise") + self.output_ph_amount) TypeError: unsupported operand type(s) for +: 'NoneType' and 'float'

Additional context

Just to avoid any unknown errors, I am using pH input for the EC values.

kizniche commented 1 year ago

Thanks for the bug report. I just pushed a fix. Let me know if you find any more issues with it.

VitorFrost commented 1 year ago

Thanks for the fast fix! New error, should I put it here or in a new bug report?

2022-11-27 16:40:53,502 - ERROR - mycodo.controllers.controller_function_52ac7e80 - initialize_variables() Exception: reset_all_totals() missing 1 required positional argument: 'args_dict' Traceback (most recent call last): File "/var/mycodo-root/mycodo/controllers/base_controller.py", line 72, in run self.initialize_variables() File "/var/mycodo-root/mycodo/controllers/controller_function.py", line 115, in initialize_variables self.run_function = function_loaded.CustomModule(self.function) File "/home/pi/Mycodo/mycodo/functions/custom_functions/regulate_ph_ec.py", line 633, in __init__ self.reset_all_totals() TypeError: reset_all_totals() missing 1 required positional argument: 'args_dict' 2022-11-27 16:40:53,503 - INFO - mycodo.controllers.controller_function_52ac7e80 - Activated in 154.1 ms 2022-11-27 16:40:53,503 - ERROR - mycodo.controllers.controller_function_52ac7e80 - Deactivated unexpectedly

VitorFrost commented 1 year ago

I have tried to temporary fix it by deactivating lines 632 and 633

# Reset totals
#if self.total["sec_ph_raise"] is None:
#    self.reset_all_totals()

But got the previous problems again! 

2022-11-27 17:00:37,917 - DEBUG - mycodo.daemon - Function controller with ID dc9394fc-4236-4dfc-aa46-07c12360ac66 activated.
2022-11-27 17:00:47,791 - DEBUG - mycodo.function.regulate_ph_ec_dc9394fc - Most recent timestamp and measurement for pH: 2022-11-27T20:00:45.940971Z, 3.681312588550714
2022-11-27 17:00:47,881 - DEBUG - mycodo.function.regulate_ph_ec_dc9394fc - Most recent timestamp and measurement for EC: 2022-11-27T20:00:45.940971Z, 3.681312588550714
2022-11-27 17:00:47,882 - DEBUG - mycodo.function.regulate_ph_ec_dc9394fc - Measurements: EC: 3.681312588550714, pH: 3.681312588550714
2022-11-27 17:00:47,882 - DEBUG - mycodo.function.regulate_ph_ec_dc9394fc - pH is dangerously low: 3.68. Should be > 6.00. Dispensing 10.0 ml base
2022-11-27 17:00:47,882 - ERROR - mycodo.controllers.controller_function_dc9394fc - Exception while running loop()
Traceback (most recent call last):
  File "/var/mycodo-root/mycodo/controllers/controller_function.py", line 76, in loop
    self.run_function.loop()
  File "/home/pi/Mycodo/mycodo/functions/custom_functions/regulate_ph_ec.py", line 770, in loop
    "ml_ph_raise", self.get_custom_option("ml_ph_raise") + self.output_ph_amount)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'float'

I just don't undurstand completelly how the function self.set_custom_option( ) works to try any fix. What are the required and optional inputs for it?

Thanks!

kizniche commented 1 year ago

New error, should I put it here or in a new bug report?

Keep posting them here until we get the whole module working without issue.

I just pushed a fix for the missing argument.

kizniche commented 1 year ago

You can see the options of set_custom_option() and get_custom_option() at https://kizniche.github.io/Mycodo/Functions/#conditional-setup-guide

self.reset_all_totals() needs to be called to change the dictionary values of None to actual numerical values. These are totals that are stored in the database.

VitorFrost commented 1 year ago

My configs: image

Same error:

2022-11-27 17:19:54,303 - DEBUG - mycodo.function.regulate_ph_ec_721c3bf3 - Most recent timestamp and measurement for EC: 2022-11-27T20:19:45.857060Z, 3.6685998718222663
2022-11-27 17:19:54,304 - DEBUG - mycodo.function.regulate_ph_ec_721c3bf3 - Measurements: EC: 3.6685998718222663, pH: 3.6685998718222663
2022-11-27 17:19:54,304 - DEBUG - mycodo.function.regulate_ph_ec_721c3bf3 - pH is dangerously low: 3.67. Should be > 6.00. Dispensing 2.0 ml base
2022-11-27 17:19:54,304 - ERROR - mycodo.controllers.controller_function_721c3bf3 - Exception while running loop()
Traceback (most recent call last):
  File "/var/mycodo-root/mycodo/controllers/controller_function.py", line 76, in loop
    self.run_function.loop()
  File "/home/pi/Mycodo/mycodo/functions/custom_functions/regulate_ph_ec.py", line 770, in loop
    "ml_ph_raise", self.get_custom_option("ml_ph_raise") + self.output_ph_amount)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'float'

Why do you need to add the self.get_custom_option("ml_ph_raise") value toself.output_ph_amount ? Can't it just be self.set_custom_option("ml_ph_raise", self.output_ph_amount) ?

kizniche commented 1 year ago

If you're seeing the original error, it doesn't seem like you're actually running the latest code on the master branch. Please double check.

kizniche commented 1 year ago

Can't it just be self.set_custom_option("ml_ph_raise", self.output_ph_amount) ?

This would not keep a running total, but rather merely the last amount used.

kizniche commented 1 year ago

Also, why do you have an EC Measurement selected (the pH input?) and no EC dose Outputs set?

VitorFrost commented 1 year ago

Also, why do you have an EC Measurement selected (the pH input?) and no EC dose Outputs set?

To avoid no probe error:

2022-11-27 17:46:56,435 - INFO - mycodo.controllers.controller_function_906ddf83 - Activated in 965.4 ms
2022-11-27 17:46:56,436 - DEBUG - mycodo.daemon - Function controller with ID 906ddf83-b604-464a-acdd-4d7faa973c5d activated.
2022-11-27 17:47:06,341 - DEBUG - mycodo.function.regulate_ph_ec_906ddf83 - Most recent timestamp and measurement for pH: 2022-11-27T20:42:16.554319Z, 3.6851264035692486
2022-11-27 17:47:06,389 - ERROR - mycodo.function.regulate_ph_ec_906ddf83 - Could not find a measurement in the database for Measurement A device ID None and measurement ID None in the past 360 seconds
VitorFrost commented 1 year ago

If you're seeing the original error, it doesn't seem like you're actually running the latest code on the master branch. Please double check.

Downloaded it again with a new name, and I got the same problem. I've looked through the code, and I could not find a place where self.output_ph_amount is defined.

kizniche commented 1 year ago

Downloaded it again with a new name, and I got the same problem.

What does this mean? I would recommend using the "Upgrade to Master" method so you are assured you are using the latest code.

I've made some changes to allow you to not select either the pH or EC that is not in use and the function will operate properly. I've also added a log line to print your totals dictionary before and after the reset_totals function runs, to see what is going on with your function initialization.

VitorFrost commented 1 year ago

image

Wow, lots of errors now!

2022-11-27 18:52:01,680 - INFO - mycodo.controllers.controller_function_f2291a1a - Activated in 859.2 ms
2022-11-27 18:52:01,681 - DEBUG - mycodo.daemon - Function controller with ID f2291a1a-b6da-4600-958d-31e332fa82a2 activated.
2022-11-27 18:52:11,779 - ERROR - mycodo.controllers.controller_function_f2291a1a - Exception while running loop()
Traceback (most recent call last):
  File "/var/mycodo-root/mycodo/controllers/controller_function.py", line 76, in loop
    self.run_function.loop()
  File "/home/pi/Mycodo/mycodo/functions/regulate_ph_ec.py", line 721, in loop
    last_measurement_ph = self.get_last_measurement(
  File "/var/mycodo-root/mycodo/abstract_base_controller.py", line 532, in get_last_measurement
    return get_last_measurement(device_id, measurement_id, max_age=max_age)
  File "/var/mycodo-root/mycodo/utils/influx.py", line 312, in get_last_measurement
    last_measurement = read_influxdb_single(
  File "/var/mycodo-root/mycodo/utils/influx.py", line 362, in read_influxdb_single
    data = query_string(
  File "/var/mycodo-root/mycodo/utils/influx.py", line 291, in query_string
    ret_value = query_flux(
  File "/var/mycodo-root/mycodo/utils/influx.py", line 276, in query_flux
    tables = client.query_api().query(query)
  File "/var/mycodo-root/env/lib/python3.9/site-packages/influxdb_client/client/query_api.py", line 203, in query
    response = self._query_api.post_query(org=org, query=self._create_query(query, self.default_dialect, params),
  File "/var/mycodo-root/env/lib/python3.9/site-packages/influxdb_client/service/query_service.py", line 281, in post_query
    (data) = self.post_query_with_http_info(**kwargs)  # noqa: E501
  File "/var/mycodo-root/env/lib/python3.9/site-packages/influxdb_client/service/query_service.py", line 307, in post_query_with_http_info
    return self.api_client.call_api(
  File "/var/mycodo-root/env/lib/python3.9/site-packages/influxdb_client/_sync/api_client.py", line 343, in call_api
    return self.__call_api(resource_path, method,
  File "/var/mycodo-root/env/lib/python3.9/site-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api
    response_data = self.request(
  File "/var/mycodo-root/env/lib/python3.9/site-packages/influxdb_client/_sync/api_client.py", line 388, in request
    return self.rest_client.POST(url,
  File "/var/mycodo-root/env/lib/python3.9/site-packages/influxdb_client/_sync/rest.py", line 311, in POST
    return self.request("POST", url,
  File "/var/mycodo-root/env/lib/python3.9/site-packages/influxdb_client/_sync/rest.py", line 261, in request
    raise ApiException(http_resp=r)
influxdb_client.rest.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Content-Type': 'text/plain; charset=utf-8', 'Request-Id': 'bf5c444d-6e9d-11ed-8208-e45f018611d2', 'X-Content-Type-Options': 'nosniff', 'X-Influxdb-Build': 'OSS', 'X-Influxdb-Version': '1.8.10', 'X-Request-Id': 'bf5c444d-6e9d-11ed-8208-e45f018611d2', 'Date': 'Sun, 27 Nov 2022 21:52:11 GMT', 'Content-Length': '100'})
HTTP response body: b'Flux query service disabled. Verify flux-enabled=true in the [http] section of the InfluxDB config.\n'
kizniche commented 1 year ago

Please attach a txt file with an excerpt from your upgrade log of your most recent upgrade. Also. it's just one error.

kizniche commented 1 year ago 
[2022-11-27 18:32:29] parse error: Invalid numeric literal at line 1, column 7
[2022-11-27 18:32:29] parse error: Invalid numeric literal at line 1, column 7

Your system didn't run a critical part of the upgrade.

What is the output when you execute the following commands:

~/Mycodo/env/bin/python ~/Mycodo/mycodo/scripts/measurement_db.py -i
jq --version
VitorFrost commented 1 year ago 
~/Mycodo/env/bin/python ~/Mycodo/mycodo/scripts/measurement_db.py -i
jq --version
Trying host: localhost
{"db_name": "influxdb", "db_version": "1", "influxdb_installed": true, "influxdb_retention_policy": "autogen", "influxdb_version": "1.8.10", "influxdb_host": "localhost", "influxdb_port": 8086}
jq-1.6
kizniche commented 1 year ago

Thanks. Just pushed a fix. Upgrade to master again and that should fix the Flux issue and let us get back on track fixing the regulation Function.

kizniche commented 1 year ago

Looks like it fixed the Flux error, but you'll need to check the Daemon log to be sure. Deleting your post with the log in case there's sensitive info in it.

VitorFrost commented 1 year ago 
2022-11-27 19:43:30,978 - INFO - mycodo.controllers.controller_function_a4fce84a - Deactivated in 216.3 ms
2022-11-27 19:43:30,979 - DEBUG - mycodo.daemon - Function controller with ID a4fce84a-54da-4953-8d70-33502ea92479 deactivated.
2022-11-27 19:43:43,870 - INFO - mycodo.function.regulate_ph_ec_a4fce84a - DEBUG00: {'sec_ph_raise': 0, 'sec_ph_lower': 0, 'sec_ec_a': 0, 'sec_ec_b': 0, 'sec_ec_c': 0, 'sec_ec_d': 0, 'ml_ph_raise': 0, 'ml_ph_lower': 0, 'ml_ec_a': 0, 'ml_ec_b': 0, 'ml_ec_c': 0, 'ml_ec_d': 0}
2022-11-27 19:43:43,871 - INFO - mycodo.function.regulate_ph_ec_a4fce84a - DEBUG01: {'sec_ph_raise': 0, 'sec_ph_lower': 0, 'sec_ec_a': 0, 'sec_ec_b': 0, 'sec_ec_c': 0, 'sec_ec_d': 0, 'ml_ph_raise': 0, 'ml_ph_lower': 0, 'ml_ec_a': 0, 'ml_ec_b': 0, 'ml_ec_c': 0, 'ml_ec_d': 0}
2022-11-27 19:43:43,931 - INFO - mycodo.controllers.controller_function_a4fce84a - Activated in 278.2 ms
2022-11-27 19:43:43,931 - DEBUG - mycodo.daemon - Function controller with ID a4fce84a-54da-4953-8d70-33502ea92479 activated.
2022-11-27 19:43:54,047 - DEBUG - mycodo.function.regulate_ph_ec_a4fce84a - Most recent timestamp and measurement for pH: 1669589024.868923, 3.681312588550714
2022-11-27 19:43:54,048 - ERROR - mycodo.controllers.controller_function_a4fce84a - Exception while running loop()
Traceback (most recent call last):
  File "/var/mycodo-root/mycodo/controllers/controller_function.py", line 76, in loop
    self.run_function.loop()
  File "/home/pi/Mycodo/mycodo/functions/regulate_ph_ec.py", line 754, in loop
    if ((enabled_ph and None in [last_measurement_ec, last_measurement_ph]) or
UnboundLocalError: local variable 'last_measurement_ec' referenced before assignment
VitorFrost commented 1 year ago

image image image

kizniche commented 1 year ago

Fix pushed.

VitorFrost commented 1 year ago

Thank you a lot! No errors now, but no pump got activated!

image

2022-11-27 20:01:31,495 - INFO - mycodo.function.regulate_ph_ec_a4fce84a - DEBUG00: {'sec_ph_raise': 0, 'sec_ph_lower': 0, 'sec_ec_a': 0, 'sec_ec_b': 0, 'sec_ec_c': 0, 'sec_ec_d': 0, 'ml_ph_raise': 0, 'ml_ph_lower': 0, 'ml_ec_a': 0, 'ml_ec_b': 0, 'ml_ec_c': 0, 'ml_ec_d': 0}
2022-11-27 20:01:31,496 - INFO - mycodo.function.regulate_ph_ec_a4fce84a - DEBUG01: {'sec_ph_raise': 0, 'sec_ph_lower': 0, 'sec_ec_a': 0, 'sec_ec_b': 0, 'sec_ec_c': 0, 'sec_ec_d': 0, 'ml_ph_raise': 0, 'ml_ph_lower': 0, 'ml_ec_a': 0, 'ml_ec_b': 0, 'ml_ec_c': 0, 'ml_ec_d': 0}
2022-11-27 20:01:31,568 - INFO - mycodo.controllers.controller_function_a4fce84a - Activated in 332.2 ms
2022-11-27 20:01:31,569 - DEBUG - mycodo.daemon - Function controller with ID a4fce84a-54da-4953-8d70-33502ea92479 activated.
2022-11-27 20:01:41,736 - DEBUG - mycodo.function.regulate_ph_ec_a4fce84a - Most recent timestamp and measurement for pH: 1669590092.699565, 3.706738022007613
2022-11-27 20:03:21,891 - DEBUG - mycodo.function.regulate_ph_ec_a4fce84a - Most recent timestamp and measurement for pH: 1669590197.746001, 3.655887155093815
2022-11-27 20:05:01,703 - DEBUG - mycodo.function.regulate_ph_ec_a4fce84a - Most recent timestamp and measurement for pH: 1669590287.676319, 3.681312588550714
2022-11-27 20:06:41,731 - DEBUG - mycodo.function.regulate_ph_ec_a4fce84a - Most recent timestamp and measurement for pH: 1669590392.662443, 3.655887155093815
2022-11-27 20:08:21,782 - DEBUG - mycodo.function.regulate_ph_ec_a4fce84a - Most recent timestamp and measurement for pH: 1669590497.736295, 3.655887155093815

I will let it running for about 2 hours just to be sure. I need to to other stuffs in the next hours.

kizniche commented 1 year ago

I found and fixed a typo in a variable name that was preventing the pH regulation from working.

VitorFrost commented 1 year ago

Now seems to work fine! I have only tested without any liquid being added. Tomorrow I will try it adding real liquid.

2022-11-28 01:01:39,746 - INFO - mycodo.controllers.controller_function_f17bf2df - Activated in 290.8 ms
2022-11-28 01:01:39,746 - DEBUG - mycodo.daemon - Function controller with ID f17bf2df-d5d1-43cd-b9fa-9aec9e87cb2b activated.
2022-11-28 01:01:49,920 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - Most recent timestamp and measurement for pH: 1669608108.622033, 3.6431744383653673
2022-11-28 01:01:49,920 - INFO - mycodo.function.regulate_ph_ec_f17bf2df - TEST00: True, 3.6431744383653673, 6.0
2022-11-28 01:01:49,921 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - pH is dangerously low: 3.64. Should be > 6.00. Dispensing 10.0 ml base
2022-11-28 01:01:49,975 - DEBUG - mycodo.controllers.controller_output - output_on_off(5b06a78c-52e3-465c-9592-c569744f7922, on, 0, vol, 10.0, 0.0, True)
2022-11-28 01:02:49,969 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - Most recent timestamp and measurement for pH: 1669608168.61007, 3.6685998718222663
2022-11-28 01:02:49,969 - INFO - mycodo.function.regulate_ph_ec_f17bf2df - TEST00: True, 3.6685998718222663, 6.0
2022-11-28 01:02:49,969 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - pH is dangerously low: 3.67. Should be > 6.00. Dispensing 10.0 ml base
2022-11-28 01:02:50,010 - DEBUG - mycodo.controllers.controller_output - output_on_off(5b06a78c-52e3-465c-9592-c569744f7922, on, 0, vol, 10.0, 0.0, True)
2022-11-28 01:03:31,363 - DEBUG - mycodo.pyro_monitor - Pyro5 daemon monitor: daemon_status() response: 'alive'
2022-11-28 01:03:50,004 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - Most recent timestamp and measurement for pH: 1669608228.635254, 3.655887155093815
2022-11-28 01:03:50,004 - INFO - mycodo.function.regulate_ph_ec_f17bf2df - TEST00: True, 3.655887155093815, 6.0
2022-11-28 01:03:50,004 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - pH is dangerously low: 3.66. Should be > 6.00. Dispensing 10.0 ml base
2022-11-28 01:03:50,773 - DEBUG - mycodo.controllers.controller_output - output_on_off(5b06a78c-52e3-465c-9592-c569744f7922, on, 0, vol, 10.0, 0.0, True)
2022-11-28 01:04:49,995 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - Most recent timestamp and measurement for pH: 1669608288.562504, 3.681312588550714
2022-11-28 01:04:49,996 - INFO - mycodo.function.regulate_ph_ec_f17bf2df - TEST00: True, 3.681312588550714, 6.0
2022-11-28 01:04:49,996 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - pH is dangerously low: 3.68. Should be > 6.00. Dispensing 10.0 ml base
2022-11-28 01:04:50,064 - DEBUG - mycodo.controllers.controller_output - output_on_off(5b06a78c-52e3-465c-9592-c569744f7922, on, 0, vol, 10.0, 0.0, True)
2022-11-28 01:05:50,067 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - Most recent timestamp and measurement for pH: 1669608348.556283, 3.6940253052791654
2022-11-28 01:05:50,067 - INFO - mycodo.function.regulate_ph_ec_f17bf2df - TEST00: True, 3.6940253052791654, 6.0
2022-11-28 01:05:50,067 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - pH is dangerously low: 3.69. Should be > 6.00. Dispensing 10.0 ml base
2022-11-28 01:05:50,110 - DEBUG - mycodo.controllers.controller_output - output_on_off(5b06a78c-52e3-465c-9592-c569744f7922, on, 0, vol, 10.0, 0.0, True)
2022-11-28 01:06:49,854 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - Most recent timestamp and measurement for pH: 1669608408.557414, 3.655887155093815
2022-11-28 01:06:49,854 - INFO - mycodo.function.regulate_ph_ec_f17bf2df - TEST00: True, 3.655887155093815, 6.0
2022-11-28 01:06:49,855 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - pH is dangerously low: 3.66. Should be > 6.00. Dispensing 10.0 ml base
2022-11-28 01:06:49,900 - DEBUG - mycodo.controllers.controller_output - output_on_off(5b06a78c-52e3-465c-9592-c569744f7922, on, 0, vol, 10.0, 0.0, True)
2022-11-28 01:07:49,892 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - Most recent timestamp and measurement for pH: 1669608468.600766, 3.630461721636916
2022-11-28 01:07:49,893 - INFO - mycodo.function.regulate_ph_ec_f17bf2df - TEST00: True, 3.630461721636916, 6.0
2022-11-28 01:07:49,893 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - pH is dangerously low: 3.63. Should be > 6.00. Dispensing 10.0 ml base
2022-11-28 01:07:49,936 - DEBUG - mycodo.controllers.controller_output - output_on_off(5b06a78c-52e3-465c-9592-c569744f7922, on, 0, vol, 10.0, 0.0, True)
2022-11-28 01:08:50,052 - DEBUG - mycodo.function.regulate_ph_ec_f17bf2df - Most recent timestamp and measurement for pH: 1669608528.63391, 3.6940253052791654
2022-11-28 01:08:50,052 - INFO - mycodo.function.regulate_ph_ec_f17bf2df - TEST00: True, 3.6940253052791654, 6.0