Closed alexvermeersch closed 3 years ago
I hope zbeacon or Ilya Barkov is not ill...
i solved the problem by adding code in json_rest_uplink_converter.py
insert on line 30 y = loads(data) tijd = y["timestamp"] then modify line 63 to if datatype == 'timeseries' : dict_result[datatypes[datatype]].append({"ts": tijd, 'values': {full_key: full_value}})
et voila, problem solved!
Maybe better to insert in line 30 :+1: tijd = 0 try: y = loads(data) tijd = y["timestamp"] except Exception as e: log.error("The data string does not have te element timestamp.") log.exception(e) (have not tried it out...)
Working json_rest_uplink_converter.py:
#
#
#
from time import time from simplejson import dumps, loads from thingsboard_gateway.connectors.rest.rest_converter import RESTConverter, log from thingsboard_gateway.tb_utility.tb_utility import TBUtility
class JsonRESTUplinkConverter(RESTConverter):
def __init__(self, config):
self.__config = config
def convert(self, config, data):
datatypes = {"attributes": "attributes",
"timeseries": "telemetry"}
dict_result = {"deviceName": None, "deviceType": None, "attributes": [], "telemetry": []}
y = loads(data)
tijd = y["timestamp"]
try:
if self.__config.get("deviceNameExpression") is not None:
dict_result["deviceName"] = TBUtility.get_value(self.__config.get("deviceNameExpression"), data, expression_instead_none=True)
else:
log.error("The expression for looking \"deviceName\" not found in config %s", dumps(self.__config))
if self.__config.get("deviceTypeExpression") is not None:
dict_result["deviceType"] = TBUtility.get_value(self.__config.get("deviceTypeExpression"), data, expression_instead_none=True)
else:
log.error("The expression for looking \"deviceType\" not found in config %s", dumps(self.__config))
except Exception as e:
log.error('Error in converter, for config: \n%s\n and message: \n%s\n', dumps(self.__config), data)
log.exception(e)
try:
for datatype in datatypes:
dict_result[datatypes[datatype]] = []
for datatype_config in self.__config.get(datatype, []):
value = TBUtility.get_value(datatype_config["value"], data, datatype_config["type"], expression_instead_none=True)
value_tag = TBUtility.get_value(datatype_config["value"], data, datatype_config["type"], get_tag=True)
key = TBUtility.get_value(datatype_config["key"], data, datatype_config["type"], expression_instead_none=True)
key_tag = TBUtility.get_value(datatype_config["key"], data, get_tag=True)
if ("${" not in str(value) and "}" not in str(value)) \
and ("${" not in str(key) and "}" not in str(key)):
is_valid_key = isinstance(key, str) and "${" in datatype_config["key"] and "}" in datatype_config["key"]
is_valid_value = isinstance(value, str) and "${" in datatype_config["value"] and "}" in datatype_config["value"]
full_key = datatype_config["key"].replace('${' + key_tag + '}', key) if is_valid_key else key
full_value = datatype_config["value"].replace('${' + value_tag + '}', value) if is_valid_value else value
if datatype == 'timeseries' :
dict_result[datatypes[datatype]].append({"ts": tijd, 'values': {full_key: full_value}})
else:
dict_result[datatypes[datatype]].append({full_key: full_value})
except Exception as e:
log.error('Error in converter, for config: \n%s\n and message: \n%s\n', dumps(self.__config), str(data))
log.exception(e)
return dict_result
Hi @alexvermeersch ,
Thank you for your investigations and for hopes. I am glad to inform that I'm okay and I just was on vacation for two weeks. I hope you are well too. Please create a pull request with your fixes and I will merge it into the repository.
Hi Ilya,
releaved to hear you're OK, hope you enjoyed your holliday...
I have solved the issue in 2 steps: First inserted 2 lines at pos 30: y = loads(data) tijd = y["timestamp"] then modified line 58 and 59 to: if datatype == 'timeseries' : dict_result[datatypes[datatype]].append({"ts": tijd, 'values': {full_key: full_value}})
That worked well!
Then i added some safety : tijd = 0 timeformat = '%d/%m/%Y %H:%M:%S' try: y = loads(data) if "timestamp" in data: my_date = y["timestamp"] if "ts" in data: my_date = y["ts"] if not (isinstance(my_date, int)): tijd = (int(datetime.datetime.strptime(my_date, timeformat).strftime("%s")))*1000 else: tijd = my_date except Exception as e: log.error("The data string does not have the element timestamp or ts.") log.exception(e)
If no "timestamp" or "ts" in the data string tijd remains 0 (1 jan 1970) if there is either one and it is an int (unix timestamp) that is copied into tijd, if it is a string timeformat it is converted into unix timestamp (*1000 to get milliseconds)
that works also So the attached json_.....py is the one in my system and it does what it has to do...
kind regards,
Alex.
PS i am not familiar with github so could you tell me how to create a pull?
Van: "Illia Barkov" notifications@github.com Aan: "thingsboard/thingsboard-gateway" thingsboard-gateway@noreply.github.com Cc: "alex vermeersch" alex.vermeersch@telenet.be, "Mention" mention@noreply.github.com Verzonden: Dinsdag 22 december 2020 09:58:26 Onderwerp: Re: [thingsboard/thingsboard-gateway] [BUG]AttributeError: 'str' object has no attribute 'get' (#442)
Hi [ https://github.com/alexvermeersch | @alexvermeersch ] ,
Thank you for your investigations and for hopes. I am glad to inform that I'm okay and I just was on vacation for two weeks. I hope you are well too. Please create a pull request with your fixes and I will merge it into the repository.
— You are receiving this because you were mentioned. Reply to this email directly, [ https://github.com/thingsboard/thingsboard-gateway/issues/442#issuecomment-749429260 | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/ASFFGVCUFMGK7IRBY32ABJLSWBNTFANCNFSM4U74OOEQ | unsubscribe ] .
""2020-12-17 12:52:30" - ERROR - [json_rest_uplink_converter.py] - json_rest_uplink_converter - 62 - 'str' object has no attribute 'get'", Traceback (most recent call last):, File "/usr/local/lib/python3.8/site-packages/thingsboard_gateway-2.5.5.2-py3.8.egg/thingsboard_gateway/connectors/rest/json_rest_uplink_converter.py", line 56, in convert, if datatype == 'timeseries' and (data.get("ts") is not None or data.get("timestamp") is not None):, AttributeError: 'str' object has no attribute 'get' Connector name (If bug in the some connector): gateway using rest.json connector
Thingsboard and gateway installed under docker