kfcook commented 5 years ago

Would it be possible to create a Docker container for rtlamr-collect? I already have rtlamr running in a container, and would love to run the collect variant alongside it.

Thank you for all your work thus far! This is really an awesome project.

jreiners commented 5 years ago

Yes, but you need to run in privileged mode to access the USB port it seems, I've been running this way for a long time now.

On Fri, Jan 11, 2019 at 2:35 PM kfcook notifications@github.com wrote:

Would it be possible to create a Docker container for rtlamr-collect? I already have rtlamr running in a container, and would love to run the collect variant alongside it.

Thank you for all your work thus far! This is really an awesome project.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bemasher/rtlamr-collect/issues/12, or mute the thread https://github.com/notifications/unsubscribe-auth/AHjzfLGWBnGk04YysL7dYVysawMah_QCks5vCPWGgaJpZM4Z8Ism .

-- Justin Reiners

kfcook commented 5 years ago

I'm currently running rtl-sdr as a container in privileged mode and that's working well. I also got rtlamr in a container talking to my rtl-sdr container. How would I go about getting rtlamr-collect in a container and talking back to my rtlamr container?

Thanks!!

jreiners commented 5 years ago

I run my whole setup in docker-compose so it's possible, I run mysql, grafana, a weather script, rtlamr, and rtlamr-collect in multiple containers right now.

On Fri, Jan 11, 2019 at 3:23 PM kfcook notifications@github.com wrote:

I'm currently running rtl-sdr as a container in privileged mode and that's working well. I also got rtlamr in a container talking to my rtl-sdr container. How would I go about getting rtlamr-collect in a container and talking back to my rtlamr container?

Thanks!!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bemasher/rtlamr-collect/issues/12#issuecomment-453660887, or mute the thread https://github.com/notifications/unsubscribe-auth/AHjzfNfpSU7puHYOuBv7iThk2Y1Jmw-Eks5vCQDBgaJpZM4Z8Ism .

-- Justin Reiners

kfcook commented 5 years ago

Awesome! Do you mind sharing your dockerfile/docker-compose files for rtlamr-collect? I already have containers set up for the rest of the components. Thanks!

jreiners commented 5 years ago

I forgot... I just realized that I'm not using relamr-collect anymore, as I had problems with it all stopping randomly. I ended up modifying someone else's code that more closely matched my setup. It still uses rtlamr, I cloned utilitymon and removed a bunch of stuff I didn't need and it works perfectly. I have my setup uploaded to github under jreiners/metermaid. Sorry bemasher :\

Instead of my metermaid.py. you should be able to substitute rtlamr-collect and use his DB stuff as well.

metermaid.yml

version: '3.1'

services:

weatherdb: image: mariadb restart: always environment: MYSQL_ROOT_PASSWORD: weatherdb ports:

3306:3306 volumes:
weather-mysql:/var/lib/mysql networks:
weather

grafana: image: grafana/grafana restart: always ports:
3000:3000 volumes:
grafana-storage:/var/lib/grafana networks:
weather depends_on:
weatherdb

metermaid: build: ./metermaid/ ports:
1234:1234 volumes:
- metermaid-storage:/app
/dev/bus/usb:/dev/bus/usb

privileged: True networks:
weather depends_on:
weatherdb

weatherscript: build: ./weather/ restart: always networks:
weather depends_on:
weatherdb

volumes: weather-mysql: grafana-storage: metermaid-storage:

networks: weather:

rtlamr docker container From centos:latest RUN yum -y update RUN yum -y install wget vim tar epel-release git nc tee tree && yum -y install rtl-sdr python34 python34-mysql python34-pip RUN wget https://dl.google.com/go/go1.10.1.linux-amd64.tar.gz RUN tar -xvf go1.10.1.linux-amd64.tar.gz && mv go /usr/local/ RUN mkdir /projects/ RUN mkdir -p /app/bin && cd /app/bin ENV GOROOT=/usr/local/go ENV GOPATH=/projects ENV PATH=$GOPATH/bin:$GOROOT/bin:$PATH:/app/bin/ COPY code /app/bin/ RUN python3 -m pip install pymysql RUN /usr/local/go/bin/go get github.com/bemasher/rtlamr && /usr/local/go/bin/go get github.com/bemasher/rtlamr-collect RUN python3 -m pip install mysql-connector-python simplejson mysql-connector requests ENTRYPOINT python3 /app/bin/metermaid.py

stats collector:

!/usr/bin/env python3.4

Depends:

netcat, rtlamr, rtl_tcp

import subprocess import socket import re import time import pymysql import utility_meters

adjustable path variables.

meters = {} electricMeterTypes = [4, 5, 7, 8] gasMeterTypes = [2, 9, 12] waterMeterTypes = [11, 13]

exec(open("dbConnect.py").read())

dbConn = pymysql.connect(user='root', password='weatherdb', host='weatherdb', database='metermaid', autocommit=True) dbCur = dbConn.cursor()

start the rtl device and server

print("Sarting RTL_TCP Out Port 5566") rtl_tcp = subprocess.Popen("rtl_tcp -p 5566", stdout=subprocess.PIPE, shell=True) # have to manually start it ??? print("sleeping 15 seconds") time.sleep(15)

print("Sarting RTLAMR In Port 5566; Out on 5577") print("starting rtlamr") rtlamr = subprocess.Popen("rtlamr -server 127.0.0.1:5566 | nc -l -p 5577", stdout=subprocess.PIPE, shell=True) print("sleeping 5 seconds") time.sleep(5)

rtlSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) rtlSocket.connect(('127.0.0.1', 5577))

rtlSocket.recv(498) # skip the initial lines while True:

recieve line, always same length when using plaintext format

pkt = rtlSocket.recv(111).decode("ascii")
rtlSocket.recv(1) # recieve '\n'
# print(pkt)
# reg split the line; figure out why it sometimes doesn't work
# {Time:2015-10-10T12:48:47.704 SCM:{ID:42354463 Type: 5 Tamper:{Phy:01

Enc:03} Consumption: 5559353 CRC:0xBB11}} try: pktRegex = re.search("Time\:(\S+) .+ID\:(\d+) +Type\: (\d{1,2}) .+Consumption: (\d+)", pkt) pDate = pktRegex.group(1) pEpoch = time.mktime(time.strptime(pDate, "%Y-%m-%dT%H:%M:%S.%f")) pId = str(pktRegex.group(2)) pType = str(pktRegex.group(3))

consumption (xxxyy) kWh * 10 = Wh

    pConsumption = int(pktRegex.group(4))
    pWh = pConsumption * 10
except AttributeError:
    print("Error: Received a corrupted packet from stream")
    continue

# electric; type 4, 5, 7, 8
if int(pType) in electricMeterTypes:
    if pId not in meters:
        meters[pId] = utility_meters.ElectricMeter(pId, pType, pEpoch,

pWh, dbCur) watts = meters[pId].getCurrentWatts(pEpoch, pWh)

# gas; type 2, 9, 12
elif int(pType) in gasMeterTypes:
    if pId not in meters:
        meters[pId] = utility_meters.GasMeter(pId, pType, pEpoch,

pConsumption, dbCur) gasPerSec = meters[pId].getGasPerSec(pEpoch, pConsumption)

# water; type 11, 13
elif int(pType) in waterMeterTypes:
    if pId not in meters:
        meters[pId] = utility_meters.WaterMeter(pId, pType, pEpoch,

pConsumption, dbCur) waterPerSec = meters[pId].getWaterPerSec(pEpoch, pConsumption) else: print("Meter type not recognized")

mysql table structure:

CREATE TABLE utilities ( mPrimaryKey int(6) unsigned NOT NULL AUTO_INCREMENT, mId varchar(50) DEFAULT NULL, mType int(2) DEFAULT NULL, mTime int(11) DEFAULT NULL, mTotalConsumption int(11) DEFAULT NULL, mConsumed float DEFAULT NULL, PRIMARY KEY (mPrimaryKey) ) ENGINE=InnoDB AUTO_INCREMENT=1068623 DEFAULT CHARSET=latin1 ROW_FORMAT=COMPRESSED

grafana query that seems to work well:

GAS:

SELECT mId as metric, mTime as time_sec, mConsumed as value FROM metermaid.utilities WHERE mId='65921122' and $__unixEpochFilter(mTime) ;

ELECTRIC:

SELECT mId as metric, mTime as time_sec, mConsumed as value FROM metermaid.utilities WHERE mId = '57444563' and $__unixEpochFilter(mTime) ;

My water meter doesn't work with my SDR :(

On Fri, Jan 11, 2019 at 3:48 PM kfcook notifications@github.com wrote:

Awesome! Do you mind sharing your dockerfile/docker-compose files for rtlamr-collect? I already have containers set up for the rest of the components. Thanks!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bemasher/rtlamr-collect/issues/12#issuecomment-453664920, or mute the thread https://github.com/notifications/unsubscribe-auth/AHjzfBsBgqtKakqmiGAtdce6TKYWIskcks5vCQQsgaJpZM4Z8Ism .

-- Justin Reiners

jreiners commented 5 years ago

It's located under /app/bin in my docker container (rtlamr-collect that is)

On Fri, Jan 11, 2019 at 4:17 PM Justin Reiners justin.reiners@gmail.com wrote:

I forgot... I just realized that I'm not using relamr-collect anymore, as I had problems with it all stopping randomly. I ended up modifying someone else's code that more closely matched my setup. It still uses rtlamr, I cloned utilitymon and removed a bunch of stuff I didn't need and it works perfectly. I have my setup uploaded to github under jreiners/metermaid. Sorry bemasher :\

Instead of my metermaid.py. you should be able to substitute rtlamr-collect and use his DB stuff as well.

metermaid.yml

version: '3.1'

services:

weatherdb: image: mariadb restart: always environment: MYSQL_ROOT_PASSWORD: weatherdb ports:

3306:3306 volumes:

weather-mysql:/var/lib/mysql networks:

weather

grafana: image: grafana/grafana restart: always ports:

3000:3000 volumes:

grafana-storage:/var/lib/grafana networks:

weather depends_on:

weatherdb

metermaid: build: ./metermaid/ ports:

1234:1234 volumes:
- metermaid-storage:/app

/dev/bus/usb:/dev/bus/usb

privileged: True networks:

weather depends_on:

weatherdb

weatherscript: build: ./weather/ restart: always networks:

weather depends_on:

weatherdb

volumes: weather-mysql: grafana-storage: metermaid-storage:

networks: weather:

rtlamr docker container From centos:latest RUN yum -y update RUN yum -y install wget vim tar epel-release git nc tee tree && yum -y install rtl-sdr python34 python34-mysql python34-pip RUN wget https://dl.google.com/go/go1.10.1.linux-amd64.tar.gz RUN tar -xvf go1.10.1.linux-amd64.tar.gz && mv go /usr/local/ RUN mkdir /projects/ RUN mkdir -p /app/bin && cd /app/bin ENV GOROOT=/usr/local/go ENV GOPATH=/projects ENV PATH=$GOPATH/bin:$GOROOT/bin:$PATH:/app/bin/ COPY code /app/bin/ RUN python3 -m pip install pymysql RUN /usr/local/go/bin/go get github.com/bemasher/rtlamr && /usr/local/go/bin/go get github.com/bemasher/rtlamr-collect RUN python3 -m pip install mysql-connector-python simplejson mysql-connector requests ENTRYPOINT python3 /app/bin/metermaid.py

stats collector:

!/usr/bin/env python3.4

Depends:

netcat, rtlamr, rtl_tcp

import subprocess import socket import re import time import pymysql import utility_meters

adjustable path variables.

meters = {} electricMeterTypes = [4, 5, 7, 8] gasMeterTypes = [2, 9, 12] waterMeterTypes = [11, 13]

exec(open("dbConnect.py").read())

dbConn = pymysql.connect(user='root', password='weatherdb', host='weatherdb', database='metermaid', autocommit=True) dbCur = dbConn.cursor()

start the rtl device and server

print("Sarting RTL_TCP Out Port 5566") rtl_tcp = subprocess.Popen("rtl_tcp -p 5566", stdout=subprocess.PIPE, shell=True) # have to manually start it ??? print("sleeping 15 seconds") time.sleep(15)

print("Sarting RTLAMR In Port 5566; Out on 5577") print("starting rtlamr") rtlamr = subprocess.Popen("rtlamr -server 127.0.0.1:5566 | nc -l -p 5577", stdout=subprocess.PIPE, shell=True) print("sleeping 5 seconds") time.sleep(5)

rtlSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) rtlSocket.connect(('127.0.0.1', 5577))

rtlSocket.recv(498) # skip the initial lines while True:

recieve line, always same length when using plaintext format
pkt = rtlSocket.recv(111).decode("ascii")
rtlSocket.recv(1) # recieve '\n'
# print(pkt)
# reg split the line; figure out why it sometimes doesn't work
# {Time:2015-10-10T12:48:47.704 SCM:{ID:42354463 Type: 5
Tamper:{Phy:01 Enc:03} Consumption: 5559353 CRC:0xBB11}} try: pktRegex = re.search("Time\:(\S+) .+ID\:(\d+) +Type\: (\d{1,2}) .+Consumption: (\d+)", pkt) pDate = pktRegex.group(1) pEpoch = time.mktime(time.strptime(pDate, "%Y-%m-%dT%H:%M:%S.%f")) pId = str(pktRegex.group(2)) pType = str(pktRegex.group(3))

consumption (xxxyy) kWh * 10 = Wh
    pConsumption = int(pktRegex.group(4))
    pWh = pConsumption * 10
except AttributeError:
    print("Error: Received a corrupted packet from stream")
    continue

# electric; type 4, 5, 7, 8
if int(pType) in electricMeterTypes:
    if pId not in meters:
        meters[pId] = utility_meters.ElectricMeter(pId, pType, pEpoch,
pWh, dbCur) watts = meters[pId].getCurrentWatts(pEpoch, pWh)
# gas; type 2, 9, 12
elif int(pType) in gasMeterTypes:
    if pId not in meters:
        meters[pId] = utility_meters.GasMeter(pId, pType, pEpoch,
pConsumption, dbCur) gasPerSec = meters[pId].getGasPerSec(pEpoch, pConsumption)
# water; type 11, 13
elif int(pType) in waterMeterTypes:
    if pId not in meters:
        meters[pId] = utility_meters.WaterMeter(pId, pType, pEpoch,
pConsumption, dbCur) waterPerSec = meters[pId].getWaterPerSec(pEpoch, pConsumption) else: print("Meter type not recognized")

mysql table structure:

CREATE TABLE utilities ( mPrimaryKey int(6) unsigned NOT NULL AUTO_INCREMENT, mId varchar(50) DEFAULT NULL, mType int(2) DEFAULT NULL, mTime int(11) DEFAULT NULL, mTotalConsumption int(11) DEFAULT NULL, mConsumed float DEFAULT NULL, PRIMARY KEY (mPrimaryKey) ) ENGINE=InnoDB AUTO_INCREMENT=1068623 DEFAULT CHARSET=latin1 ROW_FORMAT=COMPRESSED

grafana query that seems to work well:

GAS:

SELECT mId as metric, mTime as time_sec, mConsumed as value FROM metermaid.utilities WHERE mId='65921122' and $__unixEpochFilter(mTime) ;

ELECTRIC:

SELECT mId as metric, mTime as time_sec, mConsumed as value FROM metermaid.utilities WHERE mId = '57444563' and $__unixEpochFilter(mTime) ;

My water meter doesn't work with my SDR :(

On Fri, Jan 11, 2019 at 3:48 PM kfcook notifications@github.com wrote:

Awesome! Do you mind sharing your dockerfile/docker-compose files for rtlamr-collect? I already have containers set up for the rest of the components. Thanks!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bemasher/rtlamr-collect/issues/12#issuecomment-453664920, or mute the thread https://github.com/notifications/unsubscribe-auth/AHjzfBsBgqtKakqmiGAtdce6TKYWIskcks5vCQQsgaJpZM4Z8Ism .

-- Justin Reiners

-- Justin Reiners

bemasher commented 5 years ago

This is out of scope for this project, I don't have the resources to test and keep something like this up to date.

bemasher / rtlamr-collect

Create Docker container for rtlamr-collect #12

- metermaid-storage:/app

!/usr/bin/env python3.4

Depends:

netcat, rtlamr, rtl_tcp

adjustable path variables.

exec(open("dbConnect.py").read())

start the rtl device and server

recieve line, always same length when using plaintext format

consumption (xxxyy) kWh * 10 = Wh

- metermaid-storage:/app

!/usr/bin/env python3.4

Depends:

netcat, rtlamr, rtl_tcp

adjustable path variables.

exec(open("dbConnect.py").read())

start the rtl device and server

recieve line, always same length when using plaintext format

consumption (xxxyy) kWh * 10 = Wh