Ecowitt WS68 Anemometer

tolip-ydob commented 4 years ago

I'd like to point out that I tried many times to attach a .zip, .gz, or .tar file to this post. I did manage to get github to format this post properly.

Ecowitt WS68
Lux, Battery, Wind speed, Wind direction, Gust speed,  and UV

I'm posting what I have so far here. I am willing to submit to git repository but I am ignorant, I never added files or submitted a pull request before.
Is there a cheat sheet for doing this? I do read docs but I'm netested too deep at the moment.

My command line
rtl_433 -d 0 -f 915e6 -g 14 -M newmodel -X "n=WS68_WAS_WH51_soil_sensor,m=FSK_PCM,s=58,l=58,r=5000,g=4000,preamble=aa2dd4" > X.68hex.stdxxx.out 2>&1

I found the Flex decoder in rtl_433/src/devices/fineoffset.c line # 516  as of 20200130 

I used low-ish gain because device is close to rx and it has too much juice otherwise.
I am also in a target rich environment and too much stuff spews forth otherwise. 

The fun begins :-)

680000c500004b0fffff00000000d0af104 ### WS68            4b-VDC                   00-Wind-Direction-North 
680000c500004b0fffff005a0000d0af104 ### WS68            4b-VDC                   5a-Wind-Direction-East
680000c500004b0fffff00b4000079b2102 ### WS68            4b-VDC                   b4-Wind-Direction-South
680000c500004b0fffff7ee0940075ec102 ### WS68            4b-VDC  7e-Wind-Speed    e0-Wind-Direction-SouthWest      94-Wind-Gust
680000c500004b2fffff000e00008033208 ### WS68            4b-VDC                  10e-wind-Direction-West
680000c5000f4b0fffff002e0000d395108 ### WS68   0f-Lux   4b-VDC                   2e-Wind-Direction-NorthEast-ish
680000c501074b0fffff002e0002a663100 ### WS68   107-Lux  4b-VDC                   2e-Wind-Direction-NorthEast-ish

The .cu8 files are labeled similar to the descriptions above, the filenames contain the hex value to expect. 

Use a small fixed width font and my ASCIIdrawing(below) will look correct on a 180 column terminal. They are capital "i" and capital "o" because I used sed LOL.

OIIO IOOO OOOO OOOO OOOO OOOO IIOO OIOI OOOO OOOO OOOO OOOO OIOO IOII OOOO IIII IIII IIII IIII IIII OOOO OOOO OOOO OOOO OOOO OOOO OOOO OOOO IIOI OOOO IOIO IIII OOOI OOOO OIOO 
OIIO IOOO OOOO OOOO OOOO OOOO IIOO OIOI OOOO OOOO OOOO OOOO OIOO IOII OOOO IIII IIII IIII IIII IIII OOOO OOOO OIOI IOIO OOOO OOOO OOOO OOOO IIOI OOOO IOIO IIII OOOI OOOO OIOO 
OIIO IOOO OOOO OOOO OOOO OOOO IIOO OIOI OOOO OOOO OOOO OOOO OIOO IOII OOOO IIII IIII IIII IIII IIII OOOO OOOO IOII OIOO OOOO OOOO OOOO OOOO OIII IOOI IOII OOIO OOOI OOOO OOIO 
OIIO IOOO OOOO OOOO OOOO OOOO IIOO OIOI OOOO OOOO OOOO OOOO OIOO IOII OOOO IIII IIII IIII IIII IIII OIII IIIO IIIO OOOO IOOI OIOO OOOO OOOO OIII OIOI IIIO IIOO OOOI OOOO OOIO 
OIIO IOOO OOOO OOOO OOOO OOOO IIOO OIOI OOOO OOOO OOOO OOOO OIOO IOII OOIO IIII IIII IIII IIII IIII OOOO OOOO OOOO IIIO OOOO OOOO OOOO OOOO IOOO OOOO OOII OOII OOIO OOOO IOOO 
OIIO IOOO OOOO OOOO OOOO OOOO IIOO OIOI OOOO OOOO OOOO IIII OIOO IOII OOOO IIII IIII IIII IIII IIII OOOO OOOO OOIO IIIO OOOO OOOO OOOO OOOO IIOI OOII IOOI OIOI OOOI OOOO IOOO 
OIIO IOOO OOOO OOOO OOOO OOOO IIOO OIOI OOOO OOOI OOOO OIII OIOO IOII OOOO IIII IIII IIII IIII IIII OOOO OOOO OOIO IIIO OOOO OOOO OOOO OOIO IOIO OIIO OIIO OOII OOOI OOOO OOOO 
|||| ||||                                       | |||| |||| |||| ||||   |  \-static-(20bits)------/ |||| |||| |||| |||| |||| ||||
   6    8                                       1    0    7    4    b   |     f    f    f    f    f |||| |||| |||| |||| |||| ||||
\Device-/                                       \Lux------/ \Batt-DVC   |                           \-speed-/ |||| |||| |||| ||||
 WS68                                                                   |                                     |||| |||| |||| ||||
                                                                        \-Wind-Direction-(9bits)----------------------/ |||| ||||
                                                                                                                        \-gust--/

### I don't know where the UV Index is hiding, we need at least 4 more bits for this puzzle. Last year I didn't see any UV until May and it's January as I type this.
### Some error bits are probably missing and the speed may be split if the middle static bits are length 22 and 'protected' by zeros.

### I saw the following values for Batt-VDC  3b, 3c, 42, 4b, 4c, 5c.
### The battery compartment has 3 recessed conductors spaced aproptiately for typical 3.7V rechargeable lithium.
### It powered on at 3b and 5c was over 1.6VDC, 1.485VDC battery gave me 42 and 4b.
### The device runs with a single 1.5VDC AA battery.
### there is a 5.5V 3Farad supercap in there so who knows what it actually measures.
### I had RFI preventing capture of pkts during battery tests due to junk bench supply, no proof :-(,,,

### I'm not a Lux-ologist So I don't know what to expect from a lightbulb.
### I see values of 7000 from my WS24 that is outside.

Other notes not specific to the WS68 follow, someone might find it useful if they are new to mucking about! :-)
I'm new to this rx data stuff, when I was a kid they delivered data over a wire, gotta start somewhere. 

I found it useful when packet hunting with the rtl_433 -A switch to capture all screen output to file.

rtl_433 -d 0 -f 915e6 -g 14 -M newmodel -A > A.68hex.stdxxx.out 2>&1
or
rtl_433 -d 0 -f 915e6 -g 14 -M newmodel -A > A.68hex.stdxxx.out 2>&1 &

To run commands in the background just add an ampersand to the end &
If you have only one SSH connection to a remote box you can still do 2 things at once.

There are tradeoffs to this method but there are work arounds.
In both instances further steps are needed to view output.

In the first instance you need another terminal/console to look at the data while it is running.
You will not see any error messages if rtl_433 pukes. Perhaps your rtlsdr is busy etc.
It takes an extra step to look in output file for error messages to troubleshoot.

Secondly you might forget that rtl_433 is already running in the background and wonder why it won't connect to the rtlsdr.
From the command line to see what is running in the background FROM THE CONSOLE YOU LAUNCHED rtl_433 from type 
jobs

To bring it to the foreground to perhaps end the process type
fg

If you have more than 1 background process 'jobs' will tell you and "fg 2" would bring the 2nd one listed by 'jobs' to the foreground.

Then I use grep for hex 'codes' and 'flex' decoders that rtl_433 tried.
grep codes A.68hex.stdxxx.out
or
grep flex A.68hex.stdxxx.out

If I am hunting for something that sends data every 'x' seconds I  time my capture to only gather 'y' packets.
x * y -1 = T

For example a device xmits every 16 seconds and I want exactly 3 packets.
16 * 4 - 1 = 63
So I add "-T 63" to my rtl_433 command line.

I only have 1/2 a clue where to begin with flex decoders, so I grepped in the source code for hints.
Once I have a flex decoder producing repeatable output that might be correct I still capture to a file but view the hex in real time thusly.

rtl_433 yadda yadda & ### run it in the background.

tail -f output.file.from.rtl_433

tail only shows the end of a file and then new data(follow) if you use the -f switch.
If you do not get anything use 'tail -f -n +1' instead to start from line #1. 
You can quit from tail and rtl_433 is still chugging along in the background gathering data.
As mentioned above, you can add 'grep' to the end of commands to filter it.

tail -f output.file.from.rtl_433 | grep codes

And tail will keep tailing, passing its output to grep as it arrives.
You can filter directly from rtl_433 but not everything easily.
In the beginning it may be helpful to add timestamps to the codes.

tail -f output.file.from.rtl_433 | grep -e time -e codes

Depending on your environment you may be getting lots of unneeded unrelated data.
grep can ignore stuff too by adding "-v"
Refine the search by adding an ignore filter.
or
Suppose you have a device that misses packets sometimes and you want to see who else is 'chatty' possibly stepping on your tx/rx.

tail -f output.file.from.rtl_433 | grep codes | grep -v some.pattern.to.ignore

If it's still too much pipe into another grep.
It may look cumbersome bit just hit the up arrow and refine your search.
Added benifit is your 'history' command will remember them when you 'forgot'.

history | grep rtl_433

I made an alias called hgrep it comes in handy when tangents are involved.

alias hgrep="history | grep $1"

If you like such things long term add them to your  .profile or .bashrc or whatever your OS uses for such storage of such things.

If I have rtl_433 reliably decoding packets
Then I can try and make sense of the packets while forcing my intentions on the environment to cause sensor data to change predictably.
I do this by some combination of grepping for codes and narrowing it down to the ones of interest.

I put that output in a destination file and edit it down to just the string of hex digits.
You can do it with sed but the curly braces make the commands a bit cryptic.

I brute forced the hex2bin with sed because my hand made chart of hex2bin values is fun but tedious.
I put the following in a file called "sedhex2bin". Usage "sed -f sedhex2bin filename_of_file_containing_ONLY_hex_values_to_translate". 
s/0/OOOO /g 
s/1/OOOI /g
s/2/OOIO /g 
s/3/OOII /g
s/4/OIOO /g
s/5/OIOI /g
s/6/OIIO /g
s/7/OIII /g
s/8/IOOO /g
s/9/IOOI /g
s/a/IOIO /g
s/b/IOII /g
s/c/IIOO /g
s/d/IIOI /g
s/e/IIIO /g
s/f/IIII /g
s/A/IOIO /g
s/B/IOII /g
s/C/IIOO /g
s/D/IIOI /g
s/E/IIIO /g
s/F/IIII /g

The other tools I tried vexed me due to arbitrary length lines of input that pkts may exhibit. Input file must contain only hex to produce neat output. sed can do that too!
This works with both uppercase and lowercase hex, I'm a caveman with a just rock and a thigh bone so I did not use to-upper or to-lower, I just pounded more.
To turn the i's and o's back to 1's and 0's add   " | sed s/I/1/g | sed s/O/0/g >new_filename"
To turn the i's and o's      to -'s and _'s add   " | sed s/I/-/g | sed s/O/_/g >new_filename" for logic_analyzer-ish output
Can use any printable characters that do not appear in input file for fun or variety.
_--_ -___ ____ ____ ____ ____ --__ _-_- ____ ____ ____ ____ _-__ -_-- ____ ---- ---- ---- ---- ---- ____ ____ ____ ____ ____ ____ ____ ____ --_- ____ -_-_ ---- ___- ____ _-__ 
_--_ -___ ____ ____ ____ ____ --__ _-_- ____ ____ ____ ____ _-__ -_-- ____ ---- ---- ---- ---- ---- ____ ____ _-_- -_-_ ____ ____ ____ ____ --_- ____ -_-_ ---- ___- ____ _-__ 
_--_ -___ ____ ____ ____ ____ --__ _-_- ____ ____ ____ ____ _-__ -_-- ____ ---- ---- ---- ---- ---- ____ ____ -_-- _-__ ____ ____ ____ ____ _--- -__- -_-- __-_ ___- ____ __-_ 
_--_ -___ ____ ____ ____ ____ --__ _-_- ____ ____ ____ ____ _-__ -_-- ____ ---- ---- ---- ---- ---- _--- ---_ ---_ ____ -__- _-__ ____ ____ _--- _-_- ---_ --__ ___- ____ __-_ 
_--_ -___ ____ ____ ____ ____ --__ _-_- ____ ____ ____ ____ _-__ -_-- __-_ ---- ---- ---- ---- ---- ____ ____ ____ ---_ ____ ____ ____ ____ -___ ____ __-- __-- __-_ ____ -___ 
_--_ -___ ____ ____ ____ ____ --__ _-_- ____ ____ ____ ---- _-__ -_-- ____ ---- ---- ---- ---- ---- ____ ____ __-_ ---_ ____ ____ ____ ____ --_- __-- -__- _-_- ___- ____ -___ 
_--_ -___ ____ ____ ____ ____ --__ _-_- ____ ___- ____ _--- _-__ -_-- ____ ---- ---- ---- ---- ---- ____ ____ __-_ ---_ ____ ____ ____ __-_ -_-_ _--_ _--_ __-- ___- ____ ____

The entire command line to produce the above from a file full of hex values in 'logic analyzer ish' output to a file.
sed -f sedhex2bin filename_of_file_containing_ONLY_hex_values_to_translate | sed s/I/-/g | sed s/O/_/g >hex2bin.logic.outfile
If you are using a variable width font you may need to get creative.
Hope this is useful! :-)

zuckschwerdt commented 4 years ago

Thanks! That a really good analysis so far!

You can just drop the codes into a BitBench to try and document the decoding.

I'll try to verify the checksum soon.

zuckschwerdt commented 4 years ago

The two checksums are:

crc8(&b[0], 15, 0x31, 0x00) == 0
add_bytes(&b[0], 15) - b[15] == 0

zuckschwerdt commented 4 years ago

There is now some preliminary support, the basic output should work. All values need testing and conversion to unit values still.

tolip-ydob commented 4 years ago

I like Bitbench To add a bit more detail to the WDIR_H nibble in your example. change "WDIR_H:4h" to "?2b WDIR_H:1b ?1b" edit #1 to your BitBench example

Wow, on the basic output support. That was quick. Thank you.

I'll pull, build, and test.

tolip-ydob commented 4 years ago

compiles, runs, and decodes the packets very reliably Well done.

zuckschwerdt commented 4 years ago

It was nearly the same as EcoWitt WH40 (and AmbientWeather WH31E).

The wind dir you mention above is currently wrong and all the values need careful testing and we need to figure out the units.

tolip-ydob commented 4 years ago

north is hex 00, decimal 00 compass degrees east is hex 5a, decimal 90 south is hex b4, decimal 180 west is hex 1 0e, decimal 270

The WDIR_H shows up as a 2 right before the five fffff the five fffff might have a 'start bit' in front of it or it is used for something else.

on that bitbench link you posted I changed WDIR_H:4h to ?:2b WDIR_H:1b ?:1b

I thought the link I posted included the changes. My ASCII art shows it as a single bit, a "2" in hex.

Today is predicted 15MPH winds, I'll walk it outside and guestimate how much wind the trees are stealing. My Ambient WH25 is up a 9m tall treestump on a short pole and it is not possible to make a side by side comparison at this time.

tolip-ydob commented 4 years ago

if you change in your Bitbench example WDIR_LO:8h to WDIR_LO:8d it makes more sense visually I don't know how to add the 9th bit using Bitbench 9 bits binary displayed in decimal as compass degrees.

zuckschwerdt commented 4 years ago

Thanks. I'll fix the wdir bit together with wind speed, gust, and lux once the correct units for those are known.

tolip-ydob commented 4 years ago

More clues

Ambient says Light : 31093.6 lux

Ecowitt says lux : 4929 lux : 4727 lux : 4603

I got the cups on the Ecowitt to rotate in the wind but the Abmient up the pole was a blur and my testing on the ground was not close enough to compare. I'll try and solicit input from a weewx user, it supports both the WS68 and rtl_433 seperately.

Thanks again for the work done so far! :-)

zuckschwerdt commented 4 years ago

lux might be tricky due to the high dynamic range. Some lux fields are encoded not as just a number but as mantissa and exponent. So we might need some samples for different scales.

buelowp commented 2 years ago

I have some questions about support for this device (Ecowitt WS68BN). Since this is still open, I hope this is the right place. I have been able to make this work a bit, but I suspect I'm missing something.

I'm using a Nooelec version of the radio if that helps. This radio works fine listening to my Acurite rain gauge. But when restart it to listen to the 915 band, I can't seem to get consistent results. I'm testing with the device about 18" from the receiver antenna.

So, some quick questions.

Is the frequency right? I took it from the command above, but simply using 915 doesn't help either
I have the simple antenna this radio came with. But since it works very well with 433, I wonder if it's appropriate for 915? Although, at this range, I'm not sure it would matter, I thought I would check.
The LED blinks correctly every ~16 seconds like it's supposed to, but as you can see below, I'm not getting data very often or consistently.

Am I on a too old version of rtl_433? I didn't want to build from scratch to avoid dependency hell. I can do that though if needed. I just used apt install to get it which is super convenient.

pi@weatherradio:~ $ sudo dpkg -l rtl-433
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  rtl-433        20.11-1      armhf        Decode 433.9 Mhz data

Is there something obvious I'm doing wrong? I'm willing to learn how to help if the device needs more work to be supported, as I notice it's not on the landing README.md as a supported device.

pi@weatherradio:~ $ rtl_433 -F json -C customary -f 915e6
rtl_433 version unknown inputs file rtl_tcp RTL-SDR SoapySDR
Use -h for usage help and see https://triq.org/ for documentation.
Trying conf file at "rtl_433.conf"...
Trying conf file at "/home/pi/.config/rtl_433/rtl_433.conf"...
Trying conf file at "/usr/local/etc/rtl_433/rtl_433.conf"...
Trying conf file at "/etc/rtl_433/rtl_433.conf"...

New defaults active, use "-Y classic -s 250k" for the old defaults!

Registered 145 out of 175 device decoding protocols [ 1-4 8 11-12 15-17 19-21 23 25-26 29-36 38-60 63 67-71 73-100 102-105 108-116 119 121 124-128 130-149 151-161 163-168 170-175 ]
Detached kernel driver
Found Rafael Micro R820T tuner
Exact sample rate is: 1000000.026491 Hz
[R82XX] PLL not locked!
Sample rate set to 1000000 S/s.
Tuner gain set to Auto.
Tuned to 915.000MHz.
Allocating 15 zero-copy buffers
WARNING: Undeclared field "battery_raw" in [113] "Ambient Weather WH31E Thermo-Hygrometer Sensor, EcoWitt WH40 rain gauge"
WARNING: Undeclared field "lux_raw" in [113] "Ambient Weather WH31E Thermo-Hygrometer Sensor, EcoWitt WH40 rain gauge"
WARNING: Undeclared field "wind_avg_raw" in [113] "Ambient Weather WH31E Thermo-Hygrometer Sensor, EcoWitt WH40 rain gauge"
WARNING: Undeclared field "wind_max_raw" in [113] "Ambient Weather WH31E Thermo-Hygrometer Sensor, EcoWitt WH40 rain gauge"
{"time" : "2022-04-26 12:46:59", "model" : "EcoWitt-WS68", "id" : 1404, "battery_raw" : 74, "battery_ok" : 1, "lux_raw" : 10, "wind_avg_raw" : 0, "wind_max_raw" : 0, "wind_dir_deg" : 74, "data" : "00 210", "mic" : "CRC"}
WARNING: Undeclared field "battery_raw" in [113] "Ambient Weather WH31E Thermo-Hygrometer Sensor, EcoWitt WH40 rain gauge"
WARNING: Undeclared field "lux_raw" in [113] "Ambient Weather WH31E Thermo-Hygrometer Sensor, EcoWitt WH40 rain gauge"
WARNING: Undeclared field "wind_avg_raw" in [113] "Ambient Weather WH31E Thermo-Hygrometer Sensor, EcoWitt WH40 rain gauge"
WARNING: Undeclared field "wind_max_raw" in [113] "Ambient Weather WH31E Thermo-Hygrometer Sensor, EcoWitt WH40 rain gauge"
{"time" : "2022-04-26 12:51:56", "model" : "EcoWitt-WS68", "id" : 1404, "battery_raw" : 74, "battery_ok" : 1, "lux_raw" : 13, "wind_avg_raw" : 0, "wind_max_raw" : 0, "wind_dir_deg" : 79, "data" : "00 400", "mic" : "CRC"}

zuckschwerdt commented 2 years ago

Since this is from 2020-01 the 20.11 version (2020-11) should have the decoder.

You need to check that 915M has the signal: https://triq.org/rtl_433/ANALYZE.html#verify-a-transmission

If you have a sample which looks good and should be the WS68 but does produce an ouput then upload here as zip.

I guess nobody figured out the scaling for wind and lux. This needs someone to test and document raw readings compared to expected values.

buelowp commented 2 years ago

I'd be willing to work on the decoders, I can code. I'm familiar with how radio works, but the tools are confusing me. Certainly not an expert. I apologize for any of this that should be obvious, I'll figure it out eventually.

I just tried this to grab a capture, but something seems off when I upload to the triq.org. I suspect I'm either not understanding how it works, or am missing some command line args? Notably, the Y axis does not show banded around 915Mhz, but rather is centered on 0. I don't think I'm using it right, though I'm going to keep digging on this. I also can't seem to get gqrx to do much for me either, though I suspect that's just being not familiar with the app itself. I attached a 60 second capture if it might be useful which should have a success case at ~5 seconds in, and then there should be one that didn't get recorded ~21 seconds as well (the LED blinked, no event showed up). That's assuming I did this right. I'm actually watching for events with a second receiver on my pi so I can tell if the 60 second window would have at least something to help understand.

pete@darp7:~$ rtl_433 -w file_915_2.cu -T 60 -f 915 -S unknown
rtl_433 version unknown inputs file rtl_tcp RTL-SDR SoapySDR
Use -h for usage help and see https://triq.org/ for documentation.
Trying conf file at "rtl_433.conf"...
Trying conf file at "/home/pete/.config/rtl_433/rtl_433.conf"...
Trying conf file at "/usr/local/etc/rtl_433/rtl_433.conf"...
Trying conf file at "/etc/rtl_433/rtl_433.conf"...
Registered 145 out of 175 device decoding protocols [ 1-4 8 11-12 15-17 19-21 23 25-26 29-36 38-60 63 67-71 73-100 102-105 108-116 119 121 124-128 130-149 151-161 163-168 170-175 ]
Detached kernel driver
Found Rafael Micro R820T tuner
Exact sample rate is: 250000.000414 Hz
[R82XX] PLL not locked!
Sample rate set to 250000 S/s.
Tuner gain set to Auto.
[R82XX] PLL not locked!
Tuned to 915.000000.
Allocating 15 zero-copy buffers
^CSignal caught, exiting!
Reattached kernel driver

file_915_2.zip

zuckschwerdt commented 2 years ago

The spectrogram app reads the Y-axis from the filename, don't change the meta data parts. And… -f 915 should be -f 915M, right? ;)

buelowp commented 2 years ago

OK, I can't seem to upload a zip file right now, but am getting useful data now. My mistake, yes, I forgot the M so it clearly was not correct. With the right filename, and running rtl_433 -A, I can see results. You can see it on my google drive too, see below. I see events, with some spaced every ~16 seconds, but not consistently. And of those events, not all can be decoded.

Is it possible the device is just defective?

https://drive.google.com/file/d/13TMPlEfvULbwP4EcHRblh0b3iMzRYPa1/view?usp=sharing

Sorry, this seems to be the only way to share.

zuckschwerdt commented 2 years ago

Drop the file (use an extension of .cu8) on https://triq.org/pdv/ and you see it's empty, noise only. Best to try a GUI like Gqrx, SigDigger, CubicSDR, or SDRangel to locate the signal, then record.

buelowp commented 2 years ago

Thanks, I'm going to try to figure one of those guis out today. However, while the web seems to show it's empty, running it through rtl_433 -A shows lots of stuff, which I thought meant there is something to decode in there. But yeah, the picture just looks like noise. Anyway, I'm waiting on a second usb device now as I'm using the existing one for the 433 Mhz band to track rainfall (it's supposed to rain this week). That new device arrives today, and I'll see which gui I can figure out.

Thank you

Detected FSK package    @55.340027s
Analyzing pulses...
Total count:   57,  width: 14.42 ms             ( 3604 S)
Pulse width distribution:
 [ 0] count:    1,  width: 4108 us [4108;4108]  (1027 S)
 [ 1] count:    9,  width:  112 us [108;132]    (  28 S)
 [ 2] count:   18,  width:   76 us [68;84]      (  19 S)
 [ 3] count:   21,  width:   40 us [40;48]      (  10 S)
 [ 4] count:    4,  width:  216 us [200;240]    (  54 S)
 [ 5] count:    4,  width:  156 us [148;160]    (  39 S)
Gap width distribution:
 [ 0] count:   14,  width:  124 us [116;152]    (  31 S)
 [ 1] count:   18,  width:   44 us [40;48]      (  11 S)
 [ 2] count:   17,  width:   76 us [72;88]      (  19 S)
 [ 3] count:    1,  width:  404 us [404;404]    ( 101 S)
 [ 4] count:    5,  width:  180 us [164;204]    (  45 S)
 [ 5] count:    1,  width:  316 us [316;316]    (  79 S)
Pulse period distribution:
 [ 0] count:    1,  width: 4244 us [4244;4244]  (1061 S)
 [ 1] count:   11,  width:  272 us [236;320]    (  68 S)
 [ 2] count:   16,  width:  120 us [116;124]    (  30 S)
 [ 3] count:   20,  width:  176 us [152;212]    (  44 S)
 [ 4] count:    5,  width:   84 us [80;88]      (  21 S)
 [ 5] count:    2,  width:  376 us [360;396]    (  94 S)
 [ 6] count:    1,  width:  484 us [484;484]    ( 121 S)
Pulse timing distribution:
 [ 0] count:    1,  width: 4108 us [4108;4108]  (1027 S)
 [ 1] count:   22,  width:  116 us [108;136]    (  29 S)
 [ 2] count:   35,  width:   76 us [68;88]      (  19 S)
 [ 3] count:   40,  width:   40 us [40;48]      (  10 S)
 [ 4] count:    6,  width:  212 us [196;240]    (  53 S)
 [ 5] count:    8,  width:  156 us [148;172]    (  39 S)
 [ 6] count:    1,  width:  404 us [404;404]    ( 101 S)
 [ 7] count:    1,  width:  316 us [316;316]    (  79 S)
Level estimates [high, low]:   1418,     28
RSSI: -10.6 dB SNR: 17.0 dB Noise: -27.7 dB
Frequency offsets [F1, F2]:   12548,   1613     (+47.9 kHz, +6.2 kHz)
Guessing modulation: No clue...
view at https://triq.org/pdv/#AAB0240801100C0074004C002800D4009C0194013C8195A393B292B3A3B2B2B2B2C1D1D3A2C2A655+AAB0150801100C0074004C002800D4009C0194013C9392A555+AAB01A0801100C0074004C002800D4009C0194013CD393B2B1C3A2A19555+AAB0140801100C0074004C002800D4009C0194013CB2A455+AAB0160801100C0074004C002800D4009C0194013CC1B3B3D555+AAB0130801100C0074004C002800D4009C0194013CA755+AAB0210801100C0074004C002800D4009C0194013CA1A2B1B1B2B1A3B1B3A3B2A3A392B455+AAB0180801100C0074004C002800D4009C0194013CA1A2B3A391B355

zuckschwerdt commented 2 years ago

Sorry, you are right, there is data. I didn't zoom in enough ;) And it even work well, I guess?

$ rtl_433 -A /Users/zany/Desktop/file_915.00M_1000k.cu8

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time      : @5.507236s
model     : EcoWitt-WS68 id        : 1404
Battery Raw: 83          Battery   : 1             lux       : 2907
Wind Speed: 14  
Wind Gust : 20  
Wind dir  : 209
Extra Data: 14 212       Integrity : CRC
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time      : @22.006456s
model     : EcoWitt-WS68 id        : 1404
Battery Raw: 83          Battery   : 1             lux       : 6867
Wind Speed: 32  
Wind Gust : 56  
Wind dir  : 238
Extra Data: 32 212       Integrity : CRC
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time      : @38.478748s
model     : EcoWitt-WS68 id        : 1404
Battery Raw: 83          Battery   : 1             lux       : 3267
Wind Speed: 33  
Wind Gust : 56  
Wind dir  : 226
Extra Data: 14 212       Integrity : CRC
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time      : @54.979542s
model     : EcoWitt-WS68 id        : 1404
Battery Raw: 83          Battery   : 1             lux       : 9284
Wind Speed: 29  
Wind Gust : 41  
Wind dir  : 61
Extra Data: 3c 212       Integrity : CRC

buelowp commented 2 years ago

That's the problem, when I run it to be read, I only see two of those events returned with JSON content. Is the decoder just broken, or is it something else? If that file has content ever 16 seconds, then it's working to some degree, but not reporting data that can be consumed it seems.

Also, how did you get that output? When I run -A, I see a lot more raw data, and not what you have there. Is there a filter or config I'm missing?

zuckschwerdt commented 2 years ago

I had a look and we didn't change the decoder in some time. But we made general improvements to the receiving and demodulation. I guess the signal is just on the border to profit from those enhancements.

If you got the time then compile a fresh rtl_433 from source: https://triq.org/rtl_433/BUILDING.html#linux-mac-os-x It will be likely a month or three until a new version is ready and arrives in the Debian repos.

buelowp commented 2 years ago

Ok, finally got the new radio and am testing against the latest master, which seems to be working reliably. I guess the dpkg version was just a hair too old. Thank you very much for the help. It's working for me now, and I'm going to see if I can't figure out the UV numbers now that I have them.

buelowp commented 2 years ago

I lied, it's not doing much now. I'm going to try to move the antenna around a bit, and then try to replace it by return/exchange. Honestly, I'm picking up several different 915M devices from the neighborhood consistently. I am able to see a meat thermometer which I get 100% of the time, which I believe is my neighbor. Probably farther away, a small weather device which gives me temp/pressure/humidity, but I only see this every minute or two. This leads me to believe the device is not behaving correctly rather than my radio setup being incorrect. I'm actually seeing about 10 other devices, but they seem to be too far away as I can only see bits and pieces when I run with -vv, though I maybe reading that wrong.

Is there a way to get RSSI or some other signal quality indicator for decoded data? I have to believe now that given the intermittent nature, that it's trying to send, but the signal out is too under powered to go anywhere. So, maybe a broken antenna or amplifier on device?

zuckschwerdt commented 2 years ago

Yes, there is RSSI with -M level. The recommended options would be: -Y autolevel -Y minmax -Y magest -M level -M noise -M time:usec -C si

tolip-ydob commented 2 years ago

I suggest you put the Ecowitt under a metal baking sheet or put it 300 feet away and rerun the experiment.

buelowp commented 2 years ago

OK, I didn't see the level so thank you. Right now, based on the latest observations, this is a combination of a noise and antenna placement problem. My radio knowledge stems from SW for the most part, so while I know what a lot of this means, I'm by no means an expert. Moving the antenna around changes the noise value a lot, I get as high as -18 and have seen as low as -30 with a 2 foot delta in placement (not that simple, but I'm just moving it around my desk right now so it's also wild changes in location). Right now, with inconsistent results, rtl_433 reports

Current noise level -28.3 dB, estimated noise -27.7 dB

I don't know if that's an issue though. I don't know what too high for noise would look like in this case.

The RSSI for my device is really high though. Whether I see a result does have something to do with antenna placement and orientation. In my office, if I move it by a few inches here and there, it changes how often I get results. But they are always the really high RSSI (note about noise above). So, is there a good RP-SMA 900Mhz capable antenna? I'm going to dig around amazon and see what I can find. I tried an small unused Wifi module antenna, but that didn't seem to help. I'm not sure it's suited for 900Mhz though. I'm also going to move the RPi upstairs with a more direct line of site and play around with antenna placement this afternoon up there to see if it gets better. I also have this on USB extensions, high quality USB 3 versions to move the radios away from the RPi. Finally, I get the same results using my Ubuntu laptop AND the RPi's, so I don't think it's just the pi.

Anyway, for my device I get these results but very inconsistently in time. RSSI and Signal to Noise always seems to be that high. I just don't get one every 16 seconds.

time      : 2022-04-29 08:02:23.112517
model     : EcoWitt-WS68 id        : 1404
Battery Raw: 83          Battery   : 1             lux       : 1077          Wind Speed: 6             Wind Gust : 10            Wind dir  : 78
Extra Data: 00 210       Integrity : CRC
Modulation: FSK          Freq1     : 915.0 MHz     Freq2     : 915.0 MHz
RSSI      : -0.2 dB      SNR       : 27.2 dB       Noise     : -27.4 dB

I get 100% from this device. If I had to guess, it's about 200 feet away in my neighbors yard. He has a lot of grill/smoker equipment. By moving the antenna, I can make this device be seen 100% and 0% of the time with very little delta in location.

time      : 2022-04-29 08:09:43.263725
model     : Acurite-01185M                         id        : 0
channel   : 0            Battery   : 1             Integrity : CHECKSUM
Modulation: ASK          Freq      : 915.0 MHz
RSSI      : -22.1 dB     SNR       : 7.7 dB        Noise     : -29.7 dB

Maybe this is as simple as a better antenna?

zuckschwerdt commented 2 years ago

The 7.7 dB SNR is really tight, about 9 to 15 dB would be good. And seeing better than 20+ dB noise is a clean band to me (the RTL-SDR bottoms out around -30 practically and -42 theoretically). The only advice (esp. with a Raspi) that comes to mind: always use an USB extension cord.

Or maybe play with fixed gains. IIRC your WS68 signal looked somewhat clipped.

zuckschwerdt commented 2 years ago

Also if you can grab a signal that won't decode (-S unknown) maybe a peak with Audacity can show what's wrong. Or if you have Sigrok Pulseview installed look at the demod from rtl_433 with rtl_433 -W thefile.sr thefile.cu8

buelowp commented 2 years ago

I have tried, I don't think there is an unknown signal when I don't see results. No combination of runs with -S results in a saved file to investigate. I have been experimenting with antenna placement and isolation and currently have the unit isolated with a 5 USB extension and the antenna placed about 3 feet from the pi. This plus moving the unit upstairs did very little to help. However, when I modify your -Y options upstairs, and run the following command, I am getting consistent results.

rtl_433 -f 915M -Y minmax -Y magest -M level -M noise -M time:usec -C si

Every 16.5 seconds without fail. Running

rtl_433 -f 915M -Y autolevel -Y minmax -Y magest -M level -M noise -M time:usec -C si

I'm going to let this run for a bit and see what happens. Using autolevel seems to be netting me units from very far away with much worse signal numbers, but I never get my units values. Not using autolevel seems to fix this. I'll have to dig into using specific fixed gain values like you suggest above as a way to tune this to work more reliably. Need to figure out how to do that next I suppose. While I still miss events, I'm only missing about 10%, rather than the 100% or 90% in previous attempts.

The numbers look really good anyway. What are the Freq1 and Freq2 values though? They tend to move over time, from 914.9 to 915.1 in increments of .1. That seems weird, but I suspect there is a perfectly logical reason.

time      : 2022-04-29 12:08:30.672563
model     : EcoWitt-WS68 id        : 1404
Battery Raw: 83          Battery   : 1             lux       : 5663          Wind Speed: 10            Wind Gust : 15            Wind dir  : 83
Extra Data: 28 210       Integrity : CRC
Modulation: FSK          Freq1     : 914.9 MHz     Freq2     : 915.0 MHz
RSSI      : -0.3 dB      SNR       : 24.4 dB       Noise     : -24.6 dB

zuckschwerdt commented 2 years ago

This would be the first report that autolevel is worse than with fixed levels. I was actually planning on making it the default. Could it be that the signal is too strong and clipping?

The freq values are rough estimates for the two (FSK) frequencies. The -Y minmax is the default for 868M and above, maybe try the 433M default of -Y classic?

buelowp commented 2 years ago

I wondered about that. The RSSI and SNR are really big numbers honestly for a small device sitting 50 feet away. But again, my familiarity with those are limited to what I remember about cellular back when I worked on base stations for Motorola, so I didn't know if that might be true. There doesn't seem to be a way to reduce signal strength on the device though.

I'll try the classic shortly. Need to finish my real job so I can play with my toys.

Thank you.

zuckschwerdt commented 2 years ago

If you are up to try even more things: replace the stock osmocom rtlsdr with https://github.com/librtlsdr/librtlsdr/ It has some improvments to gain and AGC.

buelowp commented 2 years ago

Interesting, what if any options should I modify in my command line when I do that?

My plan is to keep testing to see what works the best without having two Pi's scattered around the house. I'm hoping I can make this work with a single device and two USB dongles, but I'm having mixed results at the moment. Going to try a location that isn't my office with 3 computers, a WiFi router, and a variety of other electronics.

zuckschwerdt commented 2 years ago

Both version of rtlsdr are compatible. Just make sure the librtlsdr one is found first. Usually /usr/local/lib (where the new one goes) is searched before /usr/lib (where the old one is).

buelowp commented 2 years ago

OK, learned a lot more. The librtlsdr makes this worse, not better. I can verify I'm using it with ldd and LD_LIBRARY_PATH, so I know I'm linked to the new one (master, not development), and when I use it, I get NO results at all for my devices. I do see my neighbors devices though, so it's clearly doing a better job of hearing lower quality signals. Using a master build of rtl_433 linked against the system library versions seems to work.

I thought I had zero'd in on noise being a factor, where I would never see results if the noise level was greater than ~-29, but I got a few results with a noise at -22 just now, so I don't know if that's related. It is far more likely to get results when noise <=-30 though.

time      : 2022-04-29 18:32:57
model     : EcoWitt-WS68 id        : 1404
Battery Raw: 83          Battery   : 1             lux       : 210           Wind Speed: 8             Wind Gust : 20            Wind dir  : 115
Extra Data: 00 a00       Integrity : CRC
Modulation: FSK          Freq1     : 914.9 MHz     Freq2     : 915.0 MHz
RSSI      : -0.3 dB      SNR       : 31.8 dB       Noise     : -32.1 dB

Another interesting tidbit is that I realized I don't see my other 915 Mhz device unless the same conditions are met. They are not colocated, but spread out over the backyard.

time      : 2022-04-29 18:35:57
model     : FineOffset-WH31L                       id        : 59652
Battery   : 1.000        State     : noise         Flags     : 405b          Strike Count: 0           Integrity : CRC
Modulation: FSK          Freq1     : 914.9 MHz     Freq2     : 915.0 MHz
RSSI      : -0.2 dB      SNR       : 31.6 dB       Noise     : -31.8 dB

I'm testing with two different radios, but get the same results with both.

Not sure how much this helps.

zuckschwerdt commented 2 years ago

It might also be a problem with hitting frequencies dead center. Perhaps try -f 915.1M and -f 914.8M

buelowp commented 2 years ago

Interesting, I'll give that a shot today.

Edit: It didn't change much. 915.1 and 914.8 was less reliable. The values 914.9 and 915.0 seem to behave about the same with 914.9 being a tad less reliable. I'm getting about 90% of my results now it seems and I've gone back to just 915M.

Although as I watch now, I missed about 3 in a row.

Also, I don't seem to be able to change the values for windspeed from metric to SI. It's reporting between 20 and 40 for the average right now, which seems high for what I'm looking at. Is it likely these are metric and for some reason -C isn't doing what I expect it to? Or do I just not know how to estimate windspeed? I've tried both si and customary for the -C conversion option.

[Unit]
Description=rtl_433 for 433 mhz senders
Wants=network-online.service
After=syslog.service network-online.service

[Service]
Type=exec
#ExecStart=/bin/bash -c '/usr/bin/rtl_433 -C si -F json -f 915M -Y minmax | mosquitto_pub -h 172.24.1.12 -t rtl-433/915 -l'
ExecStart=/usr/bin/rtl_433 -f 915M -Y minmax -C customary -c /etc/rtl_433/service.conf
Restart=always
RestartSec=30s
SyslogIdentifier=rtl_433
StandardOutput=syslog
StandardError=syslog

[Install]
WantedBy=default.target

buelowp commented 2 years ago

OK, just a last comment, I seem to have this wrangled now. I can get consistent results, though I still don't know exactly why. I'm using latest master from rtl-433 and -Y minmax in a room about 75 feet from the unit. Noise is not high, but not super low, RSSI is good, but not great, and yet this is the best location for results. It's a third radio though, so maybe the one I have been using had some issues? Oh well, it works now, so I'm happy.

Also, for anyone interested in this unit, I would ignore the lux values, as they are not well calibrated. I get a range of about 9k in full sun (early May here, so we're close to highest for the sun elevation) to 0 as it gets dark. It sits about 1500 in clouds. That range is just not too valuable, and if you, like me, want to use lux to figure out when it's really dark (and it shouldn't be, say a bad storm), then this unit isn't going to be terribly useful as the cloudy to dark range is way too small, and with visibility in my backyard, it reports a value of 0.

Also, I haven't been able to work out the actual values for UV sensor. Their range for display doesn't seem to match well with what's sent, and treating the output as high/low order bits or some such doesn't seem to match to other units when I use my VEML sensor from Adafruit. I'm going to ignore it, and put up a more reliable one I can control with a micro.

Thanks for all the help, hopefully something came out of this that makes the project better. I love it, I'm online with 4 unique sensors now at both 433 and 915 and think this is super cool.

zuckschwerdt commented 2 years ago

Thanks for the feedback! Can you collect some raw values with the expected approximate readings? We still don't have a unit factor for wind. For UV it might be that it's a mantissa and scale (i.e. not just plain 16 bit reading).

buelowp commented 2 years ago

I'll try to do that this weekend.

I think the wind speed is KM by default. Multiplying by .62 gets me close to local wind speed readings here. I thought about the UV too, but nothing I do gets to a value that makes a lot of sense. They display the 0-11 scale, and it seems the sensor results are blocky, So somehow, the device might be mapping to 0-11 from some unknown raw value. I dunno, nothing I do makes a lot of sense of the numbers, so I've chosen to ignore it for now. Given the lux value being so overly scaled to bright, I'm not sure how much I trust the UV sensor in this case.

buelowp commented 2 years ago

I take it back, I went and looked at using bit shifts again for the UV data, and went looking at some of the datasheets for easy to get UV sensors, and if you do the following, the numbers actually fit the VEML6070 ranges for a medium integration time.

uint16_t c = (a << 8) | b;

Where a is the first uint16_t and b is the second. It's very bright out right now, but the sun is going down a bit, and the first uint16_t is going down as expected. The values are still relatively high (28 420), but not extreme and is close to web values from weather sites. Using the equation, I get a value of 7588 which is roughly in the range of the second column of the third table of values and is close to 4.something, but that's a bit high for the current value of 3.4 which I'm getting from the web. I would think the VEML6070 is tad expensive for a unit like this, so maybe it's a cheaper i2c solution.

Not sure this is the same sensor. I don't think this is an analog sensor, as it seems to be reporting both a high and low order value over i2c. I'm going capture data for a few days and compare to real world just to see. I'm also going to look into other i2c UV sensors and see if one has ranges similar.

EDIT Nope, I was wrong. I forgot why those numbers didn't make sense. They are hex values, not decimal. The numbers actually fit a bit better for the 4T integration time in the third table though as the result right now of 7712 gets me closer to the decimal for my location (2.0). Maybe that's it?

https://www.vishay.com/docs/84310/designingveml6070.pdf

zuckschwerdt commented 2 years ago

What do you mean with a and b? We already have int lux = (b[4] << 8) | b[5]; https://github.com/merbanan/rtl_433/blob/master/src/devices/ambientweather_wh31e.c#L324 edit: so the lux_raw value is actually okay and just the I2C LSB+MSB raw from the sensor and we'd need a table?

buelowp commented 2 years ago

I'm referring to extra

sprintf(extra, "%02x %02x%01x", b[13], b[16], b[17] >> 4);

Though I am wondering as you have 3 uint8_t values, I assumed it was two. Maybe I didn't look at this the right way. What if the combination of b[16] and b[17] isn't exactly right?

Anyway, for now, the UV data is the same type of data I believe, high and low bytes from the UV sensor, but what comes out is raw hex. I think Lux is fine, if not overly weighted to brightness. If you want to use it know how dark it is, the lux sensor is pretty useless. But if you want to know the brightness difference between partly cloudy and sunny, it gives you a lot of precision.

I really want the UV values, so I do the same calculation and comparing it to the i2c data sheet I have (VEML6070), it does seem to be similar. Not perfect, and the data isn't linear. For that reason, I'm not convinced I'm right. But I can't make it fit any other scale and haven't found an i2c sensor data sheet that might match the output.

Also, I'm pretty convinced now that the windspeed is sent in KPH. It almost no wind right now, but I'm getting between 5 and 10 raw from the sensor.

zuckschwerdt commented 2 years ago

Those "extra" values are the unknown fields just before and after the checksum at the end. I assumed we are talking about the "lux_raw" field (byte 4 and 5). Is there a brightness reading (lux, unknown scaling) and then a UV reading also? I would say that only the first of the "extra" bytes has a proper value as the last two are after the checksum and likely trailing garbage.

buelowp commented 2 years ago

The device has a UV sensor built in. That's what I want to use. I'm going to hack up that decoder a bit. I'd like to know the raw values for b[8], b[9], b[13], b[14]. I wonder if UV is hidden in there. You clearly have CRC right, so what are B[16] and b[17]? They don't seem to be garbage, as they aren't random, but clearly structured. They don't make sense, but they do seem to be specific values.

Also, b[1] isn't in the decode. That seems to be consistent with other devices in that file. Is that expected?

I'm going to dump all the bytes not decoded to extra without modification so I can watch them over time. Hoping to start that effort in a little while and let it run today while I can compare to observed easily enough. I assume it won't be a big challenge to just modify the sprintf to print all those bytes directly without impacting anything else.

zuckschwerdt commented 2 years ago

This should work to dump the raw bytes: decoder_log_bitrow(decoder, 0, __func__, b, 18*8, "raw values");

buelowp commented 2 years ago

Thanks, that's a big help.

Now I just gotta wait for a sunnier day to compare. I think b[13] and b[14] are somehow related to the UV value. I think it has to be both because b[14] gets close to 250 but it's very cloudy/rainy today, so I would expect any combined value to be low, and if both are part of the UV result, then I can see why b[13] is consistently 0 at this time. If true, then a cheap analog sensor maybe the root here, with maybe 12 bit resolution? Or it's a true i2c returning two values as high and low order bytes? Either way, I'm oscillating between 100 and 250 right now, which seems like a lot and I can't compare with anything ATM.

buelowp commented 2 years ago

Ok, b[13] is the whole number part of the UV value, reported as index from 0 to 11. It's sent multiplied by 10. My guess then is, that b[14] or a subset of b[14] more likely is the 1's place, and to get to an index value of 7.2, you take b[13] and 4 bits from b[14]. I just can't figure out which bits. There isn't a bit pattern that doesn't exceed 9 in any 4 bits over a large number of readings. That index also just seems to be random data somehow.

I'm probably still looking at this wrong, lack of familiarity with small byte protocols such as this. Why send the tens place in one byte, and the rest in another when it all fits in one byte? I do think b[13] is the index value, but I have to believe the actual number includes data from elsewhere and I just don't know where.

gdt commented 11 months ago

Can you retest with current git master? A lot has changed. If not working, are you able to work on a PR?

merbanan / rtl_433

Ecowitt WS68 Anemometer #1283