opendata-stuttgart / sensors-software

sourcecode for reading sensor data
573 stars 313 forks source link

Data adjusted with formulas for humidity correction #333

Open OGVT opened 5 years ago

OGVT commented 5 years ago

Hi,

If anyone is interested, I have applied the formulas for humidity correction (posted here: https://github.com/opendata-stuttgart/meta/wiki/Luftfeuchte-Korrektur) according to HackAir, Köhler, Hänel and Soneja to my current values.

The realtime SDS011 PM25 and PM10 raw data of the past 72 hours as well as the humidity and the corrected values are displayed.

Here the humidity of the BHT22 is used (which sometimes shows 100% and is set to max. 99.99 by script). http://www.ogvt.org/sds011_pm24h.php

Here the humidity of the BME280 is used (which is in the max area usually a bit lower than the BHT22). http://www.ogvt.org/pm24h.php

CS, R


EDIT 25.11.2018 - 17:00: The SDS011 were replaced by PMS7003 for testing purposes


LuchtwachtersDelft commented 5 years ago

Nicely done!

DHT22 behavior is interesting. I think two things are happening:

  1. The 0% to 100% scale of the sensor doesn't correspond to real relative humidity.
  2. The sensor becomes saturated after prolonged exposure to moisture. It starts with drift, causing issue 1. and ends with permanent damage. Check https://opensensemap.org/ for example and you'll notice many DHT22s are stuck at 99.9%.
OGVT commented 5 years ago

Thank you!

Yes, the humidity scale of the BHT22 might be incorrect. On a sunny day they show too low values and at night the BHT22 saturate to 100%. The BME280 are slightly better (they don't saturate, but it seems to me that they also display too low values during the day).

I mounted both sensor arrangements directly on the mast of the weather station at ~1.5m (http://www.ogvt.org/fileadmin/templates/images/sensors.jpg). The weather station at 2m (with 24h ventilation by a fan) corresponds quite well with the official readings in the area. The two sensors deviate however massively from these values.

Another problem is caused by the temperature sensors. Here of course the solar radiation plays a role. As soon as the sun shines, the pipes heat up. Therefore the values are partly wrong by more than 15 degrees.

Therefore, I am thinking about installing a radiation shield on the entire sensor arrangement (as with the Davis). This could be ventilated permanently by a fan.

As long as these basic data are not correct, it is of course not possible to make a meaningful correction of the PM measuring data.

In addition, I currently have three different SDS011s in use. Each of them delivers different values for PM25 and especially PM10. In the near future I will do tests with two Plantowers PMS7003 and two Sensirion SPS30 (thanks ricki-z). Maybe I can get better results there.

It's really interesting :-)

sensors

LuchtwachtersDelft commented 5 years ago

Wow, that's quite an impressive setup you have there.

The Davis type of housing is very effective because of its reflective coating, ventilation and double isolation.

Since you have multiple sensors, I'd like to suggest to measure at different heights as well. The difference between 1 m and 4 m from ground level can be significant.

ropeters68 commented 5 years ago

Nice tests! I also have multiple kits (7), the pm2.5 values are pretty similar but I do have differences in pm10. My appoach for the humidity issue is to use a heated airinlet (max T 50 degrees Celsius) lowering th RH to below 70%. You can see 6 kits in a graph on scapeler.com (https://openiod.org/SCAPE604/images/R/apri-sensor/luftdaten_NL-Lent.png) one of the kits is not heated(L1K).

OGVT commented 5 years ago

Hi ropeters68 Thank you. Very Interesting graphs! Especially a nice correlation between the different sensors. Which ones do you use, SDS011? Do you have a description of your heated air inlet? Sounds promisingly.

bertrik commented 5 years ago

Very interesting.

In my opinion, I think it would be appropriate if the sensors just report the raw data they measure, and have corrections being performed at the backend. This way you can update the calculations with new insights / parameters without having to update all of the sensor modules in the field. It's perhaps also easier to detect faulty RH-sensors and simply ignore their contribution this way.

ropeters68 commented 5 years ago

Hi Roger, I use the sds011 sensor, the Dutch institute for health and environment (RIVM) recommends this sensor for citizen science initiatives. In the current 6 kits I use a copper tube of 9 cm (8mm outer diameter), you can coat the inside of the tube with urethan 71 spray (just to prevent chemical reactions in the tube). To heat the tube constantane wire is used (100 ohm per meter) with a resitance of 20 ohms, additionally a wire of 10 ohm is used on the sds011. So in total the resistance is 30 ohms conected to 5V. Heatsink tape is used to stick the wire onto the tube and sensor. Sensor an tube are insulated with 2cm xps hardfoam to minimize heat loss. Near the air outlet a dht22 is placed to measure the outgoing air temperature (7-11 degrees higher than the outside T). In the next model the sensor more temperature controll is added to be sure the sensor is on room temperature and and the tube is only heated if the RH is above 50%. In general it takes about one hour for the tube to reach max T. This setup compensates RH >70% but fog is still an issue (tested with an ultrasonic humidyfier). The kit uses 100% standard luftdaten firmware, ready to use temperature controllers (2 euro a piece) are used for the additional functionality. I'm also testing a solarpanel/battery solution (runs now for 3 weeks). If you need more details please let me know.

martincollignon commented 5 years ago

I would love more details/instructions on this and the solar panel! Sounds great! My ideal setup is an independent station using LoRaWan and solar panels! Maybe this warrants a longer form than a GitHub comment 🙂

søn. 25. nov. 2018 13.02 skrev ropeters68 notifications@github.com:

Hi Roger, I use the sds011 sensor, the Dutch institute for health and environment (RIVM) recommends this sensor for citizen science initiatives. In the current 6 kits I use a copper tube of 9 cm (8mm outer diameter), you can coat the inside of the tube with urethan 71 spray (just to prevent chemical reactions in the tube). To heat the tube constantane wire is used (100 ohm per meter) with a resitance of 20 ohms, additionally a wire of 10 ohm is used on the sds011. So in total the resistance is 30 ohms conected to 5V. Heatsink tape is used to stick the wire onto the tube and sensor. Sensor an tube are insulated with 2cm xps hardfoam to minimize heat loss. Near the air outlet a dht22 is placed to measure the outgoing air temperature (7-11 degrees higher than the outside T). In the next model the sensor more temperature controll is added to be sure the sensor is on room temperature and and the tube is only heated if the RH is above 50%. In general it takes about one hour for the tube to reach max T. This setup compensates RH >70% but fog is still an issue (tested with an ultrasonic humidyfier). The kit uses 100% standard luftdaten firmware, ready to use temperature controllers (2 euro a piece) are used for the additional functionality. I'm also testing a solarpanel/battery solution (runs now for 3 weeks). If you need more details please let me know.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/opendata-stuttgart/sensors-software/issues/333#issuecomment-441435395, or mute the thread https://github.com/notifications/unsubscribe-auth/ACe97m23QiXwzmvJAfnIPMNPECTaqQJ6ks5uyoa7gaJpZM4YwfXB .

OGVT commented 5 years ago

Hi ropeters68, Thank you very much for the details! I'll study it in a free minute. I'm joining martincollignon. To learn more about the solar project would be great.

martincollignon commented 5 years ago

I was very discouraged by this thesis ( https://www.researchgate.net/profile/Yannick_Verbelen/publication/324602867_Characterization_of_Self-Powered_Autonomous_Embedded_Systems_for_Complementary_Balanced_Energy_Harvesting/links/5ad8847ba6fdcc2935854506/Characterization-of-Self-Powered-Autonomous-Embedded-Systems-for-Complementary-Balanced-Energy-Harvesting.pdf ) where a solar setup required 1m2 of solar panels so would love to hear your tricks!

ropeters68 commented 5 years ago

I checked the link, it's quite a theoretical story! I used a more practical approach. My goal have solution that works the entire year, so the problem with solar power is the winter season with little sunlight. My heated kit uses max. 2 watt let's say in the winter a solar panel only produces 10-15% of it's capacity so I would need a panel with 16 watt. But that's only during the day, so if we have 6 hours daylight we need 4 times as much to feed the battery that provides the power in the dark. So it would take a 64 watt panel, there will probably some loss in loading the battery so to be on the safe side I use a 80 watt panel (18V). There are standard charge controllers for these types of panels, just connect the battery and the solar panel (check the standard settings, the minimum voltage was too low) to the controller and plug in the kit in one of the usb ports. Regarding the battery, I use a 12 V lead/acid battery, which work fine with the controller (much safer than Li-ion batteries). So the 2 watt kit draws power from a 12V the battery, that's about 0,17 A this means a battery capacity of 24x0,17 = 4,1 Ah per day in the worst case. I use a 26 Ah battery so 6 days without sun should be possible on a fully charged battery. This might be a bit disapointing but you do need quite a large solar panel (60x80), I bought mine for around 80 euro including the charge controller. The battery should be available at 35-40 euro. I use a garden box from a Swedish furniture company to keep the battery and charge controller dry. My calculations aren't very accurate but it gives a rough indication of sizing. So for it's working fine for 3 weeks now.

ricki-z commented 5 years ago

@OGVT nice work. We have a list of sensors that are mounted directly to official measurement stations (https://github.com/opendata-stuttgart/meta/wiki/Luftfeuchte-Korrektur#liste-von-sensoren-direkt-an-luftmessstationen-als-rechenbasis). Do you think you could do this calculation for these sensors also?

OGVT commented 5 years ago

@ricki-z, thx. Yes, that should be no problem. Let's discuss the details by email.

OGVT commented 5 years ago

@ropeters68, Thank you very much for your interesting remarks!

DeeKey commented 5 years ago

Is there a happy end in this story? :) Can someone share a design of this heating tube? I doubt, that math can be used to solve the fog problem but a heating tube might improve results.

ropeters68 commented 5 years ago

sensor

It's still experimental: the inlet is a 9 cm copper tube (8 mm) glued to the SDS011. 20 cm constantane wire (100 ohm/m) is wrapped around the copper tube (on double sided heatsink tape), the red wire goes to + 5V, the other end is connected to a melting fuse. From the melting fuse another 10 cm constantane wire is taped onto the sensor at the inlet (also with heatsink tape). So in total 30 ohm from red to black wire (connected to ground). The tube runs through the blue XPS hard foam (2 cm). At the SDS011 air outlet there is a DHT22 to check the humidity of the exiting air a BME280 is used to measure outside temperature/RH/pressure. The box is closed with XPS hard foam and taped so there is no heat leaking. There is 1 hole near the DTH22 though which the air can exit. The package is placed in a plastic box. image It takes about an hour before the temperature is stable (outgoing air is around 10 degrees warmer than the outside temperature so RH is alway below 70%). Of course you can make it more fancy with temperature regulation depending on humidity but this is our first version for field testing. There are 8 sensors like this running (6 months no problems). Things you need to take into account: the XPS foam max. temp is 70 degrees Celsius (won't catch fire easy but degrades at higher temperature) , the inlet should not exceed 50 degrees (particles could desintegrade). This version can handle some mild fog but is not fully fog proof. If you want to test heavy fog you can use a ultrasonic humidifyer. Also looking into PTFE/Nafion membrane drying as this requires less energy making it easier to get the kit on solar power.

DeeKey commented 4 years ago

@ropeters68 can you please share results of your experiment with us? What are pluses and what are minuses? What you think about this design: https://nettigo.eu/products/1709

ropeters68 commented 4 years ago

Still work in progress, unfortunately it's more complex than simple heating. Heating in general does improve the correlation with reference sensors (about +5%). The downsides are that you need a prefab solution (like the Nittigo) to keep assembly of the sensor fit for less experienced users and of course the increased power consumption. If you just want to get rid of fog issues, heating will do the trick but the challenge is to optimize the solution to minimize energy and get consistent results under all circumstances.

DeeKey commented 4 years ago

The downsides are that you need a prefab solution (like the Neittigo) to keep assembly of the sensor fit for less experienced users and of course the increased power consumption.

Basically, all the user need is just a PCB which contains SinsirionSPS30 humidity sensor. Plastic stuff is a general stuff one can buy in local shops. The heater can be bought from ALI or replaced with some other heater solution.

I doubt that humidity problem can be solved mathematically as water particles may have different size in different environments under the same humidity level...

tescheva commented 2 years ago

@OGVT : is it possible to somehow access the script of your first comment here? would be great! Thanks