Open unsermanninchina opened 4 years ago
@unsermanninchina thanks for being persistent on getting to the bottom of this issue. Much appreciated.
Do you have any reading from an environmental sensor? I would be interested in temperature and humidity, and ideally also pressure of that measurement location.
I think there's the possibility that there is a software bug, and there is the possibility that the pms sensors are more correct than your instance of the sds011.
The 3rd option, whether those two sensor types simply produce different readings because of their technology I would not consider likely and discard for now. There's however the above linked publication which goes into detail on those aspects.
Now with two sensors outside at the same place:
(PMSx003 PM1 65.0 µg/m³) PMSx003 PM2.5 73.0 µg/m³ PMSx003 PM10 102.3 µg/m³
SDS011 | PM2.5 | 37.6 µg/m³ SDS011 | PM10 | 61.9 µg/m³ Just for reference: BMP/E280 | Temperatur | -0.4 °C BMP/E280 | Luftdruck | 1020.45 hPa BMP/E280 | rel. Luftfeuchte | 49.3 %
A little less difference but still a lot.
Thank's again @dirkmueller ! The SMS011sensor was outside more than a day, one with PMS7003 I put outside at about 03:30 UTC. They both have a distance of 3 cm.
You can find the data here: The one with PMS7003 now: http://www.madavi.de/sensor/graph.php?sensor=esp8266-14230402-pms The one with SMS011: http://www.madavi.de/sensor/graph.php?sensor=esp8266-14230368-sds011 which has a BME280: http://www.madavi.de/sensor/graph.php?sensor=esp8266-14230368-bme280
I can't debug them outside. If needed i'll get them inside again, no problem!
side note: Humidity and Temperature has big impact on measurment results.
For my feeling: both sensors/families are "not accurate" (to say the very least) below 5°C and/or above 70% relative humidity. (with BME280 at least the reported humidity values are not totally random if actually above 70% as with DHT)
there has always been the plan to normalize the PM values with the help of the environmental values. (disclaimer: i have not followed on what stage this project is during the last 2 years)
additionally there is a common mistake by a lot of sensor builders/users: they mistake the temp/hum sensor as some kind of weather station "for their own comfort", so they optimize the placement of those sensors to be independend from the e.g. heated airflow to the PMS (due to radioton/sunlight, electrical power consumption of the cuircuit). And since there is no flag on the stored raw data if temp/hum is "from the PMS airflow" or "from the outside environment" (or at least optimized to reflect more the one or the other): the records in the database can not be used to make a large scale correction.
And even IF you have the certainty for a specific sensors, that temp/hum values are from the PMS intake airflow and IF we are (in the future) able to come up with a correction map (3D/4D...): the results will be noisy and possibly even have artefacts of it's own.
different approach: let's heat the airflow! If we know that our sensors have difficuties in low temperatures and/or when approaching dew point: then let's just avoid that and heat the air. (and still record the values from that airstream)
@Adorfer Thank you so much for your explanations. I am aware of that problem. My question is more about technical or software issues, even if those are all solved than there will be the inaccuracy you are talking about, I am aware of that.
It's not at a scientific level what we get, but swarm intelligence or citizen science is still something valuable. Actually my BME280 is just where you say it should be, I would flag that as "from the PMS airflow".
Finally I am not sure if what you mention is explaining the difference I see here. All sensors use the same laser technics to count the particles and should have more or less the same wrong results, if that's coming from the problem - especially humidity and temperature - you explained. I will try to get a bit closer to the problem. And living in Beijing, China you have a good reason to try to get more information about PM, especially in the close countryside, where is no data available.
And actually I was wondering, if normalizing is already done, if there is data for humidity, temperature and air pressure, it's not, is it?
Thanks again, @Adorfer ! I Hope you have better air then we here!
@unsermanninchina no normalization is being performed on the device.
There have been a few studies done and while it is clear that there's an influence, there is no polynomial function that would accurately compensate in all cases.
@unsermanninchina Hello! i have a lot of sds011 next to my house (at least 3 in a pair of kilometers around..) and all of these are going to have low measurements than mine (i also have a pms7003). You said that your sds is three years old; this kind of sensors are getting old fast also, it's my opinion, if they're not working (because of dust is some cases). Try to disconnect everythig from the sensor, pull in gently some air (with a compressor from at least 20cm and if you use air compressed can at least 10), not only inside the main vented hole but also from the top hole where you can connect a little tube. Repeat at least three times then stop for 5 minutes and then reconnect the sensor. Let us know please. s.
the approach "heat the airflow to avoid having whater droplets counted" is integrated into this kit, which is a fork of the luftdaten-project: https://nettigo.eu/products/nettigo-air-monitor-kit-0-3-2-build-your-own-smog-sensor (there is a parts lists in english... or for detail photos look in the polish description -which i can not read either, but the pictures are helpful: https://air.nettigo.pl/baza-wiedzy/nam-heca-montaz/ )
@stewecar Thank you for your comment. I have four sensors, two new SDS011, one new PMS7003 and one three years old but newer used PMSA003.
The old PMSA003 and the new PMS7003 have not the exact same level of PM, but quite similar. Both higher than the SMS001, which measure about 30% of the PM levels the PMSx are measuring. Since yesterday the madavi links above link to one of the new SMS011 and the new PMS7003. Both sensors are new, but they have this big difference.
@Adorfer Thanks again, I will read the links soon.
@dirkmueller Thank you too confirming that there is no compensation. I understand that it's better to have the raw data and any correction should be done later. For me personally it's ok for the moment if I have reliable data if the weather condition is dry because it's not very often humid in Beijing.
The SMS011 didn't measure some hours, the cables had been not well connected. I hope it works until tomorrow to have at least 24h data to compare. I could try to connect a Raspi 2B to both ESP8266 boards to get debug data, would that help?
@unsermanninchina there's debug output in the airRohr firmware, just set debug to 'max info'.
There's a similar comparison test setup running in Beijing (so maybe it is close enough for the values to be comparable for you): https://aqicn.org/sensor/
It has extensive write-ups about the sds011 and pms sensore with comparison and real time data.
As you can see at the end with the 5 days overview, there are periods of time where the measurements agree perfectly, and there are times where's large factors of difference between the individual sensors.
Please compare this to your setup
@Adorfer Looks very nice the system with the heating. But I want to focus first on the question @dirkmueller mentioned:
I think there's the possibility that there is a software bug, and there is the possibility that the pms sensors are more correct than your instance of the sds011.
I think, if there is a software related problem one of these sensors (or both) with airflow heating would still produce wrong data.
@unsermanninchina have you tried for sure tu put the sds upside down..in that sense that the manual indicates as correct.. In the pms series i didn't find any related "position" manual's indications..
@stewecar I installed the sensor like said in the "Nova Fitness Co., Ltd. SDS011 Laser PM2.5 Sensor specification, Product model: SDS011, Version: V1.3" You find the file here attached, copyright of the attached picture should be by Nova fitness. My sensor is installed like the upper right.
@dirkmueller Thank you! I know aqicn.org since when I bought the PMSA003. Later I found luftdaten.org and there were mentioned the SDS011 as reference for a good sensor. If you have as reference aqicn.org we see that the SDS011 has the same problem the as well:
Current BAM readings are: PM2.5 mass in mg/m3 37.1, updated on 2020-01-19 19:00:00 (China time).
Second SDS011i (new) readings are: PM2.5 mass in mg/m3 13.9433 updated on 2020/01/19 19:05:04 (China time).
The first SDS011i (old) sensors last data is updated on 2018/12/08 15:04:08 (China time), more than one year old. But if sensor (new) is a reference I see, that my SDS011 PM levels of about 35% of the PMS levels are just as much too low as sensor (new) at aqicn.org. I didn't find information about where the sensors are installed and how far they are comparable with the BAM sensor at the China Agricultural Exhibition Center in Beijing. Maybe there are other reasons to stick with the SDS011, please let me know if so. Luftdaten.info is an impressive project, I believe you have your reasons. But otherwise using Plantower sensors instead might be a better solution.
You find a comparison of PMS sensors PMS5003 and PMS7003 with a BAM reference for 2.5 and 10 μm at aqicn.org, too. The PMS7003 has the smallest difference from the BAM reference. Only looking at this data I would suggest to buy Plantower PMS7003 (no data available for A007, but it's more or less like the PMS7003). At least at taobao.com, a popular selling platform in China, I bought the PMS7003 for 51 RMB and paid 95 RMB for each SDS011, both incl. shipment from Shenzhen to Beijing. Plantower PMS7003 is about 65% the price of the SDS011 and seems to be more accurate for me an amateur.
But actually I am wondering, if there is a software or data problem with the SDS011, because at least with my sensors and with the one at aqicn.org it seems to be the same amount of too low lewels.
many thanks..i remebered only the last one position was in the manual.. so, we have finished the troubleshooting.. let's see what dirkmueller or the other extperts will find by software's side..it's an interesting point..
Please see https://github.com/opendata-stuttgart/sensors-software/issues/636#issuecomment-575640339
The statistics in there basically confirm that the pms is both a cheaper and a more accurate choice than the sds011
The SDS011 has an air inlet where we can connect a tube to. So we can place the sensor more inwards the case, protecting the electronic (i.e. against water). The PMS sensors may need to be placed nearer to an opening. As we started the project the PMS sensors were more than twice the cost of an SDS011.
Hi @dirkmueller and @ricki-z Thank you for your help understanding the issue! I saw the problem too, how to install the PMS sensors, because there is no air inlet to install a tube. I will think about a way to install it in some housing.
I saw the problem too, how to install the PMS sensors, because there is no air inlet to install a tube. > I will think about a way to install it in some housing.
perhaps come up with something similar as https://www.thingiverse.com/thing:2775946 just with PMS5003 and BME280 behind the outlet of the PMS.
@Adorfer That looks nice!
The graphs at madavi.de are actually looking very similar. The only difference is the scaling of PM: SDS011 - PM2.5 µg/m³ PMSA003 PM2.5 µg/m³ /72808988-b1dbff00-3c95-11ea-8ac4-576fe56b546b.png) SDS011 - PM10 µg/m³ PMSA003 PM10 µg/m³
measuring now 13pm2.5 and 14.2pm10 with an old sds011.. in the same time i have 25 and 28 with a pms 7003.. but a really near sds011 is now measuring 15 and 20..and the sensor has at least one year and half of activity..
I wonder if the scale between the sensors is not comparable because they use different units of measurement?! Will read datasheets.
@dirkmueller If I can help with anything please let me know.
I started uploading the data to opensensemap, too. Here the links: PMS7003 SDS011 & BME280
The difference stays the same for some weeks now, SMS011 and PMSA003 show the same ups and downs of the PM levels. Doesn't look like a measuring problem but like a scaling/software problem. @dirkmueller Did you find time to check the units for measurement? Would be strange, but I don't have another idea how to explain this variance of the values.
I briefly did check the datasheets, and I could not find a difference in unit or scale documented.
I am as puzzled as you are on this.
My Plantowers values are much closer to the closest official data and the weather conditions at this moment are very stable. So even if I take into account, that there are local differences I would tend to see the values given from the Plantower A003 as more realistic than the SMS011.
@dirkmueller I was about to write an e-mail to Plantower and Nova, but than I thought I should talk to you first. I think it would be helpful to mention luftdaten.org to get a response, just in the way that both sensors are usable and that they show this big difference. I wouldn't express that I am part of the project, but just a user. If that's ok for you. For me it would be ok as well, if you write questions in English and I translate it into Chinese. I am not perfect and not an interpreter, but it should work.
I found out I didn't get a PMS7003. What I got is called G7 and might be made by a company called Yuntong (云彤). The shop at Taobao named it PMS7003 or G7. It's similar, you don't see any difference and there are M-Versions, too.
There was another shop selling them with a table describing G7 as made by Plantower and G7 being a short name of PMS7003. I wanted to buy two more sensors but than I saw that some shops sold PMS7003 and G7 for a different price. So I asked if there is any difference what PMS7003 and G7 means. It took me some questions to get the answer, that it is not the same sensor. But they said, it is the same sensor, just a different factory. I asked Plantower and the said, they have only one factory.
When I started this topic I have had run both, a Plantower PMSA003, bought at the official store of Plantower and this Yuntong G7, and they had quite the same values. But not knowing it is different manufacturers I didn't check that over a long time. I changed my installation today around UTC 22:00 (just short after the lowering of the values at madawi) and use the Plantower PMSA003 now. I need to buy some things and will install all three sensors together later. I am very curious to check again if there is a difference between PMS and G7, but at the moment we have 80% humidity.
@unsermanninchina according to your previous comments the PMS sensor is good, the sds011 is off..
So replacing that one might be more relevant alternatively if you're interested a sensirion sps30 can be added as well as that one might become the default sensor in the future.
Thank you, @dirkmueller Looks like a very good sensor. I checked prices for the SPS30 and it is almost three times the price (at least 230 RMB) compared with a PMS7003 in China. If you think it is worth it, I would give it a try. Otherwise I would stick with Plantower. I talked to them today. They had in mind only one client in Germany, which is not a reseller. The price for one directly bought at Plantower is 85 RMB (~ 11.5 €) including a prime delivery service in China. For me it seems to be important, that the component is cheap, because more people might want it then. But, of course, a sensor needs to be reliable.
Maybe I am wrong, but the SDS011 looks actually pretty good, if I take the PMS/G7 as a reference at least. Only that the SPS uses a wrong scaling. However, in China Plantower sensors are cheaper than SPS011 (~110 RMB), not much worth thinking about it. But many SDS011 are installed in Germany, right? The slope of the SDS001 I have corrected to fit the values. The originals you can find at madavi. Or do you think, they are not usable?
I wonder if the scale between the sensors […]
My SDS011 units (firmware 18-11-16) have scale factors which can be stored on the unit's (flash?) memory, so they survive a power-cycle. You can read them out with B4 0B and set them with B4 0C.
@t0b1 interesting information. I can't find that in the datasheet, but I can confirm that I can read 0-0-100 using command 0x0b. What is the meaning of the datafields returned?
I can't find that in the datasheet, […]
I have no official documentation about commands above 0x08. I tried them out by “accident”. ;-) If I remember correctly, the observed behavior was this: 0x09 reads the current (volatile) scale factor, 0x0A sets it. That factor will not survive a power cycle. 0x0B reads the current (non-volatile) scale factor, 0x0C sets it. This factor will survive a power cycle. When the scale factor is set with 0x0C it becomes effective immediately, so after 0x0C both 0x09 and 0x0B report the same factor. Minimum factor is 10 (I guess that means 1.0). The default for new units here seems to be 100 (meaning 10.0).
I can read 0-0-100 using command 0x0b […]
The first zero indicates a ‘get’. In response to a ‘set’ command it is non-zero (one). This behavior is similar to other commands, such as for setting the measurement interval or the data reporting mode. The other two bytes are the factor. I believe it was low and high byte, similar to how PM values are reported, but I can't remember exactly.
The sensor specs say you need to divide the reported PM values by 10.0 to get [µg/m³] and they ship the units with the scale factor set to 10.0 – at least here in the metric world. My first guess about NOVA's intention: unit conversion. Second thought: Ouch! This is a potential source for wired readings …
The first zero indicates a ‘get’.
Maybe not. Have to check again when I'm back at my PC, but I just remembered that this was a bit different … Sorry for the confusion.
@dirkmueller correction to what I wrote earlier: The first (zero) byte means “subtract”, non-zero (one) means “add”. The second byte is the summand which gets added or subtracted from the sensor reading. The third byte is the factor by which the sum gets multiplied.
@unsermanninchina could you please flash https://static.dmllr.de/airrohr/beta/builds-NRZ-2020-130-B5-sds011-factor/ and add a screenshot of the /status page?
Today I was told that some of NOVA's newer sensors have a “calibration command” similar to this. They call the summand the “zero point value” and the factor “percentage”. The “calibration” vs. “unit conversion” difference might explain why it is not in the normal specs: They apparently don't want a typical application to be aware of it …
Sorry @dirkmueller, I didn't see your message. Yes, I will do.
I tried to flash, but it didn't work. To be honest, I don't have much understanding of python and esp8266. I don't understand how to flash with esptool.py (did install esptool with pip install esptool
and got the esp8266 software installed in Arduino-SDL) and luftdaten-tool can only download, right? Could someone point me in the right direction?
My problem: I tried
sudo esptool.py -vv -c auto -cb 57600 -ca 0x00000 -cp /dev/ttyUSB0 -cf /home/mmanjaro/Downloads/latest_de.bin
from inside /usr/bin/ where esptool.py is located. I get a
usage: esptool [-h] [--chip {auto,esp8266,esp32}] [--port PORT] [--baud BAUD] [--before {default_reset,no_reset,no_reset_no_sync}] [--after {hard_reset,soft_reset,no_reset}] [--no-stub] [--trace] [--override-vddsdio [{1.8V,1.9V,OFF}]] {load_ram,dump_mem,read_mem,write_mem,write_flash,run,image_info,make_image,elf2image,read_mac,chip_id,flash_id,read_flash_status,write_flash_status,read_flash,verify_flash,erase_flash,erase_region,version} ... esptool: error: argument --chip/-c: invalid choice: 'b' (choose from 'auto', 'esp8266', 'esp32')
I tried some variations but didn't success. I am not sure, if esptool.py and esptool works the same way.
@unsermanninchina with a recent esptool.py
script try this:
sudo esptool.py --port /dev/ttyUSB0 write_flash 0x0 /home/mmanjaro/Downloads/latest_de.bin
It works for me even without the --port /dev/ttyUSB0
thing, and it found the UART tty with the ESP just by itself just fine.
If the above doesn't work, try the “airrohr flashing tool”. Forget the old esptool
binary.
https://firmware.sensor.community/airrohr/flashing-tool/ https://luftdaten.info/feinstaubsensor-bauen/#firmware-einspielen
@t0b1 Tobias, thank you so much! It worked with esptool.py. I tried to use the flashing tool yesterday, but didn't find a way to flash the binary provided by dirkmueller. Is that possible? However, I prefer the esptool.py.
The ESP8266 is running the new firmware now. @dirkmueller I attached a screenshot of the status page. And thank you for all your efforts! After doing the screenshot I put the sensor back to the old location, it's again just directly beneath the PMSA003. It's values you can see at madavi.de here and the PMSA003values here.
@dirkmueller The beta firmware doesn't look like making a difference with PM2.5, but PM10 seems to have changed. At the moment:
PMSx003 | PM2.5 | 10.6 µg/m³ PMSx003 | PM10 | 10.6 µg/m³
SDS011 | PM2.5 | 4.4 µg/m³ SDS011 | PM10 | 9.3 µg/m³
BMP/E280 | Temperatur | 5.6 °C BMP/E280 | Luftdruck | 1023.09 hPa BMP/E280 | rel. Luftfeuchte | 28.3 %
What do you think?
to be honest it wasn't supposed to change anything yet. I merely changed the "SDS version" code to poll the scaling factor parameters that were pointed out in https://github.com/opendata-stuttgart/sensors-software/issues/636#issuecomment-586573899 .
your SDS011 has the same 0-0-100 (aka factory default scaling factor) than mine has. Too bad, this would have been a good explanation why values are off.
Oh. Too bad it is 0-0-100. I also hoped to see something else there. :) Now I don't know how to proceed …
I have a few more ideas: There is this strange FC scaling, which I don't quite understand. And there might be a scaling per “measurement unit” (PM 2.5 vs. PM 10). I have reached out to Nova by email last week, but I did not get a response yet. @unsermanninchina how's your Chinese? Are you motivated enough to help get in touch with them?
@dirkmueller with B4 01 you can switch from C0 (10 bytes) to A5 (19 bytes) event/measurement reporting. That this gives you access to two additional sensor readings which are apparently not affected by this calibration/scaling thing, plus the true serial number of the device (the one on the bar code sticker). Example:
AA A5 03 FE 00 ( 25.4) 90 00 ( 14.4) 35 00 ( 5.3) 1234 5F 00 ( 9.5) 06 5678 00 61 AB sn=1234-5678, pm2.5=5.3, pm10=9.5 [µg/m³]
@dirkmueller with B4 FC you can get (0) or set (1) another scaling factor. I don't know exactly how it works. It is stored and survives a power cycle. But when you set it, readings before and after a power cycle are different – maybe it is even ignored after a power cycle. Note that the response to B4 FC is unusual and has 19 bytes – like an echo of the request. Example:
AA B4 FC 00 1E 00 00 00 00 00 00 00 00 00 01 FF FF 19 AB
I have another SDS011 unit coming from China, but it is hard for me to actively test and help you, other than with my notes from way back then, until it arrives … Please also let me know if you believe that this is getting way off topic here …
@t0b1 My Chinese is good, but technical vocabulary is poor. But I am motivated to help and could try to get in contact. I have the Chinese manual too, I could lookup missing vocabulary. For me all this fits my topic. I would be happy if there is a solution for the Nova sensor. You can send me an email through github, right? Could you send me the message you have sent to them? (I am German, you can use German writing to me.)
@dirkmueller Actually I first wanted to write that it didn't change, but then I saw the PM10 data. Now that you say you haven't changed anything I got in mind that I had the thought, that the sensors deviation might change with low or high levels of PM. But I never downloaded the data and compared it, I only looked on the graphs at madavi. But they change the scaling, not easy to visually recognize the deviation.
Sorry, I pushed the wrong button and accidentally closed the topic.
@unsermanninchina Sorry I failed the “send me an email through github” test. But you can reach me at my given name dot my family name at gmail dot com, and I believe I might pass the reply test. :)
NovaFitness sent me the Laser Dust Sensor Control Protocol V1.5 for the Nova SDS011. Maybe that is of use for other people, too. I asked if I can upload it here:
liushanwen Hardware Center Nova Fitness Co.,Ltd`
Later I understood that Nova didn't give me a complete list of commands. I asked again and Mr. Liu's answer is:
Hi, The sensor is in error state if it is sending this data all the time.All the commands is open in the last email.Please do not send the command out of the open command list.
liushanwen R&D Engineer Nova Fitness Co.,Ltd 9F,Tower B,No2 Wanshou Road,Jinan,Shandong Province,China Tel:+86-531-82868935 Fax:+86-531-82868198 E-Mail:liushanwen@sd-nf.com http://www.sd-nf.com/
Means: They don't want to give more. I even gave them a more specific problem description @t0b1 gave me, but like you see, they don't give an answer.
I had mentioned this problem in another issue. My problem is that SDS sensors and PMSA003 have a huge difference in their data. I was not sure if the A003 is different from the supported Plantower sensors and it is a three year old but never used sensor. For that reason I bought a new PMS7003 to try if it is different from the old one. But it shows almost the same PM data like the A003.
So now I have the problem that I have two SDS011 with lower PM and two Plantower A003/7003 with much higher PM levels. Looks like I have to choose to trust the Nova or the Plantower sensors, which at the moment is a difference like this example, picked up at the same moment:
SDS011 | PM2.5 | 12.3 µg/m³ SDS011 | PM10 | 22.7 µg/m³ and the 7003 with (PMSx003 | PM1 | 37.6 µg/m³) PMSx003 | PM2.5 | 42.0 µg/m³ PMSx003 | PM10 | 50.0 µg/m³
Both sensors placed on the same table and their air input with 6 cm distance. And for a lot more measurings show the same difference of about 3-4 times higher PM levels measured with the PMS sensors. The EPS8266 have all the same firmware. I gonna run them with debugging the next days, but @dirkmueller had looked at the A003 log a week ago and not found any hints That there is a problem with the firmware.
Until now I can only ask if other users have had similar problems or any idea what I could check. I must say, that I don't understand the programming of Arduino or ESP8266 until now, I have never tried it. I am not afraid to check anything if I know how to do it. Thank you to anybody in advance!