pi3g / bme68x-python-library

Python 3 Library for BME688 and BME680 (Bosch Sensortec sensors), supports Bosch BSEC
MIT License
45 stars 17 forks source link

Compilation fails with BSEC_2.2.0.0_Generic_Release_30052022 #17

Open falste opened 1 year ago

falste commented 1 year ago

I followed the "How to install the extension with BSEC" readme steps. I am using BSEC_2.2.0.0_Generic_Release_30052022. Since the BSEC library name changes depending on version (see also https://github.com/pi3g/bme68x-python-library/issues/14), I just named the folder libbsec and changed the following files:

Click to expand ``` $ git diff diff --git a/bme68xmodule.c b/bme68xmodule.c index d43671b..999286e 100644 --- a/bme68xmodule.c +++ b/bme68xmodule.c @@ -18,7 +18,7 @@ uint8_t bsec_state[BSEC_MAX_STATE_BLOB_SIZE]; uint8_t work_buffer[BSEC_MAX_WORKBUFFER_SIZE]; uint32_t bsec_state_len = 0; bsec_library_return_t bsec_status = BSEC_OK; -const char *bsec_conf_path = "BSEC_2.0.6.1_Generic_Release_04302021/config/bsec_sel_iaq_33v_4d/2021_04_29_02_51_bsec_h2s_nonh2s_2_0_6_1 .config"; +const char *bsec_conf_path = "libbsec/config/bsec_sel_iaq_33v_4d/2021_04_29_02_51_bsec_h2s_nonh2s_2_0_6_1 .config"; FILE *bsec_conf; #endif diff --git a/internal_functions.h b/internal_functions.h index 9bc1c6e..934f109 100644 --- a/internal_functions.h +++ b/internal_functions.h @@ -19,8 +19,8 @@ #include "BME68x-Sensor-API/bme68x_defs.h" #ifdef BSEC -#include "BSEC_2.0.6.1_Generic_Release_04302021/algo/normal_version/bin/RaspberryPi/PiThree_ArmV6/bsec_interface.h" -#include "BSEC_2.0.6.1_Generic_Release_04302021/algo/normal_version/bin/RaspberryPi/PiThree_ArmV6/bsec_datatypes.h" +#include "libbsec/algo/normal_version/bin/RaspberryPi/PiThree_ArmV6/bsec_interface.h" +#include "libbsec/algo/normal_version/bin/RaspberryPi/PiThree_ArmV6/bsec_datatypes.h" #endif /* CPP guard */ diff --git a/setup.py b/setup.py index e41a4e3..27c66d3 100644 --- a/setup.py +++ b/setup.py @@ -7,7 +7,7 @@ if BSEC: ext_comp_args = ['-D BSEC'] libs = ['pthread', 'm', 'rt', 'algobsec'] lib_dirs = ['/usr/local/lib', - 'BSEC_2.0.6.1_Generic_Release_04302021/algo/normal_version/bin/RaspberryPi/PiThree_ArmV6'] + 'libbsec/algo/normal_version/bin/RaspberryPi/PiThree_ArmV6'] else: ext_comp_args = [] libs = ['pthread', 'm', 'rt'] @@ -49,7 +49,7 @@ setup(name='bme68x', py_modules=['bme68xConstants', 'bsecConstants'], package_data={ 'bme68x': [ - 'BSEC_2.0.6.1_Generic_Release_04302021/config/bsec_sel_iaq_33v_4d/2021_04_29_02_51_bsec_h2s_nonh2s_2_0_6_1 .config', + 'libbsec/config/bsec_sel_iaq_33v_4d/2021_04_29_02_51_bsec_h2s_nonh2s_2_0_6_1 .config', ] }, headers=['BME68x-Sensor-API/bme68x.h', ```

Sadly, the installation still fails, mainly because BSEC_SAMPLE_RATE_HIGH_PERFORMANCE and BSEC_OUTPUT_COMPENSATED_GAS are undeclared:

Click to expand ``` bme68x-python-library $ sudo python3 setup.py install running install running bdist_egg running egg_info writing bme68x.egg-info/PKG-INFO writing dependency_links to bme68x.egg-info/dependency_links.txt writing top-level names to bme68x.egg-info/top_level.txt reading manifest file 'bme68x.egg-info/SOURCES.txt' writing manifest file 'bme68x.egg-info/SOURCES.txt' installing library code to build/bdist.linux-armv7l/egg running install_lib running build_py running build_ext building 'bme68x' extension arm-linux-gnueabihf-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/include -I/usr/include/python3.9 -c BME68x-Sensor-API/bme68x.c -o build/temp.linux-armv7l-3.9/BME68x-Sensor-API/bme68x.o -D BSEC arm-linux-gnueabihf-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/local/include -I/usr/include/python3.9 -c bme68xmodule.c -o build/temp.linux-armv7l-3.9/bme68xmodule.o -D BSEC bme68xmodule.c:11: warning: "BSEC" redefined 11 | #define BSEC | : note: this is the location of the previous definition bme68xmodule.c: In function ‘bme_get_sensor_id’: bme68xmodule.c:338:50: warning: passing argument 2 of ‘bme68x_get_regs’ from incompatible pointer type [-Wincompatible-pointer-types] 338 | rslt = bme68x_get_regs(BME68X_REG_UNIQUE_ID, &id_regs, len, &(self->bme)); | ^~~~~~~~ | | | uint8_t (*)[4] {aka unsigned char (*)[4]} In file included from internal_functions.h:18, from bme68xmodule.c:4: BME68x-Sensor-API/bme68x.h:122:51: note: expected ‘uint8_t *’ {aka ‘unsigned char *’} but argument is of type ‘uint8_t (*)[4]’ {aka ‘unsigned char (*)[4]’} 122 | int8_t bme68x_get_regs(uint8_t reg_addr, uint8_t *reg_data, uint32_t len, struct bme68x_dev *dev); | ~~~~~~~~~^~~~~~~~ bme68xmodule.c: In function ‘bme_subscribe_gas_estimates’: bme68xmodule.c:388:52: error: ‘BSEC_SAMPLE_RATE_HIGH_PERFORMANCE’ undeclared (first use in this function) 388 | requested_virtual_sensors[i].sample_rate = BSEC_SAMPLE_RATE_HIGH_PERFORMANCE; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ bme68xmodule.c:388:52: note: each undeclared identifier is reported only once for each function it appears in bme68xmodule.c: In function ‘bme_subscribe_ai_classes’: bme68xmodule.c:408:48: error: ‘BSEC_SAMPLE_RATE_HIGH_PERFORMANCE’ undeclared (first use in this function) 408 | requested_virtual_sensors[0].sample_rate = BSEC_SAMPLE_RATE_HIGH_PERFORMANCE; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ bme68xmodule.c: In function ‘bme_bsec_process_data’: bme68xmodule.c:820:15: warning: unused variable ‘data’ [-Wunused-variable] 820 | PyObject *data = PyDict_New(); | ^~~~ bme68xmodule.c:818:13: warning: unused variable ‘num_bsec_outputs’ [-Wunused-variable] 818 | uint8_t num_bsec_outputs = 0; | ^~~~~~~~~~~~~~~~ bme68xmodule.c:817:19: warning: unused variable ‘bsec_outputs’ [-Wunused-variable] 817 | bsec_output_t bsec_outputs[BSEC_NUMBER_OUTPUTS]; | ^~~~~~~~~~~~ bme68xmodule.c:740:18: warning: variable ‘inputs’ set but not used [-Wunused-but-set-variable] 740 | bsec_input_t inputs[BSEC_MAX_PHYSICAL_SENSOR]; | ^~~~~~ bme68xmodule.c:821:1: warning: no return statement in function returning non-void [-Wreturn-type] 821 | } | ^ bme68xmodule.c: In function ‘bme_get_digital_nose_data’: bme68xmodule.c:829:20: warning: comparison of integer expressions of different signedness: ‘int64_t’ {aka ‘long long int’} and ‘uint64_t’ {aka ‘long long unsigned int’} [-Wsign-compare] 829 | if (time_stamp >= self->next_call) | ^~ bme68xmodule.c:1069:42: error: ‘BSEC_OUTPUT_COMPENSATED_GAS’ undeclared (first use in this function) 1069 | case BSEC_OUTPUT_COMPENSATED_GAS: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ bme68xmodule.c: In function ‘bme_get_bsec_data’: bme68xmodule.c:1120:20: warning: comparison of integer expressions of different signedness: ‘int64_t’ {aka ‘long long int’} and ‘uint64_t’ {aka ‘long long unsigned int’} [-Wsign-compare] 1120 | if (time_stamp >= self->next_call) | ^~ bme68xmodule.c:1331:34: error: ‘BSEC_OUTPUT_COMPENSATED_GAS’ undeclared (first use in this function) 1331 | case BSEC_OUTPUT_COMPENSATED_GAS: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ bme68xmodule.c: In function ‘bme_enable_gas_estimates’: bme68xmodule.c:1587:48: error: ‘BSEC_SAMPLE_RATE_HIGH_PERFORMANCE’ undeclared (first use in this function) 1587 | requested_virtual_sensors[0].sample_rate = BSEC_SAMPLE_RATE_HIGH_PERFORMANCE; | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ At top level: bme68xmodule.c:737:18: warning: ‘bme_bsec_process_data’ defined but not used [-Wunused-function] 737 | static PyObject *bme_bsec_process_data(BMEObject *self, bsec_bme_settings_t *sensor_settings, uint8_t i, int64_t time_stamp) | ^~~~~~~~~~~~~~~~~~~~~ error: command '/usr/bin/arm-linux-gnueabihf-gcc' failed with exit code 1 ```

I guess it is ignoring the bsecConstants.py, because BSEC_SAMPLE_RATE_HIGH_PERFORMANCE is defined there, but I am not sure what the deal with BSEC_OUTPUT_COMPENSATED_GAS is.

My system:

$ uname -a
Linux raspberrypi 5.15.70-v7l+ #1590 SMP Tue Sep 27 15:58:00 BST 2022 armv7l GNU/Linux
$ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 11 (bullseye)"
NAME="Raspbian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
Blandi88 commented 1 year ago

Just received my bme688 breakout board and I have this same exact problem with the newest version of bsec. Spent hours trying to fix it.

BSEC_SAMPLE_RATE_HIGH_PERFORMANCE BSEC_OUTPUT_COMPENSATED_GAS

^these have been removed from the integration manual completely. Luckily I was able to get bsec 2.0.6.1 from the “deleted” field in this repo. Bsec 2.0.6.1 compiled like a charm.

cpietsch commented 1 year ago

any news on this issue ? running into the same problem with the new BSEC_2.2.0.0 release

mcalisterkm commented 1 year ago

See the steps I set out for the last BSEC update: Issue - Can't Compile because of Library Update? The crux of it is to rename the folder BSEC_2.2.0.0_Generic_Release_30052022 to the old name BSEC_2.0.6.1_Generic_Release_04302021.

Unfortunately this folder name is hard coded into setup.py. Rename the BSEC folder and follow the notes.

It's a work around, and others have suggested the BSEC version is a parameter or parsed but that is up to PI3G.

Keith

Blandi88 commented 1 year ago

See the steps I set out for the last BSEC update: Issue - Can't Compile because of Library Update? The crux of it is to rename the folder BSEC_2.2.0.0_Generic_Release_30052022 to the old name BSEC_2.0.6.1_Generic_Release_04302021.

Unfortunately this folder name is hard coded into setup.py. Rename the BSEC folder and follow the notes.

It's a work around, and others have suggested the BSEC version is a parameter or parsed but that is up to PI3G.

Keith

This method does not work for bsec2.2.x.x

Blandi88 commented 1 year ago

any news on this issue ? running into the same problem with the new BSEC_2.2.0.0 release

After renaming all config files in the code, I got it to compile by additionally getting rid of all references to the following lines;

BSEC_SAMPLE_RATE_HIGH_PERFORMANCE BSEC_OUTPUT_COMPENSATED_GAS

^these are not used in the newest implementation of bsec 2.2.x.x

mcalisterkm commented 1 year ago

The first error is easily fixed by swapping BSEC_SAMPLE_RATE_HIGH_PERFORMANCE to BSEC_SAMPLE_RATE_SCAN which is the same value. The second error is more complex as it is an enumerated value in the BOSCH header bsec_datatypes.h, but in the PI3G code BSEC_OUTPUT_COMPENSATED_GAS is a virtual sensor and part of a data structure holding output values for the BSEC virtual sensors. I thought the easiest approach was to add the definition back into the BOSCH header (bsec_datatypes.h) like this BSEC_OUTPUT_SENSOR_HEAT_COMPENSATED_HUMIDITY = 15,
BSEC_OUTPUT_COMPENSATED_GAS = 18, BSEC_OUTPUT_GAS_PERCENTAGE = 21,

It compiles and installs but is throwing errors "SystemError: ../Objects/longobject.c:415: bad argument to internal function".

So I think BSEC_OUTPUT_COMPENSATED_GAS has to be removed in the PI3G code, from the virtual sensor data structure and its processing, which is more than a quick hack. It's annoying that BOSCH have made such breaking changes with no information/release note. It's even more annoying that BOSCH have removed the previous versions of the library from their website.

Blandi88 commented 1 year ago

The first error is easily fixed by swapping BSEC_SAMPLE_RATE_HIGH_PERFORMANCE to BSEC_SAMPLE_RATE_SCAN which is the same value. The second error is more complex as it is an enumerated value in the BOSCH header bsec_datatypes.h, but in the PI3G code BSEC_OUTPUT_COMPENSATED_GAS is a virtual sensor and part of a data structure holding output values for the BSEC virtual sensors. I thought the easiest approach was to add the definition back into the BOSCH header (bsec_datatypes.h) like this BSEC_OUTPUT_SENSOR_HEAT_COMPENSATED_HUMIDITY = 15, BSEC_OUTPUT_COMPENSATED_GAS = 18, BSEC_OUTPUT_GAS_PERCENTAGE = 21,

It compiles and installs but is throwing errors "SystemError: ../Objects/longobject.c:415: bad argument to internal function".

So I think BSEC_OUTPUT_COMPENSATED_GAS has to be removed in the PI3G code, from the virtual sensor data structure and its processing, which is more than a quick hack. It's annoying that BOSCH have made such breaking changes with no information/release note. It's even more annoying that BOSCH have removed the previous versions of the library from their website.

Go read the integration manual in the bsec2.2.x.x file you downloaded. Looks like you found the solution for BSEC_SAMPLE_RATE_HIGH_PERFORMANCE

however,

BSEC_OUTPUT_COMPENSATED_GAS

^this is no longer used at all.

mcalisterkm commented 1 year ago

I have read the integration manual. Table 1.2.4 lists the virtual sensors, and BOSCH have removed the BSEC_OUTPUT_COMPENSATED_GAS sensor. According to the guide. In the 2.0.6.1 code it was marked as internal and I think it is very much still there, its just become undocumented. So put it back into the enumerated list and bingo it compiles and installs. I have been running my burnin.py program over night and it is still running this morning, using BSEC 2.2.0 and the 1.3v bme688x code. I ran the air quality.py example and it was OK, but my sensor apps had a problem as I reported. So my next step is to let the burnin finish and see if the saved config will load, and test some more.

Just for reference from the 2.0.6.1 guide BSEC_OUTPUT_COMPENSATED_GAS is the Gas Resistance after undergoing temp and humidity compensation. I guess they are not happy with the algorithm.

I have also emailed Nathan the PI3G maintainer.

Blandi88 commented 1 year ago

I have read the integration manual. Table 1.2.4 lists the virtual sensors, and BOSCH have removed the BSEC_OUTPUT_COMPENSATED_GAS sensor. According to the guide. In the 2.0.6.1 code it was marked as internal and I think it is very much still there, its just become undocumented. So put it back into the enumerated list and bingo it compiles and installs. I have been running my burnin.py program over night and it is still running this morning, using BSEC 2.2.0 and the 1.3v bme688x code. I ran the air quality.py example and it was OK, but my sensor apps had a problem as I reported. So my next step is to let the burnin finish and see if the saved config will load, and test some more.

Just for reference from the 2.0.6.1 guide BSEC_OUTPUT_COMPENSATED_GAS is the Gas Resistance after undergoing temp and humidity compensation. I guess they are not happy with the algorithm.

I have also emailed Nathan the PI3G maintainer.

I am able to compile if I change all the fields for BSEC_SAMPLE_RATE_HIGH_PERFORMANCE to BSEC_SAMPLE_RATE_SCAN

And delete all references to BSEC_OUTPUT_COMPENSATED_GAS

I am not sure if this breaks anything, but it works. I can run bmerawdata.py

mcalisterkm commented 1 year ago

All the samples run without error and I can write and read state (but not state written by 2.0.6.1). So I have had to re-run my burn-in and create a new state file.

I have made a clone of this repo fixed to work with BSEC 2.2.0 see https://github.com/mcalisterkm/bme68x-python-library-bsec2.2.0. Note there is an additional step in the installation. This is just a temporary fix until PI3G do an update.

Regards Keith

Blandi88 commented 1 year ago

All the samples run without error and I can write and read state (but not state written by 2.0.6.1). So I have had to re-run my burn-in and create a new state file.

I have made a clone of this repo fixed to work with BSEC 2.2.0 see https://github.com/mcalisterkm/bme68x-python-library-bsec2.2.0. Note there is an additional step in the installation. This is just a temporary fix until PI3G do an update.

Regards Keith

Ran this all last night but woke up to bemrawdata.py giving me a segmentation fault

mcalisterkm commented 1 year ago

Yes there are a quite a few problems to work on: IAQ and CO2 with 2.2.0 are giving crazy values compared to 2.0.6.1. bmerawdata.py runs but I guess it is failing after some time, and segmentation fault is not great. I am running bmerawdata now to see if I can re-create.

Blandi88 commented 1 year ago

Yes there are a quite a few problems to work on: IAQ and CO2 with 2.2.0 are giving crazy values compared to 2.0.6.1. bmerawdata.py runs but I guess it is failing after some time, and segmentation fault is not great. I am running bmerawdata now to see if I can re-create.

The segmentation fault error seems it might stemming from a usb keyboard dongle I have plugged in? Is this even possible?

mcalisterkm commented 1 year ago

The crazy values were coming from the API which has been updated to 4.4.7 - https://github.com/BoschSensortec/Bosch-BME68x-Library. The PI3G library has a copy of v4.4.3 embedded as BME68x-Sensor-API. Updating that to v4.4.7 has improved the IAQ and CO2 values. I have updated my clone here: https://github.com/mcalisterkm/bme68x-python-library-bsec2.2.0.

Try collecting data for 30 min or so with bmerawdata.py.

ChuckKollar commented 1 year ago

I just wasted about 4 hours on this including making suggested changes.

Here is what I have so far. I figured out that the old version of the BSEC__GenericRelease had changed so I did a find . -type f -exec grep OLDLIB {} \; -ls in bme68x-python-library and replaced all old references with the new one. Would have been simpler to just rename it in retrospect as suggested.

Then I did as suggested I change all the fields for BSEC_SAMPLE_RATE_HIGH_PERFORMANCE to BSEC_SAMPLE_RATE_SCAN And delete all references to BSEC_OUTPUT_COMPENSATED_GAS

Now I'm getting loads of warnings, but in the end his error.... /usr/bin/ld: skipping incompatible BSEC_2.2.0.0_Generic_Release_30052022/algo/normal_version/bin/RaspberryPi/PiThree_ArmV6/libalgobsec.a when searching for -lalgobsec /usr/bin/ld: cannot find -lalgobsec collect2: error: ld returned 1 exit status

Any suggestions on how I should proceed would be welcome, and thank you so much in advance.

Charles

$ cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 11 (bullseye)" NAME="Debian GNU/Linux" VERSION_ID="11" VERSION="11 (bullseye)" VERSION_CODENAME=bullseye ID=debian HOME_URL="https://www.debian.org/"

ChuckKollar commented 1 year ago

p.s. I believe that I loaded the 64 bit version of the OS

mcalisterkm commented 1 year ago

There is an ARM V8 binary in BSEC 2.2.0 which is 64 bit. BSEC_2.2.0.0_Generic_Release_30052022/algo/bin/normal_version/RaspberryPi/PiThree_ArmV8

This code uses the ARM V6 32bit binary, in PiThree_ArmV6, so you will have some work to do. The API (https://github.com/BoschSensortec/BME68x-Sensor-API) has some C code examples.

ChuckKollar commented 1 year ago

With the v8 library it seems that I get the same error message after doing a 'clean' and then 'install' using setup.py

aarch64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-z,relro -g -fwrapv -O2 -g -ffile-prefix-map=/build/python3.9-PN012d/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-aarch64-3.9/BME68x-Sensor-API/bme68x.o build/temp.linux-aarch64-3.9/bme68xmodule.o build/temp.linux-aarch64-3.9/internal_functions.o -L/usr/local/lib -LBSEC_2.2.0.0_Generic_Release_30052022/algo/normal_version/bin/RaspberryPi/PiThree_ArmV8 -lpthread -lm -lrt -lalgobsec -o build/lib.linux-aarch64-3.9/bme68x.cpython-39-aarch64-linux-gnu.so /usr/bin/ld: skipping incompatible BSEC_2.2.0.0_Generic_Release_30052022/algo/normal_version/bin/RaspberryPi/PiThree_ArmV8/libalgobsec.a when searching for -lalgobsec /usr/bin/ld: cannot find -lalgobsec collect2: error: ld returned 1 exit status

Any other thoughts?

mcalisterkm commented 1 year ago

I just ran elf read -h on the PiThree_ArmV8 and its 32 bit.

$ readelf -h libalgobsec.a

File: libalgobsec.a(any.o) ELF Header: Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 Class: ELF32 Data: 2's complement, little endian Version: 1 (current) OS/ABI: UNIX - System V ABI Version: 0 Type: REL (Relocatable file)

ChuckKollar commented 1 year ago

Thanks, yes, it behaved liked the last 32-bit library, so I figured as much.

So, it sounds like with a 64-bit OS I'm dead in the water trying to use python with this device. Is that accurate? I really just want something to read humidity.

Again, thanks!

mcalisterkm commented 1 year ago

I think you can run 32 bit library's on 64bit linux, but it required additional config. Have a look at this article https://www.baeldung.com/linux/compile-32-bit-binary-on-64-bit-os

If all you need is to read humidity/temp then install this library without BSEC, or use the Adafruit library for BME680/688 (no support for BSEC).

I also think we need a separate issue started for 64bit support.

ChuckKollar commented 1 year ago

Hi! Thanks for the suggestions. I opted for using adafruit_bme680 library which seems to work on my '''avant garde''' 64-bit OS :-) and it's giving me numbers. Whether they are real or not is another story, but they look roughly believable. I'm using the SparkFun Environmental Sensor - BME688 (Qwiic) card.

Ha... Yes after some nearly 30 years of using 32-bit Linux (my first distribution was SUSE in 1995) you might consider that it's time to make a 64-bit available of your software available. After all, not everyone uses 4-bit Microchip CPUs! :-)

Again, I appreciate your help even though I won't be using the software.

mcalisterkm commented 1 year ago

The BSEC software is from Bosch Sensortec, and they are focussed on low power devices - mostly these are 32bit. Running 64bit on a PI 4 is great but is it not going to last long running from a 3000mAh 3.7v LiPo . My Pi Zero 2 will last a day if I am lucky, but a Puck.js will last 6 months on a CR2030 coin battery. It's horses for courses.

And if your bragging my Unix Programming Env (Kernigham and Pike) says I bought it in Dec 1985!

Regards Keith

ChuckKollar commented 1 year ago

Again, thank you again for your help! Yes, I can see that there is little marketing reason to consider a 64-bit OS. It would be nice if they posted a disclaimer, but I'm guessing that only one or two people per year try this. :-) I'm happy with the solution that I have; any solution really.

Well... I think that I'm a tad older than you are. I dropped out of the University of Pittsburgh in 1978 to take a job at Carnegie Mellon University in the Computer Science Dept. That was the first time that I used AT&T Unix on a PDP-11(34?) which probably had less processing power than the microcontroller that you are using, and used as much electricity as my house. The only thing that I miss about it were the lights and switches. If you were really good and knew the machine code for the instructions, you could toggle a patch into memory without recompiling and reloading! ;-)

Once again Thank you!

Sincerely, Charles

flipthedog commented 1 year ago

I got it up and running with the latest Bosch BSEC API library v2.2.0.0, BME680. Code in a forked repository below. Definitely hacked together, but feel free to take a look at the modifications made to get this running again.

I did the same suggested before:

Then I did as suggested I change all the fields for BSEC_SAMPLE_RATE_HIGH_PERFORMANCE to BSEC_SAMPLE_RATE_SCAN And delete all references to BSEC_OUTPUT_COMPENSATED_GAS

But also changed some buffer sizes as there is now less data coming in.

Then I rebuilt and it worked as expected after heating and calibration period.

https://github.com/flipthedog/bme68x-python-library/

mcalisterkm commented 1 year ago

I am a bit confused as you say your mods are for BSEC 2.2.0 but the code is using includes and configs for BSEC 2.0.6.1: Internal functions.h - "BSEC_2.0.6.1_Generic_Release_04302021/algo/normal_version/bin/RaspberryPi/PiThree_ArmV6/bsec_interface.h” bme68module.c is using an invalid conf path - BSEC_2.0.6.1_Generic_Release_04302021/config/

Have you got both versions of the BSEC libraries on the system you are building on?

Also you say you changed some buffer sizes as there is less data coming in - could you say a bit more on what prompted that?

Regards Keith

Keith McAlister @.***

On 5 Jan 2023, at 19:53, flipthedog @.***> wrote:

I got it up and running with the latest Bosch BSEC API library v2.2.0.0, BME680. Code in a forked repository below. Definitely hacked together, but feel free to take a look at the modifications made to get this running again.

I did the same suggested before:

Then I did as suggested I change all the fields for BSEC_SAMPLE_RATE_HIGH_PERFORMANCE to BSEC_SAMPLE_RATE_SCAN And delete all references to BSEC_OUTPUT_COMPENSATED_GAS

But also changed some buffer sizes as there is now less data coming in.

Then I rebuilt and it worked as expected after heating and calibration period.

https://github.com/flipthedog/bme68x-python-library/

— Reply to this email directly, view it on GitHub https://github.com/pi3g/bme68x-python-library/issues/17#issuecomment-1372664418, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2Y6GZJLYM6EL52PX2NEV3WQ4RE7ANCNFSM6AAAAAARAGWC6Y. You are receiving this because you commented.

MathematicallyEliminated commented 1 year ago

I was able to install and run on my Raspberry Pi 4 from the folders flipthedog provided. It appears the folders were just renamed to old schema rather than updating the setup code.

I was able to get to accuracy 3 but my IAQ will never drop below 50. Once it gets to 50, it begins to recalibrate.

flipthedog commented 1 year ago

My apologies for the confusion, the reply above is correct. I renamed the folder to the old BSEC version to avoid having to change additional code, this was very hacked together. I can fix it sometime if I have additional time.

With regards to the buffer size change, if you look at internal_functions.c, it appears the latest version of the BSEC library no longer has 14 virtual sensors outputting data. Specifically, requested_virtual_sensors[13].sensor_id = BSEC_OUTPUT_GAS_PERCENTAGE;, is no longer present, hence part of the error in this issue. By removing it and amending the n_requested_virtual_sensors = 14; line to 13, it was able to produce code that runs without errors. However, I have not read and checked this against the BSEC Bosch documentation.

Glad you were able to get it to work, I still have to run the sensor for a while, I will see if I can get a stable IAQ.

mcalisterkm commented 1 year ago

In response to IAQ never drops below 50. The Bosch release note for 2-2-0-0 has a new features list, which notes the change to the adaption points moving from 25 - 250 to 50 - 200. So that's why your IAQ does not go below 50. Pretty much all BSEC calculated values jumped up with this release.

When you burn in the sensor I suggest you save the config, and then load that config in your code going forward. It helps get to a stable IAQ quickly. I have examples of this here https://github.com/mcalisterkm/p-sensors/tree/master/src/1.3.0/BurnIn

New features list copied from Bosch bsec 2-2-0-0 release note: New features and changes implemented compared to the previous release | • configuration update, see table 1 below.  • IAQ algorithm improved w/r to interaction and performance by optimized baseline adaptation. Accordingly, the IAQ adaptation points have been updated to 50 & 200 for best user experience (former versions used 25 & 250).  • IAQ output available in continuous mode operation (ODR 1 Hz).  • Temperature-Humidity compensation in IAQ output updated (adopted from BSEC version 1.4.8.0).  • Merged BSEC 1.x and BSEC 2.x (means BSEC 2.2.0.0 incl. full functionality of BSEC 1.4.9.2).  • Static code error/warning fixes. | • configuration update, see table 1 below.  • IAQ algorithm improved w/r to interaction and performance by optimized baseline adaptation. Accordingly, the IAQ adaptation points have been updated to 50 & 200 for best user experience (former versions used 25 & 250).  • IAQ output available in continuous mode operation (ODR 1 Hz).  • Temperature-Humidity compensation in IAQ output updated (adopted from BSEC version 1.4.8.0).  • Merged BSEC 1.x and BSEC 2.x (means BSEC 2.2.0.0 incl. full functionality of BSEC 1.4.9.2).  • Static code error/warning fixes. |  

• configuration update, see table 1 below.  • IAQ algorithm improved w/r to interaction and performance by optimized baseline adaptation. Accordingly, the IAQ adaptation points have been updated to 50 & 200 for best user experience (former versions used 25 & 250).  • IAQ output available in continuous mode operation (ODR 1 Hz).  • Temperature-Humidity compensation in IAQ output updated (adopted from BSEC version 1.4.8.0).  • Merged BSEC 1.x and BSEC 2.x (means BSEC 2.2.0.0 incl. full functionality of BSEC 1.4.9.2).  • Static code error/warning fixes. |

mcalisterkm commented 1 year ago

In response to flipthedog. Thanks I now see what you have done. I left the virtual sensors alone and modified the header to add back in BSEC_OUTPUT_GAS_PERCENTAGE. This virtual sensor was marked as internal only in previous releases, and I guessed that they just removed it from the data structure which proved to be correct.

I could merge your changes to remove BSEC_OUTPUT_GAS_PERCENT with my 2.2.0.0 clone (https://github.com/mcalisterkm/bme68x-python-library-bsec2.2.0) as it already has fixes for the 2.2.0.0 references and config changes.

cpietsch commented 1 year ago

I wonder why the new IAQ range is 50 - 200 and why it results in a better user experience.

IAQ algorithm improved w/r to interaction and performance by optimized baseline adaptation. Accordingly, the IAQ adaptation points have been updated to 50 & 200 for best user experience (former versions used 25 & 250).

Would that mean, the improved IAQ algorithm is faster and more accurate in cost of the outlier (range). Maybe a new Sensor will fill this gap ? https://community.bosch-sensortec.com/t5/image/serverpage/image-id/10360i868E116504889E8C/image-size/large?v=v2&px=999

mcalisterkm commented 1 year ago

There is considerable traffic on the Bosch Sensortec Community site regarding burn-in and stability of readings for the BME688: https://community.bosch-sensortec.com/t5/Bosch-Sensortec-Community/ct-p/bst_community

Bosch say it's ok for +-15% variation between sensors in the same environment - which seems to include the same sensor taken out and put back. It would be handy if Bosch provided some calibration data, but calibration seems to be one for the customer to do.

mcalisterkm commented 1 year ago

My apologies for the confusion, the reply above is correct. I renamed the folder to the old BSEC version to avoid having to change additional code, this was very hacked together. I can fix it sometime if I have additional time.

With regards to the buffer size change, if you look at internal_functions.c, it appears the latest version of the BSEC library no longer has 14 virtual sensors outputting data. Specifically, requested_virtual_sensors[13].sensor_id = BSEC_OUTPUT_GAS_PERCENTAGE;, is no longer present, hence part of the error in this issue. By removing it and amending the n_requested_virtual_sensors = 14; line to 13, it was able to produce code that runs without errors. However, I have not read and checked this against the BSEC Bosch documentation.

Glad you were able to get it to work, I still have to run the sensor for a while, I will see if I can get a stable IAQ.

Two problems: a) Each release of the BSEC library comes with an updated API and you updated the library but not the API.
The API is in the folder BME68x-Sensor-API (v4.4.4 in your fork), and the BOSCH SENSORTEC git hub currently has v4.4.7 (https://github.com/boschsensortec/BME68x-Sensor-API). b) You included the BOSCH BSEC library in your fork. Did you check out the licence agreement.

flipthedog commented 1 year ago

I understand, I will delete my fork due to Licensing issues. My goal was merely to get this Python interface up and running with the pi3g code in this repository.

mcalisterkm commented 1 year ago

The good news is that I have made your changes, using the API v4.4.7 , BSEC 2.2.0.0 with fixed references, so your work it is not wasted, You can find it here:

https://github.com/mcalisterkm/bme68x-python-library-bsec2.2.0-13sensors

Feel free to use/fork or whatever. I have kept my old version supporting 14 sensors, but perhaps best to use the one above.

MathematicallyEliminated commented 1 year ago

In order to use the v4.4.7 api do i need to do anything more than replace the API folder with the one you have in the link above?

I have also noticed that the raw pressure coming out of BSEC looks like it is in Pascal and not hecto Pascal. Not sure if this changed or was incorrect in the documentation.

mcalisterkm commented 1 year ago

The Integration Guide for 2.2.0.0 says Pa - Pressure sensor output of BME68x [Pa]. But I agree with you its changed and used to be hecto pascal, but I am not sure when. Edit - I see what you are saying the PythonDocumentation.md says hPA. Unfortunately I wrote much of that and contributed it to PI3G. I am updating my copy. Thanks for pointing it out.

The simple answer is yes it's just a replacement of the API. But the API reflects changes in the BSEC library, and those can be breaking changes as we have seen. The API contains conversion functions that convert BSEC internal values and apply compensation, to human readable values and these can change. So the API and the BSEC version have to be aligned.