mstrens / oXs_on_RP2040

version of openXsensor to be used on raspberry pi pico RP2040 (more protocols, more functionalities)
84 stars 22 forks source link

Logger for oXs_RP2040 #111

Closed Satcomix closed 10 months ago

Satcomix commented 10 months ago

Hello Mstrens, Can I open the issue here or on Logger GitHub? Board RPi 2040 Pico In the meantime I tried to play the .uf2 from the logger project onto a board. If I press ? and Enter, I don't see any help for entering commands. Greeting, Torsten

Rebooted by Watchdog!

Config parameters are OK Press ? + Enter to get help about the commands

Type any character to begin +++++++++++

mstrens commented 10 months ago

I expect that there was an issue with the upload on github. I made a reload under version 2.8.32. If the message is still generated, the Hex value of the first pos should be part of the message

Satcomix commented 10 months ago

Test: Version 2.8.32 FBUS Protocol Sometimes sensor lost,ok, by 32 telemetry values

Logger 0.0.5 max ms between 2 call to main loop= 1 max % used of Serial2 fifo= 0 max % of bytes in csvbuf= 2 max us to fill csv rec= 274 max % used of fromNbr queue= 1 max % used of lastWritten queue= 1 max us to write on sd card= 3877 max us to synchro on sd card= 17668 total bytes written in csv file= 5823430 Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx

max ms between 2 call to main loop= 1 max % used of Serial2 fifo= 0 max % of bytes in csvbuf= 3 max us to fill csv rec= 311 max % used of fromNbr queue= 1 max % used of lastWritten queue= 1 max us to write on sd card= 5465 max us to synchro on sd card= 19248 total bytes written in csv file= 6476422 Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx Reset writeIdx

mstrens commented 10 months ago

So, this seems Ok It seems that it writes a lot of data on the sd card. Did you already look at the content of the csv file on the sd card (with e.g. excel)?

Satcomix commented 10 months ago

I have a total of 3 files on the SD card. I looked at the last one with openoffice calc and everything seems plausible to me

Satcomix commented 10 months ago

Hello Mstrens, I think that version 2.8.32 and Logger 0.0.5 run well. I would hope that others who have expressed their desire for these features would also help with the tests in the future. Thank you for your work. I'll close the issue now. Greetings, Torsten

Satcomix commented 10 months ago

Hello Mstrens, I looked at the .csv file from 0.0.11 Logger with OpenOffice. The values ​​for GPS position seem implausible to me and the values ​​for GPS date/time are not displayed.

oXsLog000.zip

Greetings, Torsten

mstrens commented 10 months ago

Can you provide me a csv file that you create. For GPS position, I used a special format (trying to do the same as openTx log file). In the doc I found, position in csv is supposed to be "XXX.XXXXXX YYY.YYYYYYY". It means that long and lat are in one single column, separated by a space and in degree with 7 decimals. Perhaps can you also provide me a log file from the handset with position and date and time filled so I can compare.

Satcomix commented 10 months ago

Hello Mstrens, If I understand you correctly, you want a logger .csv file only with GPS and one log file from my handset. Since I have never saved logs on the handset, I first have to see how I can do it.

mstrens commented 10 months ago

It does not matter if the oXs logger file (and the handset log file) contains only GPS data. It can contains other data too. What matter is that the GPS got a fix and so that the gps position fields are present (in order to look at the format)

In openTX, I think you have use the menu special function in order to activate the log. Then you have also (I do not remember where but probably in telemetry menu) to specify which fields must be included. I presume it is the same for Edge or Ethos.

Satcomix commented 10 months ago

Hello Mstrens, I just noticed something else: If I deactivate V1-V4 with 255, and there was a temp sensor on V3 and it is not deleted with 255, the GPS does not initialize via TX/RX.

mstrens commented 10 months ago

When you say that you deactivate V1-V4 do you mean that you deactivate only V1 and V4 (or that you deactivate V1,V2,V3,V4).

Anyway, I am not sure that the issue is related to activating or not some voltage. I suggest that when you change some parameters, you always make a power off. After saving the parameters (after a change), oXs perform a reset but the GPS (and other I2C devices) are not automatically reset. So it could be that they do not react as expected (e.g. the GPS is not anymore expecting commands at 9600 bauds in case of M8)

Satcomix commented 10 months ago

I deactivate V1 V2 V3 V4 with 255, but i forget to deactivate TEMP=1, and so V3 was a Tempsensor. I make power off and on but nothing happens. GPS cant initialize with 10HZ and no telemetry data was on my handset.

mstrens commented 10 months ago

At power on oXs check the config with this code (note "2" here means V3 because C++ index starts at 0 and not at 1) if (config.temperature == 1 and config.pinVolt[2] == 255){ printf("Error in parameters: when 1 temperature sensor is used (TEMP = 1), a pin for V3 must be defined too)\n"); configIsValid=false; } So (in principe) oXs gives a message and the config is considered as wrong. As result, oXs does not manage telemetry,Sbus, PWM ,... It only accept new USB commands in order to correct the wrong parameters.

Satcomix commented 10 months ago

oXs give me a message, I didn't recognize it during all the testing I activate LOGS in my X20S and see the red Dot at the top of the screen, now i will look for the folder.

Satcomix commented 10 months ago

FrSky X20S Logs:

AROCS-4663-AS-2023-10-18-19-25-46.zip

mstrens commented 10 months ago

There is something abnormal in the csv. There is a discrepancy between Header and data (one data less) Also strange that column with gps timestamp has a header with Km/h.

I presume this is a bug in handset program?

Satcomix commented 10 months ago

From flight altitude everything is shifted by one column: Flight altitude empty because not activated In the next column under Airspeed is the GPS Date/Time, GPS Time is the latitude etc......

mstrens commented 10 months ago

There is a difference between openTx and Ethos https://github.com/FrSkyRC/ETHOS-Feedback-Community/issues/926

It seems that Ethos does not accept to change.

Satcomix commented 10 months ago

OK, i noticed it. Then it has to stay that way for now. Next step: RPM Can / should i make some more tests?

mstrens commented 10 months ago

I do not know how people can use google map to visualize the positions captured with Ethos. For openTx is was done using compagnion as far I know.

about rpm, I looked at the code but did not see a bug (up to know). do you have telemetry lost when you have only RPM as sensor? Does it happens also with RPM sensor when log is not activated?

Satcomix commented 10 months ago

Normally i have no Sensor lost,(1 or 2 messages in1/2hour) when i have activate all sensors but without logger port. I will make tomorow morning some more Tests. Greetings, Torsten

mstrens commented 10 months ago

In a previous message, you said that gps date and time where missing in csv file. I do not see a bug in the code. Please note that GPS date and time are provided by oXs only for some type of GPS (M10?). Perhaps you made a test with a gps where date and time are not decoded. Still, if you get gps date and time on the handset, you should also get them in the csv log file. if not, there is a bug or in oXs or in the logger. I will make a small change: when oXs does not provide gps date and time, the associated csv fields will be empty. It was already the case for the date but not yet for the time.

Satcomix commented 10 months ago

Hello Mstrens, Just got home from work at the moment. I think I would only have zeros everywhere for GPS Date/Time, but everything is always there in the FV from the terminal. I will check the .csv files again. For over a year I have been using a FrSky-X20S and now X20 Pro both with TD-R10 receiver, usually with FBUS protocol, and several of my Pimoroni Servo2040, to which all sensors, GY86,GY63,GY521,ADS.... and a Beitian BE250 M10 are connected. A lot of hardware has accumulated here over the entire test phase. I will take a look at the .csv files right after a coffee. Best regards, Torsten

Satcomix commented 10 months ago

Some posts above in the.csv zip file (oXsLog000.csv) have a look a 3D/Numer of Sats and GPS position and GPS Date/Time.

Satcomix commented 10 months ago

Just from this moment. 11Sats but no Date/Time oXsLog005.zip

Cmd to execute: FV

GPS Latitude = XX.1224800 degree GPS Longitude = X.7172760 degree GPS Groundspeed = 2 cm/s GPS Heading = 113.860000 degree GPS Altitude = 3708 cm GPS Num sat. = 111 GPS Date J M A = 20 10 23 GPS Time H M S = 9 22 16 GPS Pdop = 176 GPS Home bearing = 260 degree GPS Home distance = 10 m Volt 1 = 1790 mVolt Current (Volt 2) = 1687 mA Volt 4 = 302 mVolt Capacity (using current) = 787 mAh Temp 1 (Volt 3) = 22 degree Vspeed = 0 cm/s Baro Rel altitude = 850 cm Pitch = -1.630000 degree Roll = 0.190000 degree RPM = 998 Hertz Ads 1 1 = 119 mVolt Ads 1 2 = 119 mVolt Ads 1 3 = 119 mVolt Ads 1 4 = 119 mVolt Ads 2 1 = 119 mVolt Ads 2 2 = 117 mVolt Ads 2 3 = 118 mVolt Ads 2 4 = 119 mVolt Airspeed = 115 cm/s Compensated Vspeed = 6 cm/s Gps cumulative distance = 188 Acc X = 0.022000g Acc Y = 0.002000g Acc Z = 0.827000g Vspeed compensation = 1.10 pwmTop= 20000

Satcomix commented 10 months ago

I noticed something else unusual. You write in the .readme: Blue, slow blinking : the logger do net get data Green, slow blinking: data are recorded

Power on: no RGB LED Press Enter on Ardu IDE Serial Monitor: RGB LED lights blue. Next press ENTER in Terminal Window: RGB LED gets green blinking

Do the oXs_Logger don`t start automaticly, after power on???

mstrens commented 10 months ago

There is a bug in the data being recorded. It seems that it is not only for gps date and time (based on FV and csv file you provided). I do not yet know if it is in oXs or in the logger. I am preparing a version of logger to print (at some intervals) one set of values to be converted to csv and the csv record.

About LED: what I said is valid for a RP2040 Zero that has a RGB LED on pin 16. If you are using the challenger board, I expect that this is not valid.

Satcomix commented 10 months ago

"About LED: what I said is valid for a RP2040 Zero that has a RGB LED on pin 16. If you are using the challenger board, I expect that this is not valid."

I`m using a Challenger Board with an external RGB LED(WS...) at GPio16, it is the same way i did it all the time with my Pimoroni Boards, becaus the internal RGB LED is on GPio18 the same as the Challenger SD/RTC Board.

"I do not yet know if it is in oXs or in the logger." On the handheld, all the time i get the right Date/Time data values, because i set the X20S and Pro to get Time from GPS. I think it is a problem with the Logger, or maby on the GPio Pin to Logger from normal oXs.

mstrens commented 10 months ago

RGB Leds are not unified. It means that some colors are inverted. To control a LED, I have to send 3 bytes (one per color) but the order of the 3 bytes can differ.

In the mean time I put logger 0.0.12 It is just the same as previous version but it should also print from time to time the raw values received from oXs and the related csv record. It could be that printing those value slows down the process and generates other errors but it is just for debugging.

Satcomix commented 10 months ago

First ENTER:

Rebooted by Watchdog!

Config parameters are OK Press ? + Enter to get help about the commands

Type any character to begin +++++++++++

Autoprobing for a RTC... Autoprobe found MCP7941x at 0x6F Calibration: 0 log file will be in oXsLog007.csv end of setup No data within 30155 current: 0=0 1=0 2=0 3=0 4=0 5=0 6=0 7=0 8=0 9=0 10=0 11=0 12=0 13=0 14=0 15=0 16=0 17=0 18=0 19=0 20=0 21=0 22=0 23=0 24=0 25=0 26=0 27=0 28=0 29=0 30=0 31=0 32=0 33=255 34=0 35=0 36=0 37=0 38=0 39=0 40=0 41=0 42=0 43=0 44=0 45=0 46=0 47=0 48=0 49=0 50=0 51=0 52=0 53=0 54=0 55=0 56=0 57=0 58=0 59=0 60=0 61=0 62=0 CSV: 7080847,0 0,0,0,0,0,,,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2.55,0,0,0,0,0,0,0,0,0,0,0,0,0

Satcomix commented 10 months ago

Second ENTER:

Version = 0.0.12 Fields in CSV log (+ = included; - = not included)

Config parameters are OK Press ? + Enter to get help about the commands current: 0=253 1=255 2=5 3=129 4=185 5=117 6=255 7=0 8=173 9=35 10=12 11=180 12=169 13=0 14=62 15=44 16=22 17=0 18=254 19=2 20=104 21=12 22=0 23=237 24=119 25=119 26=119 27=118 28=120 29=119 30=120 31=122 32=44 33=0 34=0 35=0 36=239 37=22 38=2 39=60 40=0 41=0 42=0 43=0 44=0 45=0 46=0 47=0 48=0 49=0 50=0 51=0 52=0 53=0 54=0 55=0 56=0 57=0 58=0 59=0 60=0 61=0 62=0 CSV: 7141759,0.000025 0.000025,0.05,1.29,1.85,117,2000:00:00,,1.73,35,12,0.180,0.169,0,0.062,0.044,22,0,2.54,0.02,1.04,0.12,0,237,0.119,0.119,0.119,0.122,0.44,0,0,0,239,22,2,60,0,0,0,0,0,0,0

Satcomix commented 10 months ago

"RGB Leds are not unified. It means that some colors are inverted. To control a LED, I have to send 3 bytes (one per color) but the order of the 3 bytes can differ."

Did you change anything in the actual code for the LED from the oXs for the logger? Because this type of RGB LED works perfectly otherwise. It also flashes green, as described in the Readme, but only after ENTER twice

mstrens commented 10 months ago

I have to check for the led.

I found a bug in oXs in the way the data where formatted to be sent to the logger. I put version 2.8.37 I hope the csv value will be OK. If so, I will remove the last debug msg from the logger

Satcomix commented 10 months ago

"I put version 2.8.37" Can you please upload it to GH again. I see only 2.8.36

mstrens commented 10 months ago

Sorry, I forgot to upload.

Satcomix commented 10 months ago

Next i will look in the .csv

Version = 0.0.12 Fields in CSV log (+ = included; - = not included)

Config parameters are OK Press ? + Enter to get help about the commands current: 0=511224830 1=67172586 2=9 3=33627 4=4538 5=119 6=386536703 7=201394176 8=111 9=274 10=2 11=1895 12=1697 13=0 14=325 15=19 16=22 17=0 18=2 19=-6 20=-158 21=17 22=0 23=1006 24=119 25=119 26=119 27=119 28=120 29=120 30=121 31=122 32=122 33=20 34=0 35=0 36=6 37=21 38=1 39=829 40=0 41=0 42=0 43=0 44=0 45=0 46=0 47=0 48=0 49=0 50=0 51=0 52=0 53=0 54=0 55=0 56=0 57=0 58=0 59=0 60=0 61=0 62=0 CSV: 41824,51.122483 6.717258,0.09,336.27,45.38,119,2023:10:20,12:01:08,1.11,274,2,1.895,1.697,0,0.325,0.019,22,0,0.02,-0.06,-1.58,0.17,0,1006,0.119,0.119,0.119,0.122,1.22,0.20,0,0,6,21,1,829,0,0,0,0,0,0,0

mstrens commented 10 months ago

About RGB led in logger: When (for debug purpose) the logger ask to type any char to begin, the logger program is stopped (waiting for the char). So it is normal that led does not blink. Blinking is only done later on in the main loop.

When debugging will be done, I will not ask for pressing a char anymore

mstrens commented 10 months ago

with 2.8.37 it seems OK.

Satcomix commented 10 months ago

Have a look :-(( GPS Position: OK GPS date/time : empty

oXsLog008.zip

Satcomix commented 10 months ago

oXs_Logger_output.zip

mstrens commented 10 months ago

in log 008, date and time are filled after some entries in the csv. This can be the result that oXs get each sensor data one by one. For GPS there are about 10 different data's. Each time the main loop runs, oXs takes all data received since previous loop and send a message to the logger with only those "new" data's. It can be that one message contains only part of the GPS data. The other one would then be sent in the next message to the logger. The logger read the message and build the "actual" values of all data's (= update the actual values with those in the message). At the end of the message, this "actual" set of values is put in csv format and is written on the sd card.

Satcomix commented 10 months ago

OK, my mistake, I didn't scroll to line 139 in the small preview of OpenOffice. Sorry

mstrens commented 10 months ago

in logger 0.0.13, I remove the last debug msg and the need to enter a char at power on.

Satcomix commented 10 months ago

Now, after reboot no light, by enter into terminal first blue and then green blinking.

Clean boot

Config parameters are OK Press ? + Enter to get help about the commands

Autoprobing for a RTC... Autoprobe found MCP7941x at 0x6F Calibration: 0 log file will be in oXsLog011.csv end of setup No data within 21278

Version = 0.0.13 Fields in CSV log (+ = included; - = not included)

Config parameters are OK Press ? + Enter to get help about the commands

Satcomix commented 10 months ago

I absolutely need the backup battery, after every power down, I must do a DT=, or is it possible to get the first timestamp from GPS? Would it perhaps be possible for owners of other boards to have a choice for the LED GPio for oXs_Logger and oXs_RP2040?, e.g. like LED=16, LED=18 But only if it's possible and doesn't take too much work.

mstrens commented 10 months ago

"Now, after reboot no light, by enter into terminal first blue and then green blinking."

This is not clear for me. Normally I expect it should be blue (fix) after a power on or a reset. Then it can take some time (seconds) because the logger has to set up the SD card and create a new (empty) csv file. Only after this, the logger enters the main loop that should start led blinking.

It is expected that when you have no backup battery but that you have a GPS providing date and time, then the creation date & time of the log file should be updated with the first date and time received. If it is not so, there is some bug. Please note that the "last modified" date and time of the log file is never updated.

About the led I could let the user select a pin. The problem with many boards is that they do not have a RGB led but only a single color. So there should be some other convention to make the distinction between the different states. Do you have other idea's?

Satcomix commented 10 months ago

Hello Mstrens, After repower and reboot the Logger, the RGB LED at GPio16 is OFF, after first push ENTER the LED gets a small time (3-4sec) blue and than green blinking. I own 12 different RP2040 boards (24 in total) 98% of which have an RGB LED installed, only the Challenger has a dual color LED and the Pimoroni PGA2040 has none. I think for the normal user LED=16 as default and the other can choose with channel select.

Satcomix commented 10 months ago

Hello Mstrens, The logger board was repowered, the date and time are not correct because there is no backup battery, everything is ok so far, in the log.csv the GPS time starts at line 78 and the logger should now take the time from the GPS, or have I got you wrong understood

oXsLog015.zip

mstrens commented 10 months ago

in oXs 2.8.38, I added a command RGB to let the user select the gpio for RGB led. I still have to to the same for the logger.

Satcomix commented 10 months ago

Hello Mstrens, Thank you very much, this feature will make testing better. greetings, Torsten