hannobraun / embedded

I'm teaching myself embedded programming. This repository is where I put my code. There's also a blog.
http://hannobraun.de/embedded
27 stars 4 forks source link

sermon output question #1

Open nyx opened 8 years ago

nyx commented 8 years ago

I managed to get things working on Mac OS X (I hope to provide a PR with added notes for the platform in the future). I am having trouble with the upload program, had to increase the serial port timeout and then still had an issue with a failing assertion in eefc.rs, I ended up using BOSSA to upload the .bin which is fine for now. However the reason I'm opening this issue is b/c when I run sermon I am getting garbled output. I altered the blink program.rs to print * instead of the long output it was producing. On the receiving end however I get 'Z', I looked at an ASCII chart and see 'Z' - * = 48. So I changed the print to '+' (comes right after '*' in ASCII) and sure enough on the receiving end I now see '[' (48 after '+'). Do you have any idea why this would be?

hannobraun commented 8 years ago

Hi, thanks for opening this issue.

I managed to get things working on Mac OS X (I hope to provide a PR with added notes for the platform in the future).

This is great. Such a pull request would be very welcome!

I am having trouble with the upload program, had to increase the serial port timeout and then still had an issue with a failing assertion in eefc.rs, I ended up using BOSSA to upload the .bin which is fine for now.

The upload program isn't of the highest quality, so I'm not surprised it's failing with some assertion. That said, it's been exceptionally reliable for my own use, as long as I'm using it for the exact use case I wrote it for.

Even though BOSSA is working for you, I have some suggestions, in case you want to try again:

I'd also appreciate it, if you could post the exact assertion output. Although I suspect that if none of my suggestions help, I won't be able to do anything to fix it.

I plan to clean up the uploader at some point (add better error messages), and also add support for more microcontrollers. It will be a while before I'll get to it though.

However the reason I'm opening this issue is b/c when I run sermon I am getting garbled output. I altered the blink program.rs to print * instead of the long output it was producing. On the receiving end however I get 'Z', I looked at an ASCII chart and see 'Z' - * = 48. So I changed the print to '+' (comes right after '*' in ASCII) and sure enough on the receiving end I now see '[' (48 after '+'). Do you have any idea why this would be?

I have an idea: Decimal 48 is the ASCII value for '0' (zero). If you want to convert the number 0 into the ASCII character '0', you can do so by adding 48. The same goes for other digits, of course. So while it's weird that the offset is there, the offset being 48 is much less weird than any other offset would be :smiley:

I see three possible causes:

  1. You made some changes to the code that would explain this.
  2. Your hardware or software is somehow different than mine, and this is somehow causing this.
  3. I've made some change that introduced this bug and didn't test the serial output properly since then.

While I wouldn't rule out 3 as a likely cause, I suspect it's 2. Specifically, I'm guessing that serial communication on OS X works differently than it does on Linux, but I don't know. I suggest you download a more proven application for serial communication and see what happens with that.

I'll test it again with my setup to try and rule out cause number 3. Probably on Monday.

nyx commented 8 years ago

Good point with your idea re conversion offset. I did update the vendor Rust to: 4db1874f4c26a2c88acb5e46b413e9c4adce3477

Didn't try with the original one. I wonder if perhaps something changed there and now that offset is being doubly applied? As you suggested I tried another more established serial monitor - the Arduino IDE one, and it shows the same behavior I see with Sermon. Sorry about the lack of details here I've just had time to pop in and out periodically today.

hannobraun commented 8 years ago

Hmm, I don't think a Rust update could have caused this. At least I can't think of a way that wouldn't have broken a lot more code than this. Interesting that the Arduino IDE shows the same behavior. That indicates that the problem lies with the code on the Arduino side, not with Sermon.

Well, we'll hopefully learn more once both of us had a chance to take a closer look.

nyx commented 8 years ago

I can verify that switching to the native USB port allows the upload program to work properly.

hannobraun commented 8 years ago

I've tested this and everything works on my side. I also updated the Rust version to the latest nightly, and updated sermon's dependencies to the latest versions. No change, everything works as expected.

No idea what causes your problem. I'd like to look at your code, if that's possible.

nyx commented 8 years ago

I forked this repo and made a mac-osx branch there showing my current test setup. I hacked uart.rs to send bytes with values from 0 to 127 and modified sermon to output the byte values as it reads them instead of printing as strings. I see from this experiment that it's not as simple as a constant offset of 48, rather there is something more complex happening that I haven't figured out yet.

Here's my version: https://github.com/nyx/embedded/tree/mac-osx

Nothing ready for a PR, but at least you can see exactly what I'm working with. Do you have access to a Mac to test this with? I will be adding more details regarding my setup to that branch soon.

nyx commented 8 years ago

I've attached a file here showing the byte sequence recieved in sermon in binary representation, I've marked a few of the jumps (of 17) with a < so you can see odd the behavior more clearly. sequence-discrepancies.txt

hannobraun commented 8 years ago

I forked this repo and made a mac-osx branch there showing my current test setup. [...]

Thanks!

Nothing ready for a PR, but at least you can see exactly what I'm working with. Do you have access to a Mac to test this with? I will be adding more details regarding my setup to that branch soon.

Unfortunately I don't have a Mac.

I've attached a file here showing the byte sequence recieved in sermon in binary representation, I've marked a few of the jumps (of 17) with a < so you can see odd the behavior more clearly.

This is very weird. There seems to be a pattern to it. Every time we'd expect the lower five bits to jump from 00111 to 01000, it jumps to 11000 instead. There's at least one additional discrepancy that doesn't fit that pattern though (where two of your < markers are next to each other). I don't know what any of that means.

I have three ideas at this point:

  1. The USB port (or related hardware) on your Arduino Due might be busted. To test this, I suggest you upload a regular Arduino program that uses the serial port and verify it works. If I recall correctly, there's an example Arduino sketch that prints out an ASCII table included in the Arduino IDE. That one should be perfect to find any errors.
  2. The USB port (or related hardware) on your Mac might be busted. I suggest you repeat the test with all the USB ports on your Mac. Might be there's just a problem with the one you're using right now.
  3. You might be using the wrong device file. I know very little about OS X, so I'm not at all sure about this. However, I found an Arduino tutorial for Mac OS X that uses the /dev/tty.* device instead of /dev/cu.*. I'm not sure if that's relevant, but it might be worth a shot.
nyx commented 8 years ago

Re: 1) I've tested the board using the normal Arduino IDE workflow - serial works as expected there. I've also tested with a second board and see consistent results.

Re: 2) I've tried all the USB ports on my Mac and see consistent results.

Re: 3) I don't think this is the case because /dev/tty.* files do not exist, and I have success using the device paths when using "official" Arduino software.

You're interpretation of my data fits my own understanding, it's very odd and confounding.

hannobraun commented 8 years ago

Well, unless I'm missing something, that only leaves the worst case: A subtle bug in my UART code that for some reason manifests itself with your setup, but not with mine. I'll review the code and think some more about your data. Maybe I can come up with more ideas.

hannobraun commented 8 years ago

I'd like you to try something. I've pushed a new branch, issue-1. The branch contains the following changes:

The new program sends single bytes, starting at 0. It counts up to 255, then counts down to 0 and so forth. If it detects an error, it stops sending bytes and starts a specific blinking pattern, depending on the error encountered.

When I run this, sermon detects a few errors in the beginning in the bytes received before sermon was reading them. Once the program restarts, no more errors are detected. No errors at all are detected on the board.

My hope is that you will see an error reported by the program on the board that tells us more about what could cause this. If you encounter an error, you can identify it by its blinking pattern:

The patterns are defined here: https://github.com/hannobraun/embedded/blob/1d3be4b91e606476610ebbf7a9386949114e2727/blink/src/program.rs#L29

This is example fo sermon output that I'm seeing:

Number of Errors: 0 / Value: 0x78
Number of Errors: 0 / Value: 0x79
Number of Errors: 0 / Value: 0x7A
Number of Errors: 0 / Value: 0xF6 <= Error
Number of Errors: 1 / Value: 0xF7
Number of Errors: 1 / Value: 0xF8
Number of Errors: 1 / Value: 0xF9
Number of Errors: 1 / Value: 0xFA
Number of Errors: 1 / Value: 0xFB
Number of Errors: 1 / Value: 0xFC
Number of Errors: 1 / Value: 0xFD
Number of Errors: 1 / Value: 0xFE
Number of Errors: 1 / Value: 0xFF
Number of Errors: 1 / Value: 0xFE
Number of Errors: 1 / Value: 0xFD
Number of Errors: 1 / Value: 0xFC
Number of Errors: 1 / Value: 0xFB
Number of Errors: 1 / Value: 0xFA
Number of Errors: 1 / Value: 0xF9
Number of Errors: 1 / Value: 0xF8
Number of Errors: 1 / Value: 0xF7
Number of Errors: 1 / Value: 0xF6
Number of Errors: 1 / Value: 0xF5
Number of Errors: 1 / Value: 0xF4
Number of Errors: 1 / Value: 0xF3
Number of Errors: 1 / Value: 0xF2
Number of Errors: 1 / Value: 0xF1
Number of Errors: 1 / Value: 0xF0
Number of Errors: 1 / Value: 0xEF
Number of Errors: 1 / Value: 0xEE
Number of Errors: 1 / Value: 0xED
Number of Errors: 1 / Value: 0xEC
Number of Errors: 1 / Value: 0xEB
Number of Errors: 1 / Value: 0xEA
Number of Errors: 1 / Value: 0xE9
Number of Errors: 1 / Value: 0xE8
Number of Errors: 1 / Value: 0xE7
Number of Errors: 1 / Value: 0xE6
Number of Errors: 1 / Value: 0xE5
Number of Errors: 1 / Value: 0xE4
Number of Errors: 1 / Value: 0xE3
Number of Errors: 1 / Value: 0xE2
Number of Errors: 1 / Value: 0xE1
Number of Errors: 1 / Value: 0xE0
Number of Errors: 1 / Value: 0xDF
Number of Errors: 1 / Value: 0xDE
Number of Errors: 1 / Value: 0xDD
Number of Errors: 1 / Value: 0xDC
Number of Errors: 1 / Value: 0xDB
Number of Errors: 1 / Value: 0xDA
Number of Errors: 1 / Value: 0xD9
Number of Errors: 1 / Value: 0xD8
Number of Errors: 1 / Value: 0xD7
Number of Errors: 1 / Value: 0xD6
Number of Errors: 1 / Value: 0xD5
Number of Errors: 1 / Value: 0xD4
Number of Errors: 1 / Value: 0xD3
Number of Errors: 1 / Value: 0xD2
Number of Errors: 1 / Value: 0xD1
Number of Errors: 1 / Value: 0xD0
Number of Errors: 1 / Value: 0xCF
Number of Errors: 1 / Value: 0xCE
Number of Errors: 1 / Value: 0xB1 <= Error
Number of Errors: 2 / Value: 0xB2
Number of Errors: 2 / Value: 0xB3
Number of Errors: 2 / Value: 0xB4
Number of Errors: 2 / Value: 0xB5
Number of Errors: 2 / Value: 0xB6
Number of Errors: 2 / Value: 0xB7
Number of Errors: 2 / Value: 0xB8
Number of Errors: 2 / Value: 0xB9
Number of Errors: 2 / Value: 0xBA
Number of Errors: 2 / Value: 0xBB
Number of Errors: 2 / Value: 0xBC
Number of Errors: 2 / Value: 0xBD
Number of Errors: 2 / Value: 0xBE
Number of Errors: 2 / Value: 0xBF
Number of Errors: 2 / Value: 0xC0
Number of Errors: 2 / Value: 0xC1
Number of Errors: 2 / Value: 0xC2
Number of Errors: 2 / Value: 0xC3
Number of Errors: 2 / Value: 0xC4
Number of Errors: 2 / Value: 0xC5
Number of Errors: 2 / Value: 0xC6
Number of Errors: 2 / Value: 0xC7
Number of Errors: 2 / Value: 0xC8
Number of Errors: 2 / Value: 0xC9
Number of Errors: 2 / Value: 0xCA
Number of Errors: 2 / Value: 0xCB
Number of Errors: 2 / Value: 0xCC
Number of Errors: 2 / Value: 0xCD
Number of Errors: 2 / Value: 0xCE
Number of Errors: 2 / Value: 0xCF
Number of Errors: 2 / Value: 0xD0
Number of Errors: 2 / Value: 0xD1
Number of Errors: 2 / Value: 0xD2
Number of Errors: 2 / Value: 0xD3
Number of Errors: 2 / Value: 0xD4
Number of Errors: 2 / Value: 0xD5
Number of Errors: 2 / Value: 0xD6
Number of Errors: 2 / Value: 0xD7
Number of Errors: 2 / Value: 0xD8
Number of Errors: 2 / Value: 0xD9
Number of Errors: 2 / Value: 0xDA
Number of Errors: 2 / Value: 0xDB
Number of Errors: 2 / Value: 0xDC
Number of Errors: 2 / Value: 0xDD
Number of Errors: 2 / Value: 0xDE
Number of Errors: 2 / Value: 0xDF
Number of Errors: 2 / Value: 0xE0
Number of Errors: 2 / Value: 0xE1
Number of Errors: 2 / Value: 0xE2
Number of Errors: 2 / Value: 0xE3
Number of Errors: 2 / Value: 0xE4
Number of Errors: 2 / Value: 0xE5
Number of Errors: 2 / Value: 0xE6
Number of Errors: 2 / Value: 0xE7
Number of Errors: 2 / Value: 0xE8
Number of Errors: 2 / Value: 0xE9
Number of Errors: 2 / Value: 0xEA
Number of Errors: 2 / Value: 0xEB
Number of Errors: 2 / Value: 0xEC
Number of Errors: 2 / Value: 0xED
Number of Errors: 2 / Value: 0xEE
Number of Errors: 2 / Value: 0xEF
Number of Errors: 2 / Value: 0xF0
Number of Errors: 2 / Value: 0xF1
Number of Errors: 2 / Value: 0xF2
Number of Errors: 2 / Value: 0xF3
Number of Errors: 2 / Value: 0xF4
Number of Errors: 2 / Value: 0xF5
Number of Errors: 2 / Value: 0xF6
Number of Errors: 2 / Value: 0xF7
Number of Errors: 2 / Value: 0xF8
Number of Errors: 2 / Value: 0xF9
Number of Errors: 2 / Value: 0xFA
Number of Errors: 2 / Value: 0xFB
Number of Errors: 2 / Value: 0xFC
Number of Errors: 2 / Value: 0xFD
Number of Errors: 2 / Value: 0xFE
Number of Errors: 2 / Value: 0xFF
Number of Errors: 2 / Value: 0xFE
Number of Errors: 2 / Value: 0xFD
Number of Errors: 2 / Value: 0xFC
Number of Errors: 2 / Value: 0xFB
Number of Errors: 2 / Value: 0xFA
Number of Errors: 2 / Value: 0xF9
Number of Errors: 2 / Value: 0xF8
Number of Errors: 2 / Value: 0xF7
Number of Errors: 2 / Value: 0xF6
Number of Errors: 2 / Value: 0xF5
Number of Errors: 2 / Value: 0xF4
Number of Errors: 2 / Value: 0xF3
Number of Errors: 2 / Value: 0xF2
Number of Errors: 2 / Value: 0xF1
Number of Errors: 2 / Value: 0xF0
Number of Errors: 2 / Value: 0xEF
Number of Errors: 2 / Value: 0xEE
Number of Errors: 2 / Value: 0xED
Number of Errors: 2 / Value: 0xEC
Number of Errors: 2 / Value: 0xEB
Number of Errors: 2 / Value: 0xEA
Number of Errors: 2 / Value: 0xE9
Number of Errors: 2 / Value: 0xE8
Number of Errors: 2 / Value: 0xE7
Number of Errors: 2 / Value: 0xE6
Number of Errors: 2 / Value: 0xE5
Number of Errors: 2 / Value: 0xE4
Number of Errors: 2 / Value: 0xE3
Number of Errors: 2 / Value: 0xE2
Number of Errors: 2 / Value: 0xE1
Number of Errors: 2 / Value: 0xE0
Number of Errors: 2 / Value: 0xDF
Number of Errors: 2 / Value: 0xDE
Number of Errors: 2 / Value: 0xDD
Number of Errors: 2 / Value: 0xDC
Number of Errors: 2 / Value: 0xDB
Number of Errors: 2 / Value: 0xDA
Number of Errors: 2 / Value: 0xD9
Number of Errors: 2 / Value: 0xD8
Number of Errors: 2 / Value: 0xD7
Number of Errors: 2 / Value: 0xD6
Number of Errors: 2 / Value: 0xD5
Number of Errors: 2 / Value: 0xD4
Number of Errors: 2 / Value: 0xD3
Number of Errors: 2 / Value: 0xD2
Number of Errors: 2 / Value: 0xD1
Number of Errors: 2 / Value: 0xD0
Number of Errors: 2 / Value: 0xCF
Number of Errors: 2 / Value: 0xCE
Number of Errors: 2 / Value: 0xFF <= Error
Number of Errors: 3 / Value: 0x00 <= Error
Number of Errors: 4 / Value: 0x01
Number of Errors: 4 / Value: 0x02
Number of Errors: 4 / Value: 0x03
Number of Errors: 4 / Value: 0x04
Number of Errors: 4 / Value: 0x05
Number of Errors: 4 / Value: 0x06
Number of Errors: 4 / Value: 0x07
Number of Errors: 4 / Value: 0x08
Number of Errors: 4 / Value: 0x09
Number of Errors: 4 / Value: 0x0A
Number of Errors: 4 / Value: 0x0B
Number of Errors: 4 / Value: 0x0C
Number of Errors: 4 / Value: 0x0D
Number of Errors: 4 / Value: 0x0E
Number of Errors: 4 / Value: 0x0F
Number of Errors: 4 / Value: 0x10
Number of Errors: 4 / Value: 0x11
Number of Errors: 4 / Value: 0x12
Number of Errors: 4 / Value: 0x13
Number of Errors: 4 / Value: 0x14
Number of Errors: 4 / Value: 0x15
Number of Errors: 4 / Value: 0x16
Number of Errors: 4 / Value: 0x17
Number of Errors: 4 / Value: 0x18
Number of Errors: 4 / Value: 0x19

There are 4 errors in this case. The last error is where the program is restarted (value is 0x00). From this point on, I haven't seen an error yet. Please let me know what you're seeing on your side.

nyx commented 8 years ago

I attached a file showing my sermon output running the code in issue-1 branch. Unfortunately I see plenty of errors (and they do not go away upon reset as you see). The light stays on constantly with no blinking to indicate error. I will try this again with a GNU/Linux machine and if that works as you've had success.

sermon-output.txt

hannobraun commented 8 years ago

I attached a file showing my sermon output running the code in issue-1 branch.

Thank you!

Unfortunately I see plenty of errors (and they do not go away upon reset as you see). The light stays on constantly with no blinking to indicate error.

Too bad. There is still the possibility that the error detection code has a bug. I tested the blinking patterns, but since I don't know how to cause those errors, I obviously wasn't able to test the error detection. I reviewed that code again and couldn't find any relevant bugs (one irrelevevant one though; the error flags are never reset after an error occurs).

I will try this again with a GNU/Linux machine and if that works as you've had success.

Thanks, I'm looking forward to that. Please test all of the following combinations, if possible:

In the meantime, I did some semi-automated editing of your output file:

Here's the result: debug-output.txt

I find it interesting how the offset not only builds up as the number increases, it also goes down again as the number decreases. That leads me to believe that the offset just depends on the current value being sent, not on anything that was sent previously. We could try to confirm that later by writing random bytes instead of a specific sequence.

Edit: A part of my messages was garbled due to misuse of < and >. Should be fixed now.

nyx commented 7 years ago

I finally had a chance to put more time in here, and in this iteration I setup a Gentoo Linux VM in VMWare fusion, setup cross compiler using crossdev:

crossdev -S -v -t arm-none-eabi

I built blink and sermon using rustc 1.12.0-nightly (b30eff7ba 2016-08-05) and I've attached the issue-1 sermon output here: output.txt

It appears that I'm getting very similar behavior still. My next step planned is to run sermon on a Raspberry Pi 2 Model B so I can take the Apple Macbook Pro USB out of the picture.

hannobraun commented 7 years ago

Hi, thanks for checking back in, and sorry for the late reply. I've been away for a few days.

Interesting output. That regular error still seems to be there, but there are many more one-off errors than before. I guess that's not that surprising, as the VM still runs on the Mac, and the serial communication presumably still goes through the same codepaths (I'm not sure about that last part, as I'm not familiar with VMware Fusion). It will be interesting to see the output from the Raspberry Pi.

nyx commented 7 years ago

I tested connecting the DUE to the Raspberry Pi 2 B and running sermon there and see the same behavior. I found that I can send a fixed value repeatedly, and depending on the value will see consistent intermittent errors. So I think at this point I've ruled out the Mac variable.

nyx commented 7 years ago

Grasping at straws here about what might be afoot - are you running your board powered via USB or via a dedicated power supply?

nyx commented 7 years ago

I was experimenting with baud rate and upped it from 9600 to 115200 and it now appears that my serial transmission error is resolved. I still cannot explain this however which is unsatisfying.

hannobraun commented 7 years ago

Grasping at straws here about what might be afoot - are you running your board powered via USB or via a dedicated power supply?

My board is powered via USB.

I was experimenting with baud rate and upped it from 9600 to 115200 and it now appears that my serial transmission error is resolved. I still cannot explain this however which is unsatisfying.

That's very interesting. Given that you see the error in all your test scenarios and I don't, the most likely explanation seems to be that something's wrong with your Due. So far I've ruled that out, because you tested successfully with an Arduino sketch. Now that we know the baud rate is relevant, is it possible your Arduino sketch was communicating with 115200, or maybe another baud rate != 9600?

nyx commented 7 years ago

I verified the Arduino sketch baud rate was 9600.

On Aug 14, 2016, at 7:40 AM, Hanno Braun notifications@github.com wrote:

Grasping at straws here about what might be afoot - are you running your board powered via USB or via a dedicated power supply?

My board is powered via USB.

I was experimenting with baud rate and upped it from 9600 to 115200 and it now appears that my serial transmission error is resolved. I still cannot explain this however which is unsatisfying.

That's very interesting. Given that you see the error in all your test scenarios and I don't, the most likely explanation seems to be that something's wrong with your Due. So far I've ruled that out, because you tested successfully with an Arduino sketch. Now that we know the baud rate is relevant, is it possible your Arduino sketch was communicating with 115200, or maybe another baud rate != 9600?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

nyx commented 7 years ago

For the record I have two DUE I have been testing with, one is an official Arduino board and the other is not. I see the same behavior between them so I'm fairly certain it's not a hardware issue.

hannobraun commented 7 years ago

Oh man, no simple solutions here :)

One more thing I can think of: If you haven't already, can you try to compile and upload the program on the Raspberry Pi, cutting the Mac out completely? I'm not very hopeful that will change anything, but it might be worth a try.