Write new profiles via serial - Anyone interested to collaborate ?

[duncanellison]

Hi all. Spent the last couple of slack days on a control interface for the T-962 based on the current serial interface in vb.net, I have a nice real time graphing interface going on (see attached pic). and I'm working on storing / retrieving the actual burn data for each batch of boards.

I can make the files available if anyone is interested in looking at / improving it.

This was a mod to a program I wrote to control the Beta V3 toaster oven controller, where I developed a way to read a text based config file and used it to set profiles in the controller.

Is anyone willing to collaborate at the NXP microcontroller end to develop a way to set new profiles into the EEPROM via the serial interface ? I've taken a look at the code and the it looks fairly straightforward, but it's too big a learning curve for me to get into at the moment.

[jieter]

Nice work, it's a pity your ui is written in vb.net...

[duncanellison]

Jieter, not sure why that's an issue ??? Happy to share the executable with you, I just need someone to handle the EEPROM work on the NXP side.

[jieter]

Some people use linux or mac exclusively...

On 25 March 2016 06:24:42 CET, duncanellison notifications@github.com wrote:

Jieter, not sure why that's an issue ??? Happy to share the executable with you, I just need someone to handle the EEPROM work on the NXP side.

---

You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/UnifiedEngineering/T-962-improvements/issues/85#issuecomment-201141581

[machineria]

duncanellison - it's look very good to me. I think it is good start to improving your application. Please share files -

[davidhooau]

duncanellison, I'd love to be able to use your logging/graphing application. Could you contact me at dhooke at gmail dot com, or post a link please? Thanks!

[radensb]

I am curious why this has not gotten more attention. Have there been any updates?

I particularly liked the idea of uploading custom profiles via the serial interface. I have noticed that the temperature profiles vary from board to board (as they should) and location to location within the drawer and thought about the idea of making a tool (or addition to an existing tool) that could generate a custom profile and uploading it. A user could then run a profile on a bare PCB with a thermocouple taped to it to see how that particular PCB design responds to a given profile. Then, the tool could adjust the profile so that the temperature curve more closely matches the expected profile. This adjusted profile could then be uploaded to the EEPROM and executed on a loaded board.

I think people underestimate the importance of adjusting profiles on a board to board basis. It can make all the difference. Now, generating that adjusted profile and making it easy to upload would be a huge advantage.

[GitLang]

I'm not sure if Duncan is still here. He freely shared the executable, but never seemed too keen on sharing the source. I, for one, would happily extend it and link in with the NXP for profile work. Same as Jieter though, I would want to something other than VB first.

Are you here Duncan?. If so, how about sharing the source to you nice utility please?

Personally I would put a couple of other improvements higher on the list.

1) Replace the rear cooling fan with a good quality 12VDC model and make a small interface board if necessary so that it can be smoothly speed controlled over the whole speed range. That would give us better temperature control.

2) Fit an air circulation fan, maybe something directly out of a kitchen fan oven, to get better heat distribution and temperature control.

3. Fit ancillary heater(s). At the moment the rate of change of temperature is not good on the T962A and I believe it is even worse on the small oven. This needs some caereful thought though. I wonder if the best solution would be a powerful temperature controlled air flow generator?. I think everything in the oven could then quickly obtain and follow the temperature of that airflow. Radiated heat is not the answer as different surfaces will obtain different temperatures depending on the reflectivity, conductivity, distance from the radiation source, etc, etc.

Just thinking out loud...

[duncanellison]

@GitLang Just picked this up. I've moved away from the project that gave the initial motivation to do this controller software. Add in a move from South Africa to the UK and a current deign project for a client means my ability to support this code is non-existent at the moment.

I'm very grateful to this project for the amazing firmware and hardware that's been shared, so I have no issue sharing the source files with you, but the reason I did not initially share them was because they do include licensable components (.net controls) and I don't think anyone else will be able to get the solution to build without the required license, but you are welcome to use the code as-is.

I don't think it's a realistic proposition to recode this outside the Microsoft .net framework as it relies heavily on Windows forms to do the graphics, but you are welcome to try.

I've put it up on GitHub at https://github.com/duncanellison/Reflow-Controller-T-962-

Note the final '-', not sure how that got there.

It's a VB.net V4.0 project and I did just manage to get it to build and run, however I don't have my oven with me at the moment so I can't test if the code all still works.

Duncan

[GitLang]

Hi Duncan, good to hear from you! Hope you're settling in the UK ok. I'd lost the impetus until recently when I picked up my active filter triamp loudspeakers and I've had several pcb's to do.

No, I realise moving a .NET outside .NET is close to impossible, my thought was to see if it would move over to C# which is more akin to the sort of langyage most programmers are used to, compared to VB. I've downloaded the project and I'll have a look. Many thanks for the contribution.

[duncanellison]

No problem.

VB.Net and C#.Net are pretty much the same thing (LOL) at least under the covers. You can probably just throw most of it though the Telerik converter and get a workable result.

I’m coming up to 45 years coding experience soon (still not so many can say that yet) having coded in Assembler, COBOL, Pascal, Fortran, C, C++, and regular C, but I choose VB.net as my weapon of choice as these days I prefer to spend my doing coding not looking for missing semicolons 😉

I can’t remember which of the form objects rely on the third party controls, it might just be the progress bars.

Good luck !

From: GitLang notifications@github.com Sent: 03 March 2018 22:24 To: UnifiedEngineering/T-962-improvements T-962-improvements@noreply.github.com Cc: duncanellison duncan_ellison@tesco.net; Author author@noreply.github.com Subject: Re: [UnifiedEngineering/T-962-improvements] Write new profiles via serial - Anyone interested to collaborate ? (#85)

Hi Duncan, good to hear from you! Hope you're settling in the UK ok. I'd lost the impetus until recently when I picked up my active filter triamp loudspeakers and I've had several pcb's to do.

No, I realise moving a .NET outside .NET is close to impossible, my thought was to see if it would move over to C# which is more akin to the sort of langyage most programmers are used to, compared to VB. I've downloaded the project and I'll have a look. Many thanks for the contribution.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/UnifiedEngineering/T-962-improvements/issues/85#issuecomment-370184701 , or mute the thread https://github.com/notifications/unsubscribe-auth/ABmot7JDCLnP-wkp4M_P51qMt3Zfd8XVks5taxgAgaJpZM4H4W-l . https://github.com/notifications/beacon/ABmotz1ZKFcEDP1K8scw6In_tYw44ukhks5taxgAgaJpZM4H4W-l.gif

[radensb]

I have implemented a new serial command that allows a profile to be uploaded to one of the two custom profile spaces and saved to the EEPROM. As soon as I figure out how to compile this, I will test. set customProfile <ID> <48 16-bit temp values> where: ID = 1 or 2 Temp values are single spaced delimited uint16 values

IE: set customProfile 1 50 50 50 60 73 86 100 113 126 140 143 147 150 154 157 161 164 168 171 175 179 183 195 207 215 207 195 183 168 154 140 125 111 97 82 68 54 0 0 0 0 0 0 0 0 0 0 0

Having a program generate the command based on user or test data input would be awesome. I was thinking if rewriting the above application, but now we have the source. I think that a good start would be a C# .NET application as suggested. Finding a nice (free) graphing interface would be nice.

[GitLang]

Hi Duncan.

Similar age and experience I guess. Started with PDP8 assembler, then 8080 and TMS9900. Went on to Fortran, Algol (superb language), also Pascal, Oberon, C, (C++ - hate it) etc. Most of my life in industrial control. Now retired. My choice these days to bash out a quick app is CBuilder for a quick GUI or Pelles C for command line.

Yes, it converted just fine with SharpDevelop 4.4 (Export as... is missing in later versions). Loading it in VS, there is just one error : "Error 3 'ComponentModel' is ambiguous, imported from the namespaces or types 'System, System.Windows.Forms'. " and two warnings about the missing DevComponents.DotNetBar2.

The error should not be hard to sort out, and I'll have a look for a trial of the DevExpress components. I'd like to get it fully built and working before seeing if something can replace the DevExpress component.

[radensb]

I have successfully tested the the new serial command and can now set an EEPROM profile from the serial interface. However, it only works sometimes. When the entire command is captured, it operates as expected, but there seems to be an intermittent timing issue that prevents the command from being captured properly on occasion. I confirmed this by printing the captured command in the event the code didn't understand it and can see it had been corrupted. There are missing bytes, corrupt bytes, and bytes that got concatenated with other bytes to generate incorrect values. For example: set EEprofil 50 50 16355 0 0 0 2 ... The above illustrates the 'e' missing in the word "profile", the profile ID is missing and one value is way to high. Sometimes, I get a command string that satisfies the sscanf() in the code, but only has a few of the temp values so the resulting profile that gets loaded is only 25% complete.

I think the problem is in the serial code. I read that the 1-wire interface is a bit-bang implementation that disables interrupts to maintain timing and I am using four MAX31850K's. Perhaps there is a conflict when I send my long command (could be up to 210 bytes or so of ascii) and the 1-wire interface is active that prevents the serial interface from working properly? The Rx buffer is set to 256 bytes, which is more than enough to capture my command in its entirety, so I'm not sure if that is what is happening here yet.

[GitLang]

What handshaking are you using?.

[radensb]

There is no handshaking. All the serial commands are a best effort communication. The set EEprofile command I implemented prints out the data it wrote to the EEPROM if the command was recognized. This is my verification. My thought was that a tool that sends the command would wait for this response. If it didn't get a response, it would assume that the command was corrupted beyond recognition and try again. If the command was recognized, but the data reported as written was not the same as the data it sent, it would send the command again. IE: set EEprofile 2 22 35 50 65 80 95 110 125 140 155 155 155 165 165 165 165 165 165 170 185 200 215 230 245 248 245 230 215 200 185 170 155 140 125 110 95 80 65 0 0 0 0 0 0 0 0 0 0 Setting EE profile 2: 22, 35, 50, 65, 80, 95, 110, 125, 140, 155, 155, 155, 165, 165, 165, 165, 165, 165, 170, 185, 200, 215, 230, 245, 248, 245, 230, 215, 200, 185, 170, 155, 140, 125, 110, 95, 80, 65, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, This would be viewed as a success.

set EEprofile 2 50 50 50 60 73 86 100 113 126 140 143 147 150 154 157 161 164 168 171 175 179 183 195 207 215 207 195 183 168 154 140 125 111 97 82 68 54 0 0 0 0 0 0 0 0 0 0 0 Cannot understand command: 154 157 161 164 168 171 175 179 183 195 207 215 207 195 183 168 154 140 125 111 97 82 68 54 0 0 0 0 0 0 0 0 0 0 0 This would be a failure as the command was not recognized.

set EEprofile 2 50 50 50 60 73 86 100 113 126 140 143 147 150 154 157 161 164 168 171 175 179 183 195 207 215 207 195 183 168 154 140 125 111 97 82 68 54 0 0 0 0 0 0 0 0 0 0 0 Setting EE profile 2: 6, 100, 113, 126, 14, 150, 154, 157, 161, 164, 168, 171, 175, 179, 183, 195, 207, 215, 207, 195, 183, 168, 154, 140, 125, 111, 97, 82, 68, 54, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, This would also be a failure as the data written was not what was sent.

I have re-written the logic in the uart_readline() to better manage the Rx buffer and flush it when appropriate to ensure that there wasn't any unwanted data in it before the next command was sent. Its return value is based on what it interprets so I can control the program flow based on that. My modifications seem to have improved the success rate, although it is still not 100%. I still think there are some interrupt things to look at. I don't see this problem with any other command, but my command is by far the longest as it has to send all the desired temp values, so that is why I think interrupts and buffer management might be playing a part.

[GitLang]

If the size is causing problems then break it into chunks. A standard way of dealing with this would be split it into, say, 8 chunks. Add a CRC on the end of each chunk before Tx. When you Rx it, if the CRC doesn't agree then discard the data and request that chunk again. Eventually you get all the chunks. Pu a grand CRC on the end of the full data block. If the Rx grand CRC fails, then send the whole block again, in the same chuck by chunk method. It's the way most data streams work, and should give perfect reliability in very noisy or broken conditions. As the noise increase, the correctly received data rate goes down, but at least you know that data is 100% correct.

[radensb]

I don't think the size is the problem, as it usually works. But I think that the size is revealing an issue with longer commands. I have thought about ways to mitigate this with CRC's, block transmissions, etc, but it seems silly to go though all that effort for a sub 256 byte transmission over a cabled UART to an ARM 7 controller. :)

[wulfmans]

hardware flow control

On 3/5/2018 8:49 PM, radensb wrote:

I don't think the size is the problem, as it usually works. But I think that the size is revealing an issue with longer commands. I have thought about ways to mitigate this with CRC's, block transmissions, etc, but it seems silly to go though all that effort for a sub 256 byte transmission over a cabled UART to an ARM 7 controller. :)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/UnifiedEngineering/T-962-improvements/issues/85#issuecomment-370653362, or mute the thread https://github.com/notifications/unsubscribe-auth/ACza7Yxg0dDj4wJEzyZPx6qcfwRcTLdhks5tbgdAgaJpZM4H4W-l.

[GitLang]

I thought you just said it was a size problem?. If you don't want to protect data blocks with CRC, and you don't want to use flow control, then just how do you expect to get a reliable connection in a real time interrupt laden system. If you give serial transmission priority, the oven control will bugger up, and if you give the oven control priority then serial comms will bugger up. It's what CRC protected blocks and/or handshaking is FOR. WHy one earth won't you use it?

[radensb]

If you re-read my post you will see that what I said was that

the size is reveling an issue with longer commands

Not that it is the issue. The fact that this is an interrupt laden system actually helps. The issue was that the interrupt is not being serviced, not that there is too much data being sent. If the interrupt was serviced every time, there would be no problems. I have already confirmed this by disabeling the 1-wire interface. When the 1-wire task is executed, it globally suspends interrupts while it bit-bangs the bus. It only takes 1.4ms to fill the 16 byte UART Rx FIFO at 115200 bps. With interrupts disabled during the 1-wire task the UART ISR does not execute and data being sent at that time is eventually lost due to FIFO overflow.

1. Hardware flow control would solve this, but is not available on this UART.
2. Software flow control is too slow to prevent data loss with the 16 byte FIFO.
3. Breaking the command into CRC protected blocks, while possible, breaks the concept of a human usable serial interface and forces the use of a program tool that can break up the command, calculate and send the CRC's, respond to NACK corrupt transmissions by re-sending blocks, ect. This also breaks the KISS model that the rest of the serial command interface is using. That is why I am not a fan in this case but am still considering it.

Maybe it would be good to design certain commands (like this one) around using a program to send them in a certain, safe way? Things to think about.

That being said, I have actually already gotten this to work properly by modifying the 1-wire interface to enable interrupts between 1-wire transfers and inserting a small delay before disabling interrupts again. These short interrupt enabled windows don't affect the timing requirements of the transfers but allow the UART IRQ to execute and capture data in the FIFO before an overflow condition. I will be testing this weekend to see that those changes don't have any adverse affect on the operation of the oven, but sitting here watching it, all the temps seem to be updating just as they did in the original 0.5.2 source I build.

[radensb]

Ok, so I have achieved 100% reliable servicing of the UART FIFO and am no longer losing bytes. I can now send my set EEprofile command and see the profile in CUSTOM#1 and CUSTOM#2 update as expected.

However, I am still hesitant to leave it as is and am thinking of going a step further utilizing the aforementioned concepts of breaking up the commands and sending CRC's protected chunks. While interleaving IRQ enabled windows in the 1-wire proved to be successful, it really clutters up the code and injecting delays is less than a polished solution in my mind. So, I am thinking of developing the concept of a binary based "Advanced Serial Command Set" that would be designed to be utilized by software tools only. It would be in addition to the currently defined human usable ASCII "Basic Serial Command Set" as I still see value in that interface.

The Advanced Command Set would be sent in chunks of < 16 bytes so that the hardware FIFO is never at risk of being overflowed. Once each chunk is received and its CRC checks out, it sends an ACK byte back informing the software tool that is sending these command chunks that it can proceed with sending the next chunk. If there is no ACK, the chunk is resent. Thoughts?

My initial concept for this Advanced Command Set would be as follows: (In HEX)

FF 55 <S1><S2><CMD><ID><CRC>
<S2><Dh><Dl><Dh><Dl><Dh><Dl>...<CRC>
<S2><Dh><Dl><Dh><Dl><Dh><Dl>...<CRC>
...
<S2><Dh><Dl><Dh><Dl><Dh><Dl>...<CRC>

Where:

• FF 55 is a sync pattern
• S1 is the size of the entire command (including overhead)
• S2 is the size of the current block(chunk) untill the CRC byte
• CMD is the command being sent
• ID is the profile this command will be changing
• CRC is the CRC for that block

So the result for this command to set CUSTOM#1 would be something like:

FF 55 76 02 EE 01 <CRC>
0C<Dh><Dl><Dh><Dl><Dh><Dl>...<CRC>
0C<Dh><Dl><Dh><Dl><Dh><Dl>...<CRC>
...
0C<Dh><Dl><Dh><Dl><Dh><Dl>...<CRC>

[mikeanton]

You shouldn't need to insert delays. If there is a pending interrupt, it will fire immediately once interrupts are enabled again. You don't need to wait for this to happen, so just periodically enable interrupts when having a delay at that point won't be critical.

[radensb]

Actually, I did experience data loss without the delays. It was minimal (2 at most bytes I think), but any data loss results in an unrecoverable situation and the ASCII command failed. Adding the delays corrected that as it gave enough time for the ISR to run multiple times to manage the FIFO enough so that it didn't overflow. Pending interrupts do get serviced but the processing speed of the code is much, much greater than the receive rate of UART bytes, thus interrupts would get disabled again too quickly for the FIFO to be depleted to a level that was safe before disabling interrupts again and too many bytes would show up in the disabled window causing overflow.

[mikeanton]

The interrupt routine should completely empty the FIFO before exiting, otherwise, you will end up with the problem you have seen. Also, if the FIFO is not empty, this results in a high interrupt rate, which is not desirable. One of the purposes of the FIFO is to help reduce the interrupt rate, which is why there is a trigger level before the interrupt is generated. I suspect if this was implemented, you wouldn't need the delays.

Now, I haven't looked at the code, but based on the symptoms you are describing, I suspect that only one byte is pulled out of the interrupt per pass, which is certainly less than ideal.

[radensb]

Nope. The ISR only pulls one byte from the FIFO and places it in the Rx ring buffer on each call and the threshold is set to 0 so the ISR gets called every time a byte shows up. The reason for this is that the code is looking for a '\n' character to terminate a potential command. If the '\n' is trapped in the FIFO because the threshold was not reached, it would prevent the command from executing unless more (unnecessary) bytes were sent.

In reality, this isn't that big of a deal. True, there are more interrupt calls, but they execute quickly and the time the code is dealing with Rx interrupts in the grand scheme of things is few and far between.

[mikeanton]

Well, it is ok that the trigger level is one, though there would be other ways to handle that. But the interrupt routine really should empty the buffer when called. Why introduce all of the extra overhead of calling the service routine again, when you are already there? As you have discovered, this does actually cause problems in your case, which could be avoided entirely. The bottom line is you need to deal with the data coming in, and the sooner you can do this, the less likely the chances of having an overrun are. If the buffer is empty, you can have interrupts disabled for a longer period, than if some bytes were left remaining in the buffer. It doesn't make sense to me to have to put in delays to waste time, in order to work around what amounts to an interrupt handler problem. You are in essence forcing it to do exactly the same function, through a workaround involving delays which allows the FIFO to empty, rather than just emptying it when you know there is data present.

[radensb]

I hear ya. I have been looking into the interrupts more closely. It looks like the threshold can be changed to something other than 1 and sub threshold receives can still be services. There is the RDA interrupt that triggers then the threshold has been reached (which is what the code has used exclusively). There is also a second level CTI (Character Time-Out Indicator) interrupt that triggers after 3.5-4.5 character times of no activity in the FIFO. So I have modified the ISR and added a compile time option so that if a threshold option other than 1 is selected, it will generate the code to loop the appropriate number of times in the ISR to empty the FIFO into the ring buffer. If there are remaining bytes due to not being a multiple of the threshold size, the CTI interrupt eventually triggers which causes the ISR to pull the bytes out one at a time. There is not FIFO size indicator but this would significantly reduce the ISR calls, however, enabling interrupt windows in the onewire interface is still necessary.

With this modification implemented and enabling interrupts followed by immediately disabling interrupts between onewire transfers has yielded a near 100% success rate in my tests. I have tried all the options available (1, 4, 8, and 14 byte thresholds) with near the same success rate (I think 14 was the worst). However, there are still failures every so often. I think there is a trade-off happening here where a higher threshold yields more ISR efficiency when harvesting multiple bytes in one go due to RDA interrupt, but we endure higher penalty when we drop back to single byte per ISR call for a greater number of bytes (putting us back in the initial situation fir the last set of FIFO accesses) when the ISR is responding to CTI. It all depends on when the command was sent in relation to when the onewire task is scheduled to run.

I think this path works if delays are re-inserted in the proper places. Being that the ISR is more efficient (most of the time) perhaps I could get away with shorter delays? I am basically guessing and assuming that any point in the onewire interface is equally susceptible to causing this issue, so I put the delays after every instance of interrupt enabling. Its not really an idea I like. If I had a scope and a spare pin, I could toggle it and get an idea of task time and ISR times....

[mikeanton]

I think you are making this more complicated than it needs to be. Regardless of the trigger level, always pull all bytes out of the FIFO. Just because the trigger level is set to one, does not mean there is only one byte in the FIFO, especially if interrupts were disabled at some point during receive. After all, why would you want to leave any bytes in the FIFO? What I do in my driver code, is remove bytes as long as the RDR flag is set, so this cleans out the FIFO without knowing how many bytes are there. More bytes could be received while you are servicing the interrupt too, and this handles that case as well. In fact, I also have a while loop around the entire service routine that loops until all flags have been cleared in the ISR.

When I was writing the UART handlers for the LPC214x series, it took me a long time to get them to be reliable, and to avoid spurious interrupts, which are also a problem in that series, so I feel your pain...

[GitLang]

I think you are making this more complicated than it needs to be

Fully agree with that. In all my career I have never handled this sort of problem differently to what Mike says. FIFO nearly full generates the interrupt, the interrupt routine empties the FIFO. If you have a very slow or sporadic (manual) data input then the interrupt can be generated by a timer started at the end of the interrupt service routine. That way, the FIFO gets emptied when nearly full, or periodically, depending on data rate. But still with CRCs...

[mikeanton]

Or, when doing a circular buffer read, and finding it empty, check the FIFO, and if there is data there, kick the interrupt to read it out into the buffer. Since the buffer is usually polled at some point, the FIFO can be polled at this point too to grab data trapped in the FIFO, with no additional interrupt required.

[radensb]

Thanks for the tips guys! The added ISR gymnastics I mentioned above was done because I didn't realize that you could just check the RDA bit in a loop. I thought that ack'ing the interrupt was required to update it and the data in the U0RBR (data) register. The datasheet makes it seem like your options are to do byte/block transfers via the threshold levels and/or using the CTI interrupt and the original code was written to save the status of the register in a variable and check if the variable was equal to the RDA interrupt status.

However, I tried this by buffering in the ISR while the RDA bit was set:

while (U0IIR & 0b00000100) {
//Don't block, as we are inside an interrupt!
add_to_circ_buf(&rxbuf, U0RBR, 0);

and it looks to be working as intended! Much more efficient and simple.

I am now optimizing the onewire code to keep disabled interrupts tightly focused only on the blocks that NEED to be uninterrupted to minimize the blackout window lengths and open up IRQ service windows where previously there were none during the onewire task. I am hopeful that this will lead to resolution on this issue.

[lolsborn]

Instead of reinventing the wheel on the front end, could we just use http://ospid.com/blog/guides/frontend-software/

[zian31]

Do you know where (at which addresses from 10000000 to 10FFF000) the waves values (48 x 16 bits values for 7 waves / or 9 waves ?) are in the .hex program file ? Because I've supposed that maybe this program .hex file is also located in the EEPROM (as the custom waves you want to program in the EEPROM from serial) ? I mean the .hex file which begins with : :100000002F0000EA24F0A0E340F0A0E32CF0A0E3EE :1000100034F0A0E30000A0E1F0FF1FE5BA00A0E388 :10002000070000EAA800A0E3050000EAAE00A0E394 :10003000030000EAB400A0E304E04EE2000000EA9E