tagyoureit / nodejs-poolController

An application to control pool equipment from various manufacturers.
GNU Affero General Public License v3.0
327 stars 97 forks source link

Intellitouch i10-3D + expansion #246

Closed ckurlinski closed 4 years ago

ckurlinski commented 4 years ago

I am not able to pull the expansion, ID (32) into njsp running capture replay. If I go into the indoor controller, under Advance, get image from outdoor device, njsp crashes, then restart njsp, the expansion is there.

Here is the replay.zip

replay.zip

tagyoureit commented 4 years ago

Hey, so I'm not getting any errors from the replay. I do have some logic in the app for the expansion panels. But there isn't enough info in here for me to debug anything.

  1. Does this replay include when your expansion panel is setup?
  2. Can you tell me what the error is when the app crashes? It should tell us what line is causing the crash. Also, can you tell me what the last packets are that were received? Since the app is crashing it may not capture these through the replay. You can turn on all logging and capture what is shown in the console log when everything goes south.

I'm pretty sure that I built the expansion panel logic from one of your previous captures. Unfortunately, I've only seen the i10X and it isn't 100% clear how to tell the different expansion panels from each other.

ckurlinski commented 4 years ago

Every time I run a replay capture, I loose the expansion. I have (20) circuits. It resets the config on the outdoor unit some how, and I have to reprogram the whole thing. Tested this yesterday afternoon again, after I started setting the new programming, just as a test.

The commands I use to program it are from the swagger ui page, and for the most part they work great. I think there is an overlap in the intellitouch board specification affecting the custom names; should start with 200, but starts with 201. So I can’t assign customs names with curl put, but I can work around that. Scheduling is not working either with curl

But aside from the replay wiping out the programming, it’s great, not being sarcastic about it. Programming it is a lot easier.

I know I’m an edge case so…. I can work around things. Thanks and keep up with the good work!!!

This is the replay capture that I took last, before it wiped my programming

This is the json config before the replay, but missing some of the curl put stuff like assigning the pumps to circuits, adjusting circuit names and such

On Nov 17, 2020, at 12:27 AM, tagyoureit notifications@github.com wrote:

Hey, so I'm not getting any errors from the replay. I do have some logic in the app for the expansion panels https://github.com/tagyoureit/nodejs-poolController/blob/next/controller/comms/messages/status/EquipmentStateMessage.ts#L226. But there isn't enough info in here for me to debug anything.

Does this replay include when your expansion panel is setup? Can you tell me what the error is when the app crashes? It should tell us what line is causing the crash. Also, can you tell me what the last packets are that were received? Since the app is crashing it may not capture these through the replay. You can turn on all logging and capture what is shown in the console log when everything goes south. I'm pretty sure that I built the expansion panel logic from one of your previous captures. Unfortunately, I've only seen the i10X and it isn't 100% clear how to tell the different expansion panels from each other.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tagyoureit/nodejs-poolController/issues/246#issuecomment-728695102, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4KGF7C2HJYPFM34PAZV6DSQICV5ANCNFSM4TWKQOUQ.

ckurlinski commented 4 years ago

Also, this is a master list of all the curl put json configs that work so far for me. I put them into a master base.json file to keep track of what’s working and what isn’t Everything in here except for the Custom names work

On Nov 17, 2020, at 12:27 AM, tagyoureit notifications@github.com wrote:

Hey, so I'm not getting any errors from the replay. I do have some logic in the app for the expansion panels https://github.com/tagyoureit/nodejs-poolController/blob/next/controller/comms/messages/status/EquipmentStateMessage.ts#L226. But there isn't enough info in here for me to debug anything.

Does this replay include when your expansion panel is setup? Can you tell me what the error is when the app crashes? It should tell us what line is causing the crash. Also, can you tell me what the last packets are that were received? Since the app is crashing it may not capture these through the replay. You can turn on all logging and capture what is shown in the console log when everything goes south. I'm pretty sure that I built the expansion panel logic from one of your previous captures. Unfortunately, I've only seen the i10X and it isn't 100% clear how to tell the different expansion panels from each other.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tagyoureit/nodejs-poolController/issues/246#issuecomment-728695102, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4KGF7C2HJYPFM34PAZV6DSQICV5ANCNFSM4TWKQOUQ.

tagyoureit commented 4 years ago

Also, this is a master list of all the curl put json configs that work so far for me.

Nothing was attached.

Let's break this down a bit. There is a lot to unpack here...

Every time I run a replay capture, I loose the expansion.

I really don't understand what's going on here. Running a replay capture is no different from starting the app up from scratch the first time. If you delete your ./data/*.json files does that also loose your expansion panel?

The commands I use to program it are from the swagger ui page, and for the most part they work great. I think there is an overlap in the intellitouch board specification affecting the custom names; should start with 200, but starts with 201. So I can’t assign customs names with curl put, but I can work around that.

Are you saying IntelliTouch starts at 201 and not 200?

Scheduling is not working either with curl

Need more details here.

If you can provide me with some more specifics that would really be helpful. I still need the exact output from the console when you say njsPC crashes. For this, if you enable all logging and also console logging it should output everything to your screen. The you can manually copy/paste the results back here.

ckurlinski commented 4 years ago

Replay Master Branch replay-2.zip replay-3.zip replay-4.zip replay-5.zip replay.zip

ckurlinski commented 4 years ago

logs.zip

ckurlinski commented 4 years ago

logs_no_expansion_master.zip

ckurlinski commented 4 years ago

This is from main branch, no expansion, indoor / get image from outdoor NJSP crashed

packetLog-get_out_to_in.log

ckurlinski commented 4 years ago

Main Branch, no expansion, indoor send image to outdoor No crashes 2020-11-17_21-47-50.zip

ckurlinski commented 4 years ago

Next Branch, no expansion, capture with reset

2020-11-17_21-58-56.zip

ckurlinski commented 4 years ago

Next branch, no expansion, indoor get image from outdoor, no crash!

2020-11-17_21-58-56_2.zip

ckurlinski commented 4 years ago

Next Branch, no expansion, indoor send image to outdoor, no crash.

2020-11-17_21-58-56_3.zip

tagyoureit commented 4 years ago

Hmm... I can't seem to find where the configuration has gone for the expansion panels. Here are the 2 (status) packets:

OLD CAPTURE WITH i10X
10,47,243,63,0,0,0,1,0,[96],64,0,0,0,64,82,32,0,82,0,0,0,0,4,0,60,103,1,5

NEW WITH i10X
13,28,32,12,0,0,0,0,0,[64],64,0,0,0,136,80,0,0,78,0,0,0,[4],4,0,163,72,1,5

NEW WITHOUT i10x
17,11,33,0,0,0,0,0,0,[64],64,0,0,0,113,81,0,0,78,0,0,0,0,4,0,163,72,1,5

The number in brackets previously told me the expansions. 96 vs 64... the 7th bit told the story. But with the new firmware it is consistently a 64. I've gone through the other packets and nothing is sticking out at me as indicating an expansion panel. I'll keep looking but a bit disappointed not to find it tonight.

EDIT: One thing to try is to capture anything that might start/be broadcast with the OCP boot sequence:

  1. Start njsPC. Let it finish doing it's sync thing.
  2. Turn off OCP
  3. send a PUT to njsPC (/app/logger/clearMessages) to clean out the logs
  4. Turn on OCP
  5. Let it finish it's cycle of boot up messages and syncing with the other ICP
  6. Repeat with the i10x card removed
rstrouse commented 4 years ago

Byte 22 is an unidentified byte on the 2 message for IntelliCenter. This byte is directly after waterSensor4. Perhaps this byte is not our indicator though. To solve that mystery, the exact process @tagyoureit outlined above needs to be completed where we have two separate log files that are clearly identified. One with the i10x installed and one without the i10x installed during the time njspc is trying to get the information from the OCP. Only then can we determine if another byte (perhaps on the action 5 message) contains the data we are looking for.

Below is the 2 message for IntelliCenter and below that is the 204 message which contains the expansion panels: image

This is a big message from IntelliCenter but byte 13 identifies the installed expansion modules for master and byte 14-18 identify which modules are installed on the expansion panels to include valve expansions and personality cards (i10x, i5x). image

ckurlinski commented 4 years ago

This is (4) sets of logs. After each test, added test(#) to each folder. This was done off a clean next install, no mods, no edits to the data pool json.

Test 1 is base system, nothing removed Test 2 is with out the i10X Test 3 is without the i10x and the valve expansion board inside the i10-3D Test 4 is with everything back up and running

All test were preformed after a stop and restart of njsp, using node dist/app.js

After all the tests, I stopped, and zipped the whole logs dir. This is what I'm posting.

Thanks

2020-11-24-logs.zip

rstrouse commented 4 years ago

@tagyoureit -- Since there is no 204 message from Touch I looked at the differences between the files provided for action 2. Here they are for the 4 tests. It appears there is no difference between them.

//#1 [255, 0, 255][165, 1, 15, 16, 2, 29][9,  3, 243, 61, 0, 0, 0, 3, 0, 64, 64, 0, 0, 0, 99, 78, 0, 0, 77, 0, 0, 0, 0, 4, 0, 163, 72, 1, 5][4, 150]
//#2 [255, 0, 255][165, 1, 15, 16, 2, 29][9, 29, 33,   0, 0, 0, 0, 0, 0, 64, 64, 0, 0, 0, 99, 64, 0, 0, 77, 0, 0, 0, 0, 4, 0, 163, 72, 1, 5][3, 144]
//#3 [255, 0, 255][165, 1, 15, 16, 2, 29][9, 33, 241,  0, 0, 0, 0, 2, 0, 64, 64, 0, 0, 0, 99, 79, 0, 0, 76, 0, 0, 0, 0, 4, 0, 163, 72, 1, 5][4, 116]
//#4 [255, 0, 255][165, 1, 15, 16, 2, 29][9, 44, 243, 60, 0, 0, 0, 3, 0, 64, 64, 0, 0, 0, 99, 78, 0, 0, 76, 0, 0, 0, 0, 4, 0, 163, 72, 1, 5][4, 189]

There is a difference for an action 109 which is identified as iLink. Here it is between the tests. Note the value in byte(1) for #1 and #4 when the i10x is installed. I don't see anything related to the valve expansion though. @ckurlinski -- do you have an iLink installed?

//#1 [255, 0, 255][165, 1, 15, 16, 109, 10][1, 15, 0, 0, 0, 0, 0, 0, 0, 0][1, 76]
//#2 [255, 0, 255][165, 1, 15, 16, 109, 10][1,  0, 0, 0, 0, 0, 0, 0, 0, 0][1, 61]
//#3 [255, 0, 255][165, 1, 15, 16, 109, 10][1,  0, 0, 0, 0, 0, 0, 0, 0, 0][1, 61]
//#4 [255, 0, 255][165, 1, 15, 16, 109, 10][1, 15, 0, 0, 0, 0, 0, 0, 0, 0][1, 76]
rstrouse commented 4 years ago

Wait a minute is byte(7) part of the features? If it isn't it's right there in action 2 byte(7).

ckurlinski commented 4 years ago

No ilink installed I will run the same tests tomorrow with the Indoor unit and the Spa Controller disconnected.

rstrouse commented 4 years ago

Please do me a favor for all of the tests make sure everything is turned off. The extra messages are noise that make it harder to sift through.

tagyoureit commented 4 years ago

Hmm, I was pretty sure that 109-111 was iLink. But if it's config info that could be different. But I've never seen these packets on any other system then @ckurlinski .

ckurlinski commented 4 years ago

Could 109, 110, 111 be the new Expansion addresses? Anyway, I will do the test without the scheduled items running. Thanks

ckurlinski commented 4 years ago

Dumb Question, could 109 be NJSP? It is showing up on the logs with nothing attached and in service mode

rstrouse commented 4 years ago

No, this is a broadcast message that is on the RS485 bus. The destination is 15 which tells me it is a panel 16 sending out information to whom it may concern. Bear in mind, not all messages follow this as a rule but most do. For IntelliTouch I believe all messages come from source 33 when njspc originates them.

I don't know how the expansion panels work in IntelliTouch so are you placing the master in service mode or can you place the expansion in service mode?

ckurlinski commented 4 years ago

The system is in service mode Process for this test; rm -rf data, mv logs ~/i10-3D_{test_type}, mkdir data mkdir logs, then start it up. Doing following tests: i10-3D_only, i10-3D_Valve_expansion, i10-3D_INDOOR_UNIT, i10-3D_i10x

ckurlinski commented 4 years ago

Here's an interesting set of logs Wiped / reset , i10-3D and i10x only, nothing else.

i10-3D_AFTER_UNIT_WIPE_i10x_2nd.zip i10-3D_AFTER_UNIT_WIPE_i10x.zip

ckurlinski commented 4 years ago

And after a clean njsp next install i10-3D_AFTER_UNIT_WIPE_i10x_CLEAN_NJSP_NEXT_INSTALL.zip

rstrouse commented 4 years ago

So you placed the panel in service mode. That will probably mess with our results. Still I see 109s with a 2 now in the first byte but this could be related to the fact that the master is disabling all incoming command related messages.

Is the process you are using to enable the expansion panel described here on page 26?
https://www.pentair.com/content/dam/extranet/pentair-pool/residential/manuals/automation/intellitouch/intellitouch-pool-control-system-users-guide-english.pdf

If it is I want a capture of you doing this process. I don't want you to set service mode, have anything turned or wipe the unit, just enable the i10x using the described process. Please also do not issue any commands from njspc or press any commands on any of your controllers. All of this just becomes noise that obscures the messages that we are interested in.

Please have njspc running then turn on the capture without reset before you start the process. Then when you are finished, wait about a minute, then turn off the capture and upload those files.

ckurlinski commented 4 years ago

Just did it, forgot to turn on packet capture trying again

ckurlinski commented 4 years ago

No Pumps, Valve Expansion Boards, no Chlorinators, no INDOOR Unit attached Just the i10-3D and the i10x

i10-3D_AFTER_UNIT_WIPE_i10x_CLEAN_NJSP_NEXT_WIPED_5_FUCKING_TIMES.zip

ckurlinski commented 4 years ago

With pumps and some config to get some things running

i10-3D_AFTER_UNIT_WIPE_i10x_CLEAN_NJSP_NEXT_WIPED_ADDED_PUMPS_BASIC_CONFIG.zip

rstrouse commented 4 years ago

Ok, just to be sure. The i10-3D_AFTER_UNIT_WIPE_i10x_CLEAN_NJSP_NEXT_WIPED_5_FUCKING_TIMES.zip was the process from page 26 of the manual doing a capture without reset or was this something else?

@tagyoureit - Once his system stabilizes, a 109 is sent on periodically and address 17 responds with an ack. 17 is a likely candidate for an expansion panel. 16 is the originating address and 17 responds. image

ckurlinski commented 4 years ago

Yeah but 109-11 was showing up on a fresh wiped system, with nothing but RPI w/NJSP connected to it....

i10-3D_AFTER_UNIT_WIPE.zip

rstrouse commented 4 years ago

Yeah but that was with the comm port connected to the expansion load center... correct?

ckurlinski commented 4 years ago

no, just the i10-3D

rstrouse commented 4 years ago

So without the load center wired in at all? What else was connected to RS485 at the time? The OCP is clearly communicating with something over RS485.

ckurlinski commented 4 years ago

Only (1) ONE thing was connected to the i10-3D I will do it again Including the reset with live capture

rstrouse commented 4 years ago

Please do not do a capture with reset. That causes njspc to interrogate your controller which sends hundreds of requests and responses. Do a capture without reset

ckurlinski commented 4 years ago

I meant resetting the i10-3D while doing a capture, else NJSP does this virtual controller thing

rstrouse commented 4 years ago

Ah yes.

rstrouse commented 4 years ago

After weeding through about 1,600 messages I cam across this sequence in your i10-3D_AFTER_UNIT_WIPE.zip it seems to indicate the i10-3D asking for control panel asking for response on 109. Notice there is no ACK and byte(24) of the 2 message is 0. image

Then later in the sequence it tries to ping 18 and 19 to see if there is anything there directly. The timing between 776 and 777 however was almost a minute so this tells me that the panel was taken off line then brought back online in that time. The state and config calls indicate that you refreshed -webClient after you restarted or it detected it and refreshed on its own.

In that time byte(24) switched to 4. Unfortunately, I have no idea what is located on byte(24) and the act of taking the panel offline and bringing it back up may simply cause it to delay. image

ckurlinski commented 4 years ago

Wiped like my dainty ass....

i10-3D_AFTER_UNIT_WIPE.zip

ckurlinski commented 4 years ago

sorry wrong log

ckurlinski commented 4 years ago

People were talking to me, how dare they

i10-3D_ONLY_RESET_WIPE.zip

rstrouse commented 4 years ago

Ok so here is what I know. On startup the OCP asks on address 109 who is out there. If no expansion panel exists then it gets no response. However, when an expansion panel is installed it responds with an ACK on the address of the expansion panel. This occurs during restart until it establishes that the control panel exists or it doesn't. If the expansion panel exists it continues to ping it on 109. The payload on that ping appears to be [1,x,x,...] where byte(0) indicates the expansion number and at least byte(1) indicates the relays that should be active. So when we were seeing [255, 0, 255][165, 1, 15, 16, 109, 10][1, 15, 0, 0, 0, 0, 0, 0, 0, 0][1, 76] it was telling EXP1 to activate relays 1-4.

During startup, it goes looking for any controller on action 110 which we do not process These addresses are 19, 18, and 17 working backwards to 17. If it already has confirmation for 17 on the 109 message then it doesn't ask for it on 110. That is the process that the OCP goes through to detect whether there is an expansion panel on the system. I suspect this is true for all IntelliTouch boards at reboot of the OCP.

Now what we need to know from the OCP is whether it has detected one or not. I suppose we could look for ACKs on 109 to determine if the expansion exists but there has to be a more direct route. For example, ACK 109 from address 17 = EXP1, ACK 109 from address 18 = EXP2, ACK 109 from address 19 = EXP3.

Do you remember if there is a screen on the ICP that displays the installed panels? If there is we may be able to spy on the ICP asking whether they exist.

ckurlinski commented 4 years ago

Zoom?

rstrouse commented 4 years ago

First do you have dashPanel set up? We will need that for the MessageManager tools.

ckurlinski commented 4 years ago

AllI have to does start it

rstrouse commented 4 years ago

Do you know if there is anywhere on the ICP that displays the installed expansion panels?

ckurlinski commented 4 years ago

It does, when the expansion panels are on the system, the ICP adds additional screens , Main Display, Display 1, Display 2, Features If the expansion is missing, then Display 2 isn't there

rstrouse commented 4 years ago

Hmmm... is this automatic or does it require you to do the get image from outdoor to have the Display 2 show up?