PKTC Config Hash

[agentjdh]

Hi

I am using the 0.9.9-dev Docsis tool to generate a working MTA. I've used this in a past with a variety of modems. There is one modem that will not work though.

I've tracked the problem down to the OID used for pktcMtaDevProvConfigHash.0

It looks like there is more than just EU/NA - IETF as well? Anyway, in docsis.c, the relevant parts for generating the OID are -

NA (Hash == 1) - This corresponds with OID .1.3.6.1.4.1.4491.2.2.1.1.2.7.0 (pktcMtaDevProvConfigHash.0 from PKTC-MTA-MIB)

{
    memcpy (tlvbuf + tlvbuflen - 3, "\x0b\x28\x30\x26\x06\x0e\x2b\x06\x01\x04\x01\xa3\x0b\x02\x02\x01\x01\x02\x07\x00\x04\x14", 22);
    tlvbuflen += 19;
  }

EU (Hash == 2) - This corresponds with OID .1.3.6.1.4.1.7432.1.1.2.9.0 - I believe this is an Excentis MIB

{
    memcpy (tlvbuf + tlvbuflen - 3, "\x0b\x26\x30\x24\x06\x0c\x2b\x06\x01\x04\x01\xba\x08\x01\x01\x02\x09\x00\x04\x14", 20);
    tlvbuflen += 17;
  }

There appears to be a 3rd method. The OID is

.1.3.6.1.2.1.140.1.2.11.0

It is pktcMtaDevProvConfigHash.0 from PKTC-IETF-MTA-MIB. I need to use this OID for the modem I'm working on at the moment.

I have used OID converter from

[]((http://www.rtner.de/software/oid.html)

This generated

06 0C 01 03 06 01 02 01 81 0C 01 02 0B 00

Which is getting there, but there is a prefix that changes - the suffix looks like it is always \x04\x14

How is the string generated?

[stephb9959]

Hi @agentjdh,

I do not have the same device you have. Let me explain what is going on in the code:

That magical string you see in memcpy() is a TLV (Tag-Length-Value). For CableLabs, this is TLV 11, which is just a normal SNMP Object embedded in a config file.

So in order to do your new hash, you need to start with, followed by the total length of the TLV, which is the OID and the hash itself plus some glue bytes here and there. As you say, the encoded OID is 06 0C 01 03 06 01 02 01 81 0C 01 02 0B 00 ( but that is not ASN1 as you will see). The suffix of 0x04 0x20 is just to tell that the following field is bitsring, length of 20 bytes, which is the length of a SHA1 hash.

So if we put this all together, your magical string should be:

0x0b - the TLV type, 0x25 - the full length of everything following: 2+35=37=0x25 0x30 - The following stuff is an OID 0x23 - The full length of the OID and its value: 1+1+11+1+1+20 = 35 = 0x23 0x06 - This indicates that the following is an OID 0x0b - Length of the OID 0x2b 0x06 0x01 0x02 0x01 0x81 0x0C 0x01 0x02 0x0B 0x00 - ( encoded OID, see below) ** 0x04 - What comes next is a bitstring 0x14 - That's 20 meaning that there are 20 bytes following because this is the length of a SHA1 hash

So the total number of bytes we have here is: 1+1+1+1+1+1+11+1+1=19

Se we can add a new hash, and the code should look like this:

if (hash == 3) 
 {
    memcpy (tlvbuf + tlvbuflen - 3, "\x0b\x25\x30\x23\x06\x0b\0x2b\0x06\0x01\0x02\0x01\0x81 \0x0C\0x01\0x02\0x0B\0x00\x04\x14", 19);
    tlvbuflen += 16;
  }

Now you will need to add the following code in main() in docsis.c to let you use this "special" encoding just after the "-eu" hash, so around line 358 in docsis.c

if (!strcmp (argv[0], "-spechash")) {
      if (hash) {
        usage();
      }
      hash = 3;
      continue;
    }

Then just run the same command with -spechash to try this out...

So if you make these changes, you should get closer to get this to work. I might be off in the lengths ( I have a problem adding 1 to things for some reason).

I don't have time to compile this right now, so let me know how this goes.

Cheers...

*Encoded OID... For the encoded OID, it's not going to be what that site gave you. In their clever way, the ASN1/SNMP folks say that an OID is encoded by replacing the first 2 bytes with the following 40 Byte1 + Byte 2. So in this case, 40*1+3, that is 43, or 0x2b. Which means that your OID becomes: 0x2b 0x06 0x01 0x02 0x01 0x81 0x0C 0x01 0x02 0x0B 0x00

[stephb9959]

@agentjdh,

In the previous code, I inserted a space in the memcopy, please remove that in your code. This was a formatting mistake:

memcpy (tlvbuf + tlvbuflen - 3, "\x0b\x25\x30\x23\x06\x0b\0x2b\0x06\0x01\0x02\0x01\0x81\0x0C\0x01\0x02\0x0B\0x00\x04\x14", 19);

Thanks

[agentjdh]

@stephb9959

I'd come to the exact same numbers effectively from using xxd to take a hex dump of a working binary. So this part looks correct.

The actual Docsis tool, with your modifications, compiles OK, but using the -spechash option when converting the text to binary does not insert the Hash entry.

I got it (sort of) working by just modifying the existing -eu entry (hash ==2) - it works, but the generated binary is 1 byte larger than a working one, and the modem rejects the generated MTA file.

The working ones are created on a Windows PC using Thomson ConfigBldr.

[stephb9959]

Can you give the the working file and the one you generated that does not work?

On Tue, Oct 15, 2019 at 02:08 agentjdh notifications@github.com wrote:

@stephb9959 https://github.com/stephb9959

I'd come to the exact same numbers effectively from using xxd to take a hex dump of a working binary. So this part looks correct.

The actual Docsis tool, with your modifications, compiles OK, but using the -spechash option when converting the text to binary does not insert the Hash entry.

I got it (sort of) working by just modifying the existing -eu entry (hash ==2) - it works, but the generated binary is 1 byte larger than a working one, and the modem rejects the generated MTA file.

The working ones are created on a Windows PC using Thomson ConfigBldr.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/rlaager/docsis/issues/66?email_source=notifications&email_token=AAOF7RG2EJDLGXD2BM7KGCDQOWCCLA5CNFSM4I7WXYHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBIAUBY#issuecomment-542116359, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOF7RHBI3WLSZIS4QSWAC3QOWCCLANCNFSM4I7WXYHA .

-- Stéphane Bourque

[agentjdh]

mta_WORKING_999999.bin.gz mta_NOT_WORKING.bin.gz

[stephb9959]

Hi @agentjdh,

I built a little app to parse the files in erlang and here is the output:

Values not found but int good file:
>>> OID:".1.3.6.1.2.1.140.1.2.11.0" VALUE:<<4,20,251,207,112,173,178,112,146,
                                            205,125,193,57,6,115,73,213,223,77,
                                            198,13,230>>
>>> OID:".1.3.6.1.4.1.4413.2.2.2.1.6.1.7.0" VALUE:<<2,1,2>>
>>> OID:".1.3.6.1.2.1.140.1.1.6.0" VALUE:<<2,1,1>>

Values not in good file but in bad:
>>> OID:".1.3.6.1.4.1.4491.2.2.1.1.1.7.0" VALUE:<<2,1,1>>
>>> OID:".1.3.6.1.4.1.406.20.2.10.1.2.0" VALUE:<<2,1,2>>
>>> OID:".1.3.6.1.2.1.140.1.2.11.0" VALUE:<<4,20,130,248,25,233,168,226,50,118,
                                            72,209,166,83,160,17,75,91,122,4,
                                            156,204>>

This shows that you have 2 mismatches OIDs. Forget the ".1.3.6.1.2.1.140.1.2.11.0" because that is the has and it should not match because the files do not match.

The top shows you what the good file has that the bad file does not. And the bottom shows you what the bad file contains that is not in the good file.

So I think you have OID problems.

Both files passed the MTA decoder so they are both valid.

Hope this helps.

[stephb9959]

Here is the proper docsis.c file that compiles and will do the spechash and still lets you pick from NA/EU too (I had forgotten one more location to enable to hash being appended, it's in now)...

docsis.zip

Let me know how it goes.

[agentjdh]

@stephb9959

I've replaced docsis.c with the one you provided and re-compiled. Looks like something is going wrong - generating a binary using

./docsis -dialplan -spechash -p mta_WORKING.txt mta_WORKING.bin

SnmpMibObject pktcMtaDevProvConfigHash.0 HexString 0x58df8e779cda95ca9a7ae0cb1589e775532ca691;
/* PC20 dialplan found, dialplan.txt file created. */

Error: bad type returned (31) SnmpMibObject zeroDotZero.120.50.98.0.120.48.54.0.120.48 �txDV Variable has bad type; MtaConfigDelimiter 255; }

Running the same command with -nohash or -eu switches works OK (but the modem rejects the binary as there is no/wrong has OID used).

[agentjdh]

I don't know how that code section got in above, ignore it.

[stephb9959]

@agentjdh

OK. I have it now. This was my mistake. Some debug stuff I added a while ago in docsis.c, now removed. Also generated some test to make sure the spechash could be encoded and decoded. Now that works. Also, my encoding was wrong with the memcpy(). Now fixed. ( It's amazing what happens when you actually compile and run tests! Who knew!!)

Let me know if this finally fixes your problem. If it does, then I can submit for a pull request.

This is the good file...

docsis.zip

Thanks

[agentjdh]

@stephb9959

I am not getting those errors now, but the generated file is still +1 byte, and is rejected by the modem. files.tar.gz

I've attached the files, here is what I'm doing -

Decode the working binary (8480 bytes, not the TEST one), which generates the text configuration + dialplan text file

./docsis -d mta-working-2-lines--128_021.bin > mta-working-2-lines--128_021-TEST.txt

Encode using the following command line -

./docsis -dialplan -spechash -p mta-working-2-lines--128_021-TEST.txt mta-working-2-lines--128_021-TEST.bin

Resulting binary is 8481 and is rejected for -

error - 4000950906 (table linkages in the configuration file could not be resolved) I get this twice, one for each line

Resulting in

critical - 4000950904 (mandatory parameter of the configuration file is missing)

The dialplan.txt generated by decoding working vs non-working is identical.

So is there something in the diaplan.txt file that is causing this? It could probably be simplified for a start, which I will work on this morning.

[stephb9959]

@agentjdh

There is a 1 byte difference but it's actually not that 5 bytes. The resulting file has 1 byte size difference but it's because the 4 mismatch OIDs have different sizes. You have 2 OIDs in the good file that are not in the non-working file, and you have 2 OIDs in the test file that are not in the good file. Here's that little analysis tool I ran on the working and non-working files:

Values in mta-working-2-lines--128_021.bin not in mta-working-2-lines--128_021-TEST.bin:
>>> OID:".1.3.6.1.2.1.140.1.2.11.0" VALUE:<<4,20,88,223,142,119,156,218,149,
                                            202,154,122,224,203,21,137,231,117,
                                            83,44,166,145>>
>>> OID:".1.3.6.1.4.1.4413.2.2.2.1.6.1.7.0" VALUE:<<2,1,2>>
>>> OID:".1.3.6.1.2.1.140.1.1.6.0" VALUE:<<2,1,1>>

Values in "mta-working-2-lines--128_021-TEST.bin" not in "mta-working-2-lines--128_021.bin":
>>> OID:".1.3.6.1.4.1.4491.2.2.1.1.1.7.0" VALUE:<<2,1,1>>
>>> OID:".1.3.6.1.4.1.406.20.2.10.1.2.0" VALUE:<<2,1,2>>
>>> OID:".1.3.6.1.2.1.140.1.2.11.0" VALUE:<<4,20,7,248,124,28,236,178,235,107,
                                            171,148,80,207,219,10,75,158,100,
                                            111,187,31>>

So I decided to run my own tests with the docsis.c file I gave you. All based on your working mta-working-2-lines--128_021.bin

docsis -d mta-working-2-lines--128_021.bin > good-bin.txt

Then I remove the last line at the end of good-bin.txt, "SnmpMibObject mib-2.140.1.2.11.0...". That's the hash OID and we do not need it. Then I ran:

docsis -dialplan -spechash -p good-bin.txt my-good-bin.bin

This gives me a file of 8480 bytes.

When I run my own tool to compare the 2 bins, I get something like this:

Values in mta-working-2-lines--128_021.bin not in my-good-bin.bin:
>>> OID:".1.3.6.1.2.1.140.1.2.11.0" VALUE:<<4,20,88,223,142,119,156,218,149,
                                            202,154,122,224,203,21,137,231,117,
                                            83,44,166,145>>

Values in "my-good-bin.bin" not in "mta-working-2-lines--128_021.bin":
>>> OID:".1.3.6.1.2.1.140.1.2.11.0" VALUE:<<4,20,157,12,6,164,190,146,223,49,
                                            23,4,12,252,156,39,95,180,174,203,
                                            255,240>>

Which tells me that the only difference is the hash. This is normal because the good file has the dialplan in the middle of the file, which the docsis tool puts the dialplan at the end of the file.

I am attaching the file I produced this way. Maybe you can try it and tell me if it works?

my-good-bin.zip

This is the good-bin.txt file I get too. Compare yours to this just to make sure that your setup is ok (i.e. you are not missing MIBs or you have old MIBS that would cause a problem. good-bin-txt.zip

Other than that, this shows that the right has is getting in the file and that the docsis decode and reencodes to the same file.

Let me know how this works.

Thanks

[agentjdh]

@stephb9959

What you did there is exactly what I was doing - decoding a working binary to text + dialplan, deleting the hash line from the text file, then rebuilding the binary with exactly those switches. And the resulting binary did not match the original.

So where are these mismatched OIDs coming from? I am not editing the file apart from Hash removal.

Were you doing this with a release source code? I was using binary-master download, and replacing only docsis.c

I'll clean out the sources and start again.

[stephb9959]

That’s what I did.

I don’t know what to say. Maybe recompile the whole project? Make sure you have all the right MIBs? Restart from a fresh directory to avoid any file conflicts?

On Fri, Oct 18, 2019 at 00:22 agentjdh notifications@github.com wrote:

@stephb9959 https://github.com/stephb9959

What you did there is exactly what I was doing - decoding a working binary to text + dialplan, deleting the hash line from the text file, then rebuilding the binary with exactly those switches. And the resulting binary did not match the

Were you doing this with a release source code? I was using binary-master download, and replacing only docsis.c

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/rlaager/docsis/issues/66?email_source=notifications&email_token=AAOF7RBDVGQ4JVOBDM7YCJDQPFP2PA5CNFSM4I7WXYHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBTCEHQ#issuecomment-543564318, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOF7RG7W6RIKBGF7J3KOVTQPFP2PANCNFSM4I7WXYHA .

-- Stéphane Bourque

[stephb9959]

@agentjdh

Just clone the project to get the master and replace with the new DOCSIS.c

[agentjdh]

@stephb9959

I've done exactly that, with the same results.

The bizarre thing is, once I have encoded the binary that is +1 byte ("wrong"), if I then decode this to a text file, and delete the hash, the resulting text file is identical to the one decode from the working binary.

I have a lot of mib files so I am never entirely sure that I have the correct ones.

Could you attach the mibs you have and/or your "working" binary?

[stephb9959]

I have a OS X build on Catalina. I could build one for Debian if needed. What do you need?

On Fri, Oct 18, 2019 at 01:02 agentjdh notifications@github.com wrote:

@stephb9959 https://github.com/stephb9959

I've done exactly that, with the same results.

The bizarre thing is, once I have encoded the binary that is +1 byte ("wrong"), if I then decode this to a text file, and delete the hash, the resulting text file is identical to the one decode from the working binary.

I have a lot of mib files so I am never entirely sure that I have the correct ones.

Could you attach the mibs you have and/or your "working" binary?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/rlaager/docsis/issues/66?email_source=notifications&email_token=AAOF7RFFEER57Z2XVAUNAVLQPFUPVA5CNFSM4I7WXYHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBTG2HI#issuecomment-543583517, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOF7RDUXCDFWC2GCMRMKYLQPFUPVANCNFSM4I7WXYHA .

-- Stéphane Bourque

[agentjdh]

Debian please

[stephb9959]

Here is a Debian build for you...

debian-docsis-buster.zip

[agentjdh]

@stephb9959

Thanks Stéphane, it looks like the file created is the same size (8481 bytes) as before. So I guess it is not the binary then.

Which pretty much leaves my mibs - I have a 3rd party mib folder, .snmp/mibs, with a lot of mibs I have collected over the last year or so.

If I move this folder, so the docsis application only uses the default installed mibs in /usr/share/snmp/mibs, I get another slightly different binary created this time 8483 bytes.

I've updated the system installed mibs with download-mibs. If I diff the text files decoded from the binaries it shows

7c7
<   SnmpMibObject .1.3.6.1.4.1.4491.2.2.1.1.1.7.0 Integer 1;
---
>   SnmpMibObject .1.3.6.1.2.1.140.1.1.6.0 Integer 1;

So something is still getting mixed up.

SnmpMibObject .1.3.6.1.4.1.4491.2.2.1.1.1.7.0 Integer 1;

Is not in the original binary - so how is it being inserted into the binary created by docsis tool?

Actually, it looks like it might be modifying an IETC MIB (which works) into the equivalent Cable Labs MIB - which is what I've been trying to avoid with the hash MIB.

I'll spend some more time on this tomorrow. Thanks for your help to date.

[agentjdh]

Turns out it was the decoding to text file that was using the "wrong" OID for pktcMtaDevEnabled

The particular modem I was using required the 1.3.6.1.2.1 OID from PKTC-IETF-MTA-MIB

It also obviously required the IETF hash OID as well, which is what triggered this whole scenario.

So just getting rid of the various MIBs - or using the -o option during decode the OIDs numerically worked.

Anyway working binaries are now generated from the decoded text output.

[stephb9959]

@agentjdh

That's very good news. I was just about to send you my MIBs...

So if you use the new executable with the -spechash all things work for you now?

Thanks

[agentjdh]

Yes, -spechash works properly. It is IETF spec, not NA or EU, to my knowledge.

This is the first device I have seen that uses IETF mibs.

[gezimpula]

In my case recompiling with -spechash or -eu it give me the same hexstring value and normaly MTA reject it since it accepts only IETF hashed.

Installation was made from your site and also replaced docsis.c as you provide on the link. -rw-r--r-- 1 root root 21572 Apr 30 13:04 docsis-backup.c -rw-r--r-- 1 root root 22016 Apr 30 13:25 docsis.c

First I'm decoding the working one docsis -o -d K_IP.BIN > K_IP.txt

Removing the last line from K_IP.txt and encoding again docsis -o dialplan -spechash -p K_IP.txt K_IP_spechash.bin

Hexstring(1.3.6.1.2.1.140.1.2.11.0) are not matched from original one (KIP.BIN) and compiled one (K_IP_spechash.bin) and in this case config is rejected .

Tried with EU encoding i get same value for hexstring ( 1.3.6.1.4.1.7432.1.1.2.9.0 ) docsis -o -dialplan -eu -p K_IP.txt K_IP_eu.bin

I dont think i should get same encoding values with eu and spechash which was supposed to be IETF .

Any idea what is wrong with my compiler would be very helpful.

files1.tar.gz