search

Ylianst / MeshCentral

A complete web-based remote monitoring and management web site. Once setup you can install agents and perform remote desktop session to devices on the local network or over the Internet.
https://meshcentral.com
Apache License 2.0
3.96k stars 536 forks source link

Option-15 passed by DHCP is not detected by AMT #3691

Open alexkolomolo opened 2 years ago

alexkolomolo commented 2 years ago

# System is on Centos 8.4, no sellinux, systemd #

Output of meshcmd AmtInfo Settings: {"action":"amtinfo","debuglevel":1} DHCP error, timeout Intel AMT v12.0.35, pre-provisioning state. Wired Enabled, DHCP, C4:00:AD:5E:B0:0C Connection Status: Unknown, CIRA: Disconnected. #

AMT Version Info: BIOS Version = A2.02 Flash = 12.0.35 Netstack = 12.0.35 AMTApps = 12.0.35 AMT = 12.0.35 Sku = 16392 (AMT, Corporate) VendorID = 8086 Build Number = 1427 Recovery Version = 12.0.35 Recovery Build Num = 1427 Legacy Mode = False

# Wireshark dump - I replaced domainname text with mydomain.com for this ticket Frame 20: 370 bytes on wire (2960 bits), 370 bytes captured (2960 bits) Ethernet II, Src: Advantec_5e:b0:0c (c4:00:ad:5e:b0:0c), Dst: Advantec_62:44:4f (c4:00:ad:62:44:4f) Internet Protocol Version 4, Src: 192.168.0.108, Dst: 192.168.0.1 Internet Control Message Protocol Dynamic Host Configuration Protocol (ACK) Message type: Boot Reply (2) Hardware type: Ethernet (0x01) Hardware address length: 6 Hops: 0 Transaction ID: 0x9bf40917 Seconds elapsed: 0 Bootp flags: 0x8000, Broadcast flag (Broadcast) Client IP address: 192.168.0.108 Your (client) IP address: 0.0.0.0 Next server IP address: 192.168.0.1 Relay agent IP address: 0.0.0.0 Client MAC address: Advantec_5e:b0:0c (c4:00:ad:5e:b0:0c) Client hardware address padding: 00000000000000000000 Server host name not given Boot file name not given Magic cookie: DHCP Option: (53) DHCP Message Type (ACK) Option: (54) DHCP Server Identifier (192.168.0.1) Option: (51) IP Address Lease Time Option: (1) Subnet Mask (255.255.255.0) Option: (28) Broadcast Address (192.168.0.255) Option: (15) Domain Name Length: 12 Domain Name: mydomain.com Option: (3) Router Option: (6) Domain Name Server Option: (255) End Padding: 000000000000 # Side Note If I program the DNS suffix in Mebx, it is displayed in AmtInfo but we require to set this without manual intervention so we need option 15 to work.

Ylianst commented 2 years ago

Bryan will have to look into this. MeshCMD has to try to figure out the DNS suffix for the Intel AMT network interfaces. In Windows it's easy since the OS has a DNS suffix for each network adapter, but Linux does not have this. So, MeshCMD has to attempt to query the DHCP server to get this information. Something must be going wrong.

alexkolomolo commented 2 years ago

Thanks so much! I really need to get this working.

BTW: Centos 8 is using Network Manager, that is quite common these days, and in-fact NetworkManager does have a separate DNS suffix per adapter, you can see this when using nmcli. However using DHCP is probably a more generic way across all Linux platforms.

I turned off firewalld this also did not help.

Is meshcmd compiled somehow from meshcmd.js in the meshCentral project?

Ylianst commented 2 years ago

Yes, the meshcmd executable is actually the MeshAgent merged with meshcmd.js. The server does the merging on-the-fly when you download it.

Note that Intel AMT probably detects your DHCP option 15 correctly, but MeshCMD can't get the DNS suffix for the network interface, so it can't signal to the server to activate using that DNS suffix. The server in turn does not know what activation certificate to use.

So, the problem is all in MeshCMD, not in AMT. I do have a trick for you. Then running MeshCMD, add this argument:

--dnssuffix sample.com

This will override the DNS suffix detection in MeshCMD and use the one you set. Let me know if that works.

alexkolomolo commented 2 years ago

Great! 1 Step forwards. I thought that was finally going to be success, but it still failed for some reason...

Setting up MEI... Started LMS... Starting Intel AMT configuration... Started APF tunnel... Checking Intel AMT state... Getting ready for ACM activation... Performing ACM activation... Failed to complete ACM activation (ERR/5).

Note: This server has freshly reset AMT chip, pre-provisioning state and I can provision this server if I place the dns suffix in mebx. We are definitely passing option 15 as per tcpdump and i've tried dhcp server from ubiquity and dnsmasq.

Here are the end of the logs in debug mode. APF: CHANNEL_DATA: "5E0000000500000753504F5354202F77736D616E20485454502F312E310D0A417574686F72697A6174696F6E3A2044696765737420757365726E616D653D2224244F7341646D696E222C7265616C6D3D224469676573743A3345443530303030303030303030303030303030303030303030303030303030222C6E6F6E63653D225647314E416130424141414141414141704D7174614A53632F537176466B4D7A222C7572693D222F77736D616E222C716F703D2261757468222C726573706F6E73653D223636393633663538376461313735326330393933656437373664363464363035222C6E633D2236222C636E6F6E63653D223566323832633363653838363238303465393661663634666463336666383436220D0A486F73743A206E6F64652F2F417741434141514142514141426741484141674143514D414167414541415541414159414277414941416B44414149414241414641414147414163414341414A3A31363939320D0A436F6E74656E742D4C656E6774683A20313439350D0A0D0A3C3F786D6C2076657273696F6E3D22312E302220656E636F64696E673D227574662D38223F3E3C456E76656C6F706520786D6C6E733A7873693D22687474703A2F2F7777772E77332E6F72672F323030312F584D4C536368656D612D696E7374616E63652220786D6C6E733A7873643D22687474703A2F2F7777772E77332E6F72672F323030312F584D4C536368656D612220786D6C6E733A613D22687474703A2F2F736368656D61732E786D6C736F61702E6F72672F77732F323030342F30382F61646472657373696E672220786D6C6E733A773D22687474703A2F2F736368656D61732E646D74662E6F72672F7762656D2F77736D616E2F312F77736D616E2E7873642220786D6C6E733D22687474703A2F2F7777772E77332E6F72672F323030332F30352F736F61702D656E76656C6F706522203E3C4865616465723E3C613A416374696F6E3E687474703A2F2F696E74656C2E636F6D2F7762656D2F777363696D2F312F6970732D736368656D612F312F4950535F486F737442617365645365747570536572766963652F41646D696E53657475703C2F613A416374696F6E3E3C613A546F3E2F77736D616E3C2F613A546F3E3C773A5265736F757263655552493E687474703A2F2F696E74656C2E636F6D2F7762656D2F777363696D2F312F6970732D736368656D612F312F4950535F486F737442617365645365747570536572766963653C2F773A5265736F757263655552493E3C613A4D65737361676549443E363C2F613A4D65737361676549443E3C613A5265706C79546F3E3C613A416464726573733E687474703A2F2F736368656D61732E786D6C736F61702E6F72672F77732F323030342F30382F61646472657373696E672F726F6C652F616E6F6E796D6F75733C2F613A416464726573733E3C2F613A5265706C79546F3E3C773A4F7065726174696F6E54696D656F75743E50543630533C2F773A4F7065726174696F6E54696D656F75743E3C2F4865616465723E3C426F64793E3C723A41646D696E53657475705F494E50555420786D6C6E733A723D22687474703A2F2F696E74656C2E636F6D2F7762656D2F777363696D2F312F6970732D736368656D612F312F4950535F486F73744261736564536574757053657276696365223E3C723A4E657441646D696E50617373456E6372797074696F6E547970653E323C2F723A4E657441646D696E50617373456E6372797074696F6E547970653E3C723A4E6574776F726B41646D696E50617373776F72643E37393230396630363239663965343135386635396561303665663865383834353C2F723A4E6574776F726B41646D696E50617373776F72643E3C723A4D634E6F6E63653E2B516D55646A692F775964424D6C364E4F54686D507249414C746B3D3C2F723A4D634E6F6E63653E3C723A5369676E696E67416C676F726974686D3E323C2F723A5369676E696E67416C676F726974686D3E3C723A4469676974616C5369676E61747572653E4F4F45306D4944374C6E6572332F617579554E4450764E6679766861426F77535363595A4D7750376B394C304D694C39684C6F423253356B742B41733663784D684335527A752F74466B506A7A626C6371334F674C3842386633635153736B7431337548484E714B2B2B67626D4A6A484C6D636A74476C363843714658794B726445796655676A5948346B6872642B416F6D596179615565376375635A3639642B43432B69477531624C5876466656496A656362725137666A5533735262676E73684D7031353135534F383247767838646970466F43364162646969304634576A71636E2F3236547732314E767757353235745861356A7443784455766B74664F5A2F7A4B436166534A6E763975434943584762642F6A31452B596C34434E656B78364F7639595A445654456F74736B466E4C433632727076772B764A4E744C6932336D6430687A4535723939394F75756C667556773D3D3C2F723A4469676974616C5369676E61747572653E3C2F723A41646D696E53657475705F494E5055543E3C2F426F64793E3C2F456E76656C6F70653E" Write completed. APF: Send ChannelData: 485454502F312E3120323030204F4B0D0A446174653A205475652C2032322046656220323032322032303A35333A343920474D540D0A5365727665723A20496E74656C28522920416374697665204D616E6167656D656E7420546563686E6F6C6F67792031322E302E33352E313432370D0A582D4672616D652D4F7074696F6E733A2044454E590D0A436F6E74656E742D547970653A206170706C69636174696F6E2F6F637465742D73747265616D0D0A5472616E736665722D456E636F64696E673A206368756E6B65640D0A0D0A303433450D0A3C3F786D6C2076657273696F6E3D22312E302220656E636F64696E673D225554462D38223F3E3C613A456E76656C6F706520786D6C6E733A613D22687474703A2F2F7777772E77332E6F72672F323030332F30352F736F61702D656E76656C6F70652220786D6C6E733A623D22687474703A2F2F736368656D61732E786D6C736F61702E6F72672F77732F323030342F30382F61646472657373696E672220786D6C6E733A633D22687474703A2F2F736368656D61732E646D74662E6F72672F7762656D2F77736D616E2F312F77736D616E2E7873642220786D6C6E733A643D22687474703A2F2F736368656D61732E786D6C736F61702E6F72672F77732F323030352F30322F74727573742220786D6C6E733A653D22687474703A2F2F646F63732E6F617369732D6F70656E2E6F72672F7773732F323030342F30312F6F617369732D3230303430312D7773732D777373656375726974792D7365636578742D312E302E7873642220786D6C6E733A663D22687474703A2F2F736368656D61732E646D74662E6F72672F7762656D2F77736D616E2F312F63696D62696E64696E672E7873642220786D6C6E733A673D22687474703A2F2F696E74656C2E636F6D2F7762656D2F777363696D2F312F6970732D736368656D612F312F4950535F486F737442617365645365747570536572766963652220786D6C6E733A7873693D22687474703A2F2F7777772E77332E6F72672F323030312F584D4C536368656D612D696E7374616E6365223E3C613A4865616465723E3C623A546F3E687474703A2F2F736368656D61732E786D6C736F61702E6F72672F77732F323030342F30382F61646472657373696E672F726F6C652F616E6F6E796D6F75733C2F623A546F3E3C623A52656C61746573546F3E363C2F623A52656C61746573546F3E3C623A416374696F6E20613A6D757374556E6465727374616E643D2274727565223E687474703A2F2F696E74656C2E636F6D2F7762656D2F777363696D2F312F6970732D736368656D612F312F4950535F486F737442617365645365747570536572766963652F41646D696E5365747570526573706F6E73653C2F623A416374696F6E3E3C623A4D65737361676549443E757569643A30303030303030302D383038362D383038362D383038362D3030303030303030303035393C2F623A4D65737361676549443E3C633A5265736F757263655552493E687474703A2F2F696E74656C2E636F6D2F7762656D2F777363696D2F312F6970732D736368656D61 APF: Send ChannelData: 2F312F4950535F486F737442617365645365747570536572766963653C2F633A5265736F757263655552493E3C2F613A4865616465723E3C613A426F64793E3C673A41646D696E53657475705F4F55545055543E3C673A52657475726E56616C75653E353C2F673A52657475726E56616C75653E3C2F673A41646D696E53657475705F4F55545055543E3C2F613A426F64793E3C2F613A456E76656C6F70653E0D0A300D0A0D0A APF: JSON_CONTROL Failed to complete ACM activation (ERR/5). APF: CHANNEL_CLOSE: 5

Ylianst commented 2 years ago

You can use a site like this to convert HEX to Text. From your debug log, message to AMT:

POST /wsman HTTP/1.1
Authorization: Digest username="$$OsAdmin",realm="Digest:3ED50000000000000000000000000000",nonce="VG1NAa0BAAAAAAAApMqtaJSc/SqvFkMz",uri="/wsman",qop="auth",response="66963f587da1752c0993ed776d64d605",nc="6",cnonce="5f282c3ce8862804e96af64fdc3ff846"
Host: node//AwACAAQABQAABgAHAAgACQMAAgAEAAUAAAYABwAIAAkDAAIABAAFAAAGAAcACAAJ:16992
Content-Length: 1495

<?xml version="1.0" encoding="utf-8"?><Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:a="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:w="http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd" xmlns="http://www.w3.org/2003/05/soap-envelope" ><Header><a:Action>http://intel.com/wbem/wscim/1/ips-schema/1/IPS_HostBasedSetupService/AdminSetup</a:Action><a:To>/wsman</a:To><w:ResourceURI>http://intel.com/wbem/wscim/1/ips-schema/1/IPS_HostBasedSetupService</w:ResourceURI><a:MessageID>6</a:MessageID><a:ReplyTo><a:Address>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</a:Address></a:ReplyTo><w:OperationTimeout>PT60S</w:OperationTimeout></Header><Body><r:AdminSetup_INPUT xmlns:r="http://intel.com/wbem/wscim/1/ips-schema/1/IPS_HostBasedSetupService"><r:NetAdminPassEncryptionType>2</r:NetAdminPassEncryptionType><r:NetworkAdminPassword>79209f0629f9e4158f59ea06ef8e8845</r:NetworkAdminPassword><r:McNonce>+QmUdji/wYdBMl6NOThmPrIALtk=</r:McNonce><r:SigningAlgorithm>2</r:SigningAlgorithm><r:DigitalSignature>OOE0mID7Lner3/auyUNDPvNfyvhaBowSScYZMwP7k9L0MiL9hLoB2S5kt+As6cxMhC5Rzu/tFkPjzblcq3OgL8B8f3cQSskt13uHHNqK++gbmJjHLmcjtGl68CqFXyKrdEyfUgjYH4khrd+AomYayaUe7cucZ69d+CC+iGu1bLXvFfVIjecbrQ7fjU3sRbgnshMp1515SO82Gvx8dipFoC6Abdii0F4Wjqcn/26Tw21NvwW525tXa5jtCxDUvktfOZ/zKCafSJnv9uCICXGbd/j1E+Yl4CNekx6Ov9YZDVTEotskFnLC62rpvw+vJNtLi23md0hzE5r999OuulfuVw==</r:DigitalSignature></r:AdminSetup_INPUT></Body></Envelope>

Response from AMT:

HTTP/1.1 200 OK
Date: Tue, 22 Feb 2022 20:53:49 GMT
Server: Intel(R) Active Management Technology 12.0.35.1427
X-Frame-Options: DENY
Content-Type: application/octet-stream
Transfer-Encoding: chunked

043E
<?xml version="1.0" encoding="UTF-8"?><a:Envelope xmlns:a="http://www.w3.org/2003/05/soap-envelope" xmlns:b="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:c="http://schemas.dmtf.org/wbem/wsman/1/wsman.xsd" xmlns:d="http://schemas.xmlsoap.org/ws/2005/02/trust" xmlns:e="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:f="http://schemas.dmtf.org/wbem/wsman/1/cimbinding.xsd" xmlns:g="http://intel.com/wbem/wscim/1/ips-schema/1/IPS_HostBasedSetupService" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><a:Header><b:To>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</b:To><b:RelatesTo>6</b:RelatesTo><b:Action a:mustUnderstand="true">http://intel.com/wbem/wscim/1/ips-schema/1/IPS_HostBasedSetupService/AdminSetupResponse</b:Action><b:MessageID>uuid:00000000-8086-8086-8086-000000000059</b:MessageID><c:ResourceURI>http://intel.com/wbem/wscim/1/ips-schema/1/IPS_HostBasedSetupService</c:ResourceURI></a:Header><a:Body><g:AdminSetup_OUTPUT><g:ReturnValue>5</g:ReturnValue></g:AdminSetup_OUTPUT></a:Body></a:Envelope>
0

It's returning error 5, which according to Intel AMT SDK is "AUTH_FAILED". So, Intel AMT rejected the activation request. It may be that AMT did not get the DNS suffix, or your activation certificate is not valid.

alexkolomolo commented 2 years ago

Thanks, that's very helpful to be able to see the AMT SDK details now.

Could there be any other reasons for auth fail? Because

FYI option 15 mydomain.com is the same as my certs mydomain.com. There is no wildcard or sub domain. I presume that should work fine?

Ylianst commented 2 years ago

Wild card cert for Intel AMT activation are not allowed. If you are saying that going to MEBx and entering the DNS suffix manually works, but using a DHCP option 15 does not... then the cert is not the cause for sure. It's certainly the DHCP option 15.

Note that you need to send option 15 over the Intel AMT wired NIC. Intel AMT Wireless will not do. Also, if you have many wired network interfaces on the Intel AMT device, you must have option 15 received on the Intel AMT managed wired one, other network interfaces will not do.

alexkolomolo commented 2 years ago

Thanks for the reply.

We are not using wildcard. Provisioning works when manually entering dns suffix, so ok it's not an issue with the cert. # Machine has 2 NICs. I also know i'm using the correct adapter as I check for the mac address in meshcmd AmtNetwork, and check that with my active network interface using ip a. Only one NIC has a cable plugged in.

We are sending option 15 over the wired NIC. I understand the AMT chip is sniffing out the data from the DHCP reply so yep, it's on the same NIC. It's being ignored...

I read here that I need a server setup on provisionServer."[Option-15]". I did that also. Could there be any issue with DNS names of the server?