When I add a large number of instances - EnumerateInstances() function fails for HTTPS but works for HTTP

[ganeshkamath89]

I have a large number of instances in one of my provider classes. For confidentiality reasons I am unable to share the code.

When I run HTTPS enumeration, it fails with no response - not even error.

In both these cases, it works without any problem.: 1) HTTP with same number of instances 2) HTTPS with lesser number of instances

Since it is HTTPS, I am not able to make sense of wireshark log. But I am observing some communication from Server.

Steps to reproduce the issue: Specifically I used the CIM schema's CIM_RegisteredProfile class and used the provider for C. using omigen. omigen.exe -m CIM_RegisteredProfile CIM_RegisteredProfile.mof CIM_RegisteredProfile Location: https://github.com/microsoft/omi/blob/master/Unix/share/omischema/CIM-2.32.0/Interop/CIM_RegisteredProfile.mof

Observation

• When ever I add more than 28 instances, it is failing for HTTPS but works for HTTP.
• When ever I add less than or equal to 28 instances, it works for both HTTPS and HTTP.
• If I increase the instance count to more than 28, even then it works for HTTP but fails for HTTPS.
• Even when I increase the instances from 28 to 42, which is the maximum for my application, it works over HTTP.

Need help in resolving this issue. Issue is also observed for other classes when large number of instances exist.

Steps used to generate HTTPS certificate: https://community.amd.com/t5/amd-manageability-community/how-to-use-self-signed-certificates-for-https-communication/ba-p/418724

I am currently using EnumerateInstances() function generated by omigen.

I have noticed that in an sample file you have provided code to simulate 10000 instances: https://github.com/microsoft/omi/blob/master/Unix/samples/Providers/Network/OMI_NetworkVLAN.c

Is this tested over HTTPS?

[JumpingYang001]

Hi @ganeshkamath89 , omi doesn't maintain on Windows for many years. the samples under https://github.com/microsoft/omi/blob/master/Unix/samples/Providers/ also don't maintain. omi only maintain 3 providers now: scxcore, mysql and apache, and we don't maintain other providers.

For difference between HTTPS and HTTP, I think HTTPS will use encryption so the WSMan xml size might be bigger than HTTP, but if you say 28 instances will exceed the limitation, I think we have 250k packet size limitation and we also add some field like username,password,etc. size limitation in latest 2 years.

Some ways you can troubleshoot your issue:

1. Enable omiserver.conf loglevel to VERBOSE and restart omi, then repro your issue, check omiserver.log.
2. Enable omicli.conf loglevel to VERBOSE if you want to check omicli log.
3. Debugging your omiserver or omicli using gdb.

[ganeshkamath89]

Hi @JumpingYang001. I debugged the code with verbose log like you suggested.

I found that the logs for the two cases match almost throught. In HTTP case I see that the following lines are printed:

(E)Protocol _RequestCallback: _RequestCallbackRead fails for ProtocolSocket 03426908
closing socket due to SELECTOR_REMOVE or SELECTOR_DESTROY
EventId=45355 Priority=DEBUG Sock_Close: sock (520)
EventId=55027 Priority=VERBOSE _Strand_Schedule: 03426908(HttpSocket) state 0(S:), methodBit: 800(CloseOther)
EventId=55020 Priority=VERBOSE Strand_ExecLoop: 03426908(HttpSocket) state 801(S:*c), bit: 12(CloseOther) [-, O t:-- o:-- 0343AE80]
EventId=55027 Priority=VERBOSE _Strand_Schedule: 0343AE58(WsmanConnection) state 0(SB:), methodBit: 10000(Close_Left)
EventId=55020 Priority=VERBOSE Strand_ExecLoop: 0343AE58(WsmanConnection) state 10001(SB:*C), bit: 17(Close_Left) [-, O t:-- o:-- 03426930, O t:C- o:C- 033C7C78]
EventId=55000 Priority=VERBOSE Strand 0343AE58(WsmanConnection), action: CloseLeft
EventId=55027 Priority=VERBOSE _Strand_Schedule: 03426908(HttpSocket) state 801(S:*c), methodBit: 400(Close)
EventId=55000 Priority=VERBOSE Strand 0343AE58(WsmanConnection), action: Returning from CloseLeft
EventId=55023 Priority=VERBOSE Strand_ExecLoop: 0343AE58(WsmanConnection) Exits, state 1, ToFinish: 1 [-, O t:C- o:C- 03426930, O t:C- o:C- 033C7C78]
EventId=55053 Priority=VERBOSE Message_Release: wsman.c(727): __Message_Release(HttpResponseMsg): 05207890: refs=0
EventId=55000 Priority=VERBOSE Strand 0343AE58(WsmanConnection), action: Deleting Strand
EventId=55000 Priority=VERBOSE Strand 03426908(HttpSocket), action: Returning from CloseOther
EventId=55020 Priority=VERBOSE Strand_ExecLoop: 03426908(HttpSocket) state 401(S:*C), bit: 11(Close) [-, O t:C- o:-- 0343AE80]
EventId=55023 Priority=VERBOSE Strand_ExecLoop: 03426908(HttpSocket) Exits, state 1, ToFinish: 1 [-, O t:C- o:C- 0343AE80]
EventId=45036 Priority=DEBUG HttpSocket: 03426908 _HttpSocket_Finish
EventId=55000 Priority=VERBOSE Strand 03426908(HttpSocket), action: Deleting Strand

Where as in the HTTPS case these lines are missing.

Is it possible for you to check if this can help in root causing my problem?

[JumpingYang001]

@ganeshkamath89 if missing the lines in HTTPS seems mean the socket didn't be close and blocked when it is processing provider code, can you share me your steps to reproduce the issue?

[ganeshkamath89]

Hi @JumpingYang001, I have separated my ticket description into "Steps to reproduce" / "Observation" / "Steps to generate Self Signed Certificate" section for more clarity.

[ganeshkamath89]

Hi @JumpingYang001 , I noticed that in code http.c in function: _RequestCallback

For HTTP case it enters the if condition:

    if (((mask & SELECTOR_READ) != 0 && !handler->reverseOperations) ||
        ((mask & SELECTOR_WRITE) != 0 && handler->reverseOperations))
    {
        if (!_RequestCallbackRead(handler))
        {
            trace_RequestCallbackRead_Failed(ENGINE_TYPE, handler);
            return MI_FALSE;
        }
    }

Where as for HTTPS it enters the if condition

    if (((mask & SELECTOR_WRITE) != 0 && !handler->reverseOperations) ||
        ((mask & SELECTOR_READ) != 0 && handler->reverseOperations))
    {
        if (!_RequestCallbackWrite(handler))
        {
            trace_RequestCallbackWrite_Failed(ENGINE_TYPE, handler);
            return MI_FALSE;
        }
    }

before sending back the last response. Can you check if this can help in root cousing the issue please?