Closed kshreep closed 4 years ago
Hi,
Here’s my take on the three issues:
while deleting a session,
Was the session in stable state? Can you duplicate this issue? This is most likely not related to the card type, but rather the session state was not in one of INITIAL, STOPPED or ABNORMALLY STOPPED.
while clearing stats,
The card type matters here. Can you tell me the exact card type name that you see in ixNetwork GUI’s port resource window? What is the ixNetwork version you’re using?
while starting traffic
The error message indicates that the prior clearStats call is not done yet when you start the traffic. See error snip: “…. is blocked by: writer 200 at 5/20/2020 8:44:52 PM: clearStats(kArray[kEnumValue=waitForPortStatsRefresh\~waitForTrafficStatsRefresh])kVoid\n at SDMCsLib.ReaderWriterLoc”
Regards, Theresa
From: kshreep notifications@github.com Sent: Thursday, May 21, 2020 9:29 AM To: OpenIxia/IxNetwork IxNetwork@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: [OpenIxia/IxNetwork] Exception during delete session (#103)
[EXTERNAL]
The API was working fine, recently encountered few errors while running on 100GE ixia card. It works perfectly when the card is 10GE and sometimes it works fine with the card 100GE as well.
while deleting a session,
File "/Traffic/IxNetworkAPI.py", line 1965, in disarmConfigs self.obj.linuxServerStopAndDeleteSession() File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 1147, in linuxServerStopAndDeleteSession self.linuxServerDeleteSession() File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 1188, in linuxServerDeleteSession response = self.delete(sessionId) File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 630, in delete raise IxNetRestApiException(errMsg) Traffic.IxNetwork.RestApi.Python.Modules.IxNetRestApi.IxNetRestApiException: DELETE Exception error 5/5 retries: DELETE Exception error: { "error" : "Invalid request for session with id '3362'. Only sessions in INITIAL, STOPPED or ABNORMALLY STOPPED states can be deleted."_ }
while clearing stats,
File "/Traffic/IxNetworkAPI.py", line 1941, in clearStatistics self._statsObj.clearStats() File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApiStatistics.py", line 512, in clearStats self.ixnObj.waitForComplete(response, self.ixnObj.httpHeader + response.json()['url']) File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 955, in waitForComplete raise IxNetRestApiException('waitForComplete failed: %s' % response.json()) infra.Traffic.IxNetwork.RestApi.Python.Modules.IxNetRestApi.IxNetRestApiException: waitForComplete failed: {'executionTimeMs': 107219.0, 'id': '8', 'state': 'IN_PROGRESS', 'progress': 0, 'message': None, 'url': '/api/v1/sessions/3319/ixnetwork/operations/clearstats/8', 'resultUrl': '', 'result': None}
while starting traffic,
File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 399, in get raise IxNetRestApiException(errMsg) infra.Traffic.IxNetwork.RestApi.Python.Modules.IxNetRestApi.IxNetRestApiException: GET Exception error: {"method":"GET","pathAndQuery":"/api/v1/sessions/3319/ixnetwork/traffic/trafficItem?take=100&skip=0","errors":[{"code":10000,"detail":"API CONTENTION SlimLock waitingReaders:0 waitingWriters:0 currentReadCount:0 writeLockHeld:False readLockHeld:False LockDetails reader lock request /api/v1/sessions/3319/ixnetwork/traffic/trafficItem is blocked by: writer 200 at 5/20/2020 8:44:52 PM: clearStats(kArray[kEnumValue=waitForPortStatsRefresh\~waitForTrafficStatsRefresh])kVoid"},{"code":10000,"detail":"SDMCsLib.SdmRequestException: API CONTENTION SlimLock waitingReaders:0 waitingWriters:0 currentReadCount:0 writeLockHeld:False readLockHeld:False LockDetails reader lock request /api/v1/sessions/3319/ixnetwork/traffic/trafficItem is blocked by: writer 200 at 5/20/2020 8:44:52 PM: clearStats(kArray[kEnumValue=waitForPortStatsRefresh\~waitForTrafficStatsRefresh])kVoid\n at SDMCsLib.ReaderWriterLock..ctor (System.String newRequest, System.Boolean readerLock, System.Boolean bypassLock) [0x00034] in <3d657fd98a28481ea4ae0dad8da57771>:0 \n at SDMCsLib.RestService.SdmObject.Read () [0x000de] in <8faf1fb1b99943fcaae26e5c2fc5c6fb>:0 \n at SDMCsLib.RestService.V1ObjectController.Get (System.String path) [0x0001a] in <8faf1fb1b99943fcaae26e5c2fc5c6fb>:0 "}]}
what would be the root cause for those errors?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/OpenIxia/IxNetwork/issues/103__;!!I5pVk4LIGAfnvw!zhC4hoOGFD8NN5Ig2Ea_kHvcFtDTkP3x1-yxrZfrRB6VoCR9W6mrPtqOPTJhY6nQ1-wF$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AIRRUUJGU3IZFG5RWGZEEQ3RSVJDFANCNFSM4NG7B5ZQ__;!!I5pVk4LIGAfnvw!zhC4hoOGFD8NN5Ig2Ea_kHvcFtDTkP3x1-yxrZfrRB6VoCR9W6mrPtqOPTJhY4w-8th4$.
Another question is did you have the API browser or Web UI open throughout the execution, say looking at the stat view.
In 9.0 there are some issues with API contention. Looking at the session while the script runs could affect the script. The best way to avoid this is by looking at the UI while the script is paused, then resuming the script and pausing again before looking at the web UI more.
In 9.10 this is resolved.
Others please check me on this.
Thanks, Wes
Wesley Hand m: 818-312-6752
From: therkong notifications@github.com Sent: Thursday, May 21, 2020 10:23 AM To: OpenIxia/IxNetwork IxNetwork@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [OpenIxia/IxNetwork] Exception during delete session (#103)
[EXTERNAL] Hi,
Here’s my take on the three issues:
while deleting a session,
Was the session in stable state? Can you duplicate this issue? This is most likely not related to the card type, but rather the session state was not in one of INITIAL, STOPPED or ABNORMALLY STOPPED.
while clearing stats,
The card type matters here. Can you tell me the exact card type name that you see in ixNetwork GUI’s port resource window? What is the ixNetwork version you’re using?
while starting traffic
The error message indicates that the prior clearStats call is not done yet when you start the traffic. See error snip: “…. is blocked by: writer 200 at 5/20/2020 8:44:52 PM: clearStats(kArray[kEnumValue=waitForPortStatsRefresh\~waitForTrafficStatsRefresh])kVoid\n at SDMCsLib.ReaderWriterLoc”
Regards, Theresa
From: kshreep notifications@github.com<mailto:notifications@github.com> Sent: Thursday, May 21, 2020 9:29 AM To: OpenIxia/IxNetwork IxNetwork@noreply.github.com<mailto:IxNetwork@noreply.github.com> Cc: Subscribed subscribed@noreply.github.com<mailto:subscribed@noreply.github.com> Subject: [OpenIxia/IxNetwork] Exception during delete session (#103)
[EXTERNAL]
The API was working fine, recently encountered few errors while running on 100GE ixia card. It works perfectly when the card is 10GE and sometimes it works fine with the card 100GE as well.
while deleting a session,
File "/Traffic/IxNetworkAPI.py", line 1965, in disarmConfigs self.obj.linuxServerStopAndDeleteSession() File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 1147, in linuxServerStopAndDeleteSession self.linuxServerDeleteSession() File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 1188, in linuxServerDeleteSession response = self.delete(sessionId) File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 630, in delete raise IxNetRestApiException(errMsg) Traffic.IxNetwork.RestApi.Python.Modules.IxNetRestApi.IxNetRestApiException: DELETE Exception error 5/5 retries: DELETE Exception error: { "error" : "Invalid request for session with id '3362'. Only sessions in INITIAL, STOPPED or ABNORMALLY STOPPED states can be deleted."_ }
while clearing stats,
File "/Traffic/IxNetworkAPI.py", line 1941, in clearStatistics self._statsObj.clearStats() File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApiStatistics.py", line 512, in clearStats self.ixnObj.waitForComplete(response, self.ixnObj.httpHeader + response.json()['url']) File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 955, in waitForComplete raise IxNetRestApiException('waitForComplete failed: %s' % response.json()) infra.Traffic.IxNetwork.RestApi.Python.Modules.IxNetRestApi.IxNetRestApiException: waitForComplete failed: {'executionTimeMs': 107219.0, 'id': '8', 'state': 'IN_PROGRESS', 'progress': 0, 'message': None, 'url': '/api/v1/sessions/3319/ixnetwork/operations/clearstats/8', 'resultUrl': '', 'result': None}
while starting traffic,
File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 399, in get raise IxNetRestApiException(errMsg) infra.Traffic.IxNetwork.RestApi.Python.Modules.IxNetRestApi.IxNetRestApiException: GET Exception error: {"method":"GET","pathAndQuery":"/api/v1/sessions/3319/ixnetwork/traffic/trafficItem?take=100&skip=0","errors":[{"code":10000,"detail":"API CONTENTION SlimLock waitingReaders:0 waitingWriters:0 currentReadCount:0 writeLockHeld:False readLockHeld:False LockDetails reader lock request /api/v1/sessions/3319/ixnetwork/traffic/trafficItem is blocked by: writer 200 at 5/20/2020 8:44:52 PM: clearStats(kArray[kEnumValue=waitForPortStatsRefresh\~waitForTrafficStatsRefresh])kVoid"},{"code":10000,"detail":"SDMCsLib.SdmRequestException: API CONTENTION SlimLock waitingReaders:0 waitingWriters:0 currentReadCount:0 writeLockHeld:False readLockHeld:False LockDetails reader lock request /api/v1/sessions/3319/ixnetwork/traffic/trafficItem is blocked by: writer 200 at 5/20/2020 8:44:52 PM: clearStats(kArray[kEnumValue=waitForPortStatsRefresh\~waitForTrafficStatsRefresh])kVoid\n at SDMCsLib.ReaderWriterLock..ctor (System.String newRequest, System.Boolean readerLock, System.Boolean bypassLock) [0x00034] in <3d657fd98a28481ea4ae0dad8da57771>:0 \n at SDMCsLib.RestService.SdmObject.Read () [0x000de] in <8faf1fb1b99943fcaae26e5c2fc5c6fb>:0 \n at SDMCsLib.RestService.V1ObjectController.Get (System.String path) [0x0001a] in <8faf1fb1b99943fcaae26e5c2fc5c6fb>:0 "}]}
what would be the root cause for those errors?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/OpenIxia/IxNetwork/issues/103__;!!I5pVk4LIGAfnvw!zhC4hoOGFD8NN5Ig2Ea_kHvcFtDTkP3x1-yxrZfrRB6VoCR9W6mrPtqOPTJhY6nQ1-wF$%3E, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AIRRUUJGU3IZFG5RWGZEEQ3RSVJDFANCNFSM4NG7B5ZQ__;!!I5pVk4LIGAfnvw!zhC4hoOGFD8NN5Ig2Ea_kHvcFtDTkP3x1-yxrZfrRB6VoCR9W6mrPtqOPTJhY4w-8th4$%3E.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/OpenIxia/IxNetwork/issues/103*issuecomment-632236591__;Iw!!I5pVk4LIGAfnvw!13QSScznZ8Y7aIK2w-WA1GuEUYH4frxR8oOn2OZvWpz8ScwFgoq-16wes74Zqg$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AGJXMDCZOVXHOLBAITDMEA3RSVPOPANCNFSM4NG7B5ZQ__;!!I5pVk4LIGAfnvw!13QSScznZ8Y7aIK2w-WA1GuEUYH4frxR8oOn2OZvWpz8ScwFgoq-16wzOj3sBQ$.
while deleting a session,
Was the session in stable state? Can you duplicate this issue? This is most likely not related to the card type, but rather the session state was not in one of INITIAL, STOPPED or ABNORMALLY STOPPED.
Yeah, It was in stable state so because next test case got executed and passed by using the same ixia port. I suspect problem occurred during session deletion after deallocating ports. The problem with reproducing this issue is that it does not occur every time.
while clearing stats,
The card type matters here. Can you tell me the exact card type name that you see in ixNetwork GUI’s port resource window? What is the ixNetwork version you’re using?
card name: NOVUS100GE8Q28+FAN, IxNetwork version is 9.00.2.182
while starting traffic,
The error message indicates that the prior clearStats call is not done yet when you start the traffic. See error snip: “…. is blocked by: writer 200 at 5/20/2020 8:44:52 PM: clearStats(kArray[kEnumValue=waitForPortStatsRefresh\~waitForTrafficStatsRefresh])kVoid\n at SDMCsLib.ReaderWriterLoc”
I suspect the same here.
did you have the API browser or Web UI open throughout the execution, say looking at the stat view? yes, sometimes I do this, could it be the reason for failure? then what would be the cause while deleting the session?
what are the improvs in 9.10? sharing a link would be good!.
while deleting a session,
Was the session in stable state? Can you duplicate this issue? This is most likely not related to the card type, but rather the session state was not in one of INITIAL, STOPPED or ABNORMALLY STOPPED.
Yeah, It was in stable state so because next test case got executed and passed by using the same ixia port. I suspect problem occurred during session deletion after deallocating ports. The problem with reproducing this issue is that it does not occur every time.
TK>>> To debug this, can you set ixnetwork.Globals.Preferences.ClientTraceLevel = ‘debug’ in your script? If the issue is hit, please collect the quickTest webUI system diag as well as the ixNetwork session diag. You can open a support ticket by sending email to support.ix@keysight.commailto:support.ix@keysight.com with problem description, the diag files, and the ixNetwork activation code or license info. One of our support engineer will work with you.
while clearing stats,
The card type matters here. Can you tell me the exact card type name that you see in ixNetwork GUI’s port resource window? What is the ixNetwork version you’re using?
card name: NOVUS100GE8Q28+FAN, IxNetwork version is 9.00.2.182
TK>>> Do you have ‘waitPortStatsRefresh’ arguments to clearStats call? In 8.51 release, this card type had problem with clearStats if you pass in ‘waitPortStatsRefresh’ argument. This issue should be fixed in 9.0. You can try removing the ‘waitPortStatsRefresh’ if you have it.
To find the root cause, it’s best to set ixnetwork.Globals.Preferences.ClientTraceLevel = ‘debug’ in your script, after the issue is hit, collect the ixnetwork session diag and open a support ticket to have one of the support team member analyze the diag files.
while starting traffic,
The error message indicates that the prior clearStats call is not done yet when you start the traffic. See error snip: “…. is blocked by: writer 200 at 5/20/2020 8:44:52 PM: clearStats(kArray[kEnumValue=waitForPortStatsRefresh\~waitForTrafficStatsRefresh])kVoid\n at SDMCsLib.ReaderWriterLoc”
I suspect the same here.
did you have the API browser or Web UI open throughout the execution, say looking at the stat view? yes, sometimes I do this, could it be the reason for failure? then what would be the cause while deleting the session?
what are the improvs in 9.10? sharing a link would be good!.
TK>>> We don’t have the link yet. I will share once we have it.
Regards, Theresa
From: kshreep notifications@github.com Sent: Thursday, May 21, 2020 3:56 PM To: OpenIxia/IxNetwork IxNetwork@noreply.github.com Cc: Theresa Kong theresa.kong@keysight.com; Comment comment@noreply.github.com Subject: Re: [OpenIxia/IxNetwork] Exception during delete session (#103)
[EXTERNAL]
while deleting a session,
Was the session in stable state? Can you duplicate this issue? This is most likely not related to the card type, but rather the session state was not in one of INITIAL, STOPPED or ABNORMALLY STOPPED.
Yeah, It was in stable state so because next test case got executed and passed by using the same ixia port. I suspect problem occurred during session deletion after deallocating ports. The problem with reproducing this issue is that it does not occur every time.
while clearing stats,
The card type matters here. Can you tell me the exact card type name that you see in ixNetwork GUI’s port resource window? What is the ixNetwork version you’re using?
card name: NOVUS100GE8Q28+FAN, IxNetwork version is 9.00.2.182
while starting traffic,
The error message indicates that the prior clearStats call is not done yet when you start the traffic. See error snip: “…. is blocked by: writer 200 at 5/20/2020 8:44:52 PM: clearStats(kArray[kEnumValue=waitForPortStatsRefresh\~waitForTrafficStatsRefresh])kVoid\n at SDMCsLib.ReaderWriterLoc”
I suspect the same here.
did you have the API browser or Web UI open throughout the execution, say looking at the stat view? yes, sometimes I do this, could it be the reason for failure? then what would be the cause while deleting the session?
what are the improvs in 9.10? sharing a link would be good!.
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/OpenIxia/IxNetwork/issues/103*issuecomment-632386887__;Iw!!I5pVk4LIGAfnvw!1ASNuQY1mLxrQMTqk6aTagnm5VR6Mzwf-XYSpsPbrcZXzcJVrTL1rpNDlnKGJsoTKgBp$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AIRRUUKGUCPFSY7QTQLQY2DRSWWRBANCNFSM4NG7B5ZQ__;!!I5pVk4LIGAfnvw!1ASNuQY1mLxrQMTqk6aTagnm5VR6Mzwf-XYSpsPbrcZXzcJVrTL1rpNDlnKGJjKymond$.
Hi, I can open a support case for you and continue investigate the issues you reported. What is your email address?
Hello,
Thanks for your response. That would be very helpful. Email address is kshreep@gmail.com
Regards, Kavya
On Thu., May 21, 2020, 7:49 p.m. therkong, notifications@github.com wrote:
Hi, I can open a support case for you and continue investigate the issues you reported. What is your email address?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/OpenIxia/IxNetwork/issues/103#issuecomment-632402179, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKUE74XZPNOVZ4AFP5ZNPPLRSW4Y5ANCNFSM4NG7B5ZQ .
Have you opened a support case before? If yes, what email address did you use? If not, do you have ixNetwork activation code or license info?
I have enabled diag logs as per the suggestion, waiting for the issue to occur. I haven't opened support case before but we have a license info, will come back with the details once it occurs again.
Thanks,
Hi, have you hit this issue again? please open a support ticket if you hit this issue since we have to analyzed the diag logs.
closing this issue since we have contacted dedicated supporting team member for the issues related to ixia chassis.
Thanks
The API was working fine, recently encountered few errors while running on 100GE ixia card. It works perfectly when the card is 10GE and sometimes it works fine with the card 100GE as well.
while deleting a session,
File "/Traffic/IxNetworkAPI.py", line 1965, in disarmConfigs self._obj.linuxServerStopAndDeleteSession() File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 1147, in linuxServerStopAndDeleteSession self.linuxServerDeleteSession() File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 1188, in linuxServerDeleteSession response = self.delete(sessionId) File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 630, in delete raise IxNetRestApiException(errMsg) Traffic.IxNetwork.RestApi.Python.Modules.IxNetRestApi.IxNetRestApiException: DELETE Exception error 5/5 retries: DELETE Exception error: { _"error" : "Invalid request for session with id '3362'. Only sessions in INITIAL, STOPPED or ABNORMALLY STOPPED states can be deleted."_ }
while clearing stats,
File "/Traffic/IxNetworkAPI.py", line 1941, in clearStatistics self._statsObj.clearStats() File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApiStatistics.py", line 512, in clearStats self.ixnObj.waitForComplete(response, self.ixnObj.httpHeader + response.json()['url']) File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 955, in waitForComplete raise IxNetRestApiException('waitForComplete failed: %s' % response.json()) infra.Traffic.IxNetwork.RestApi.Python.Modules.IxNetRestApi.IxNetRestApiException: waitForComplete failed: {'executionTimeMs': 107219.0, 'id': '8', 'state': 'IN_PROGRESS', 'progress': 0, 'message': None, 'url': '/api/v1/sessions/3319/ixnetwork/operations/clearstats/8', 'resultUrl': '', 'result': None}
while starting traffic,
File "/Traffic/IxNetwork/RestApi/Python/Modules/IxNetRestApi.py", line 399, in get raise IxNetRestApiException(errMsg) infra.Traffic.IxNetwork.RestApi.Python.Modules.IxNetRestApi.IxNetRestApiException: GET Exception error: {"method":"GET","pathAndQuery":"/api/v1/sessions/3319/ixnetwork/traffic/trafficItem?take=100&skip=0","errors":[{"code":10000,"detail":"API CONTENTION SlimLock waitingReaders:0 waitingWriters:0 currentReadCount:0 writeLockHeld:False readLockHeld:False LockDetails reader lock request /api/v1/sessions/3319/ixnetwork/traffic/trafficItem is blocked by: writer 200 at 5/20/2020 8:44:52 PM: clearStats(kArray[kEnumValue=waitForPortStatsRefresh\\~waitForTrafficStatsRefresh])kVoid"},{"code":10000,"detail":"SDMCsLib.SdmRequestException: API CONTENTION SlimLock waitingReaders:0 waitingWriters:0 currentReadCount:0 writeLockHeld:False readLockHeld:False LockDetails reader lock request /api/v1/sessions/3319/ixnetwork/traffic/trafficItem is blocked by: writer 200 at 5/20/2020 8:44:52 PM: clearStats(kArray[kEnumValue=waitForPortStatsRefresh\\~waitForTrafficStatsRefresh])kVoid\n at SDMCsLib.ReaderWriterLock..ctor (System.String newRequest, System.Boolean readerLock, System.Boolean bypassLock) [0x00034] in <3d657fd98a28481ea4ae0dad8da57771>:0 \n at SDMCsLib.RestService.SdmObject.Read () [0x000de] in <8faf1fb1b99943fcaae26e5c2fc5c6fb>:0 \n at SDMCsLib.RestService.V1ObjectController.Get (System.String path) [0x0001a] in <8faf1fb1b99943fcaae26e5c2fc5c6fb>:0 "}]}
what would be the root cause for those errors?