LibreQoE / LibreQoS

A Quality of Experience and Smart Queue Management system for ISPs. Leverage CAKE to improve network responsiveness, enforce bandwidth plans, and reduce bufferbloat.
https://libreqos.io/
GNU General Public License v2.0
464 stars 50 forks source link

UISP Integration issue with LibreQOS #347

Closed gsrathore02 closed 1 month ago

gsrathore02 commented 1 year ago

We are implementing LibreQOS in the Lab environment and facing some issues with python related any idea about below mentioned error?

p9ldNR1BYKyDGesB

root@LibreQOS:/opt/libreqos/src# python3 integrationUISP.py Running Python Version 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] Loading Data from UISP Traceback (most recent call last): File "/opt/libreqos/src/integrationUISP.py", line 486, in importFromUISP() File "/opt/libreqos/src/integrationUISP.py", line 479, in importFromUISP case "full": buildFullGraph() File "/opt/libreqos/src/integrationUISP.py", line 314, in buildFullGraph sites = uispRequest("sites") File "/opt/libreqos/src/integrationUISP.py", line 22, in uispRequest return r.json() File "/usr/lib/python3/dist-packages/requests/models.py", line 900, in json return complexjson.loads(self.text, **kwargs) File "/usr/lib/python3/dist-packages/simplejson/init.py", line 525, in loads return _default_decoder.decode(s) File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode obj, end = self.raw_decode(s) File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode return self.scan_once(s, idx=_w(s, idx).end()) simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

trendal commented 1 year ago

That looks like it's complaining about malformed json from the uisp API.

Line 1, column 1 would lead me to believe that it's not returning anything. Maybe you have bad credentials or wrong url and the integration script isn't catching that?

gsrathore02 commented 1 year ago

That looks like it's complaining about malformed json from the uisp API.

Line 1, column 1 would lead me to believe that it's not returning anything. Maybe you have bad credentials or wrong url and the integration script isn't catching that?

I have resolved earlier showing credential issues (wrong token) and bad URL issues, but now I am facing the above error.

trendal commented 1 year ago

We have 3 instances of LibreQoS here and I'm not seeing that issue so I'm assuming it's unique to your environment. That error message still strongly points towards an empty query result from your UISP instance. Is your UISP instance working correctly of itself? If so I doubt that it is returning invalid JSON to LibreQoS because it uses the same API. It would seem highly likely there is a connection error that is not getting caught in higher level error checking.

gsrathore02 commented 1 year ago

Currently, I am working with one network card only, that issue is related to the network card? How can we check UISP instance working correctly ?

trendal commented 1 year ago

No the issue I am presuming your having is not with the network card.

Are you able to edit the integrationUISP.py file for testing purposes or do you not have any experience in Python programming? What needs to happen is we need to add a line in that prints out the raw data returned from the UISP API before it's sent to the JSON decoder.

Doing this via github is going to be a slow process. There is a Matrix Chat here: https://app.element.io/#/room/#libreqos:matrix.org where the developers and other users hang out that would be a lot quicker. If you can't or don't want to use that then it might be advisable to connect on some other more real time messaging service to debug this.

It appears like you have encountered a new bug that is not being handled correctly.

thebracket commented 1 year ago

Checking UISP is a little convoluted:

  1. From SSH in your shaper box, make sure you can ping it by name (to ensure it's not a DNS or routing issue). e.g. "ping uisp.myisp.com" - exactly the name you have in UISPbaseURL in ispConfig.py, but without the https:// part.
  2. Open a browser, and go to https://(my uisp)/nms/api-docs/. Replace (my uisp) with the UISPBaseURL from ispConfig.py. This will bring up a Swagger API definition.
  3. Click Authorize and enter the API key you have in uispAuthToken (from ispConfig.py). Click "OK" and the "close" button.
  4. Click "Sites", click "get /sites" and run the query by clicking "try it out", and then the wide blue "execute" button.

If your token is good, you'll see a list of your sites in JSON format. If the token isn't good, you'll see a short "Unauthorized" message.

That's the same mechanism that the integrationUISP.py system uses - so if it works in the API browser, and access to UISP (checked in step 1) works, then the credentials aren't the problem.

It does look like it's not getting the reply it expects from UISP. Could you also provide which version of both UISP and LibreQoS you are running, please?

gsrathore02 commented 1 year ago

Thanks for your response as I checked UISPBaseUrl, it's showing 404 not found Please find the below-mentioned screenshot

http://192.168.6.79:8086/nms/api-docs/

Screenshot from 2023-05-16 16-16-27

gsrathore02 commented 1 year ago

Thanks for your response.

No, I am not a Python programmer. Can you please give me the troubleshooting step? So, I can easy to solve the error.

trendal commented 1 year ago

On the URL that you showed there, you are using http not https. If that is how you have the server configured that is fine, but typically it is running on a secured port with ssl. Also I see that you are using a different port than the standard 80 or 443. If you are getting a 404 then that is also what the integrationUISP script will be getting. So you need to configure your UISP instance appropriately before it will work.

You should be seeing something like this when you access the api-docs

uisp api

gsrathore02 commented 1 year ago

On the URL that you showed there, you are using http not https. If that is how you have the server configured that is fine, but typically it is running on a secured port with ssl. Also I see that you are using a different port than the standard 80 or 443. If you are getting a 404 then that is also what the integrationUISP script will be getting. So you need to configure your UISP instance appropriately before it will work.

You should be seeing something like this when you access the api-docs

uisp api

https://github.com/LibreQoE/LibreQoS/wiki/v1.4

in above GitHub document not mentioned SSL

and web UI is running on the below URL as per your document.

http://SERVER_IP_ADDRESS:8086

then how can I run that application on SSL(443 port) ?

any alternative document?

trendal commented 1 year ago

Did you follow the install documents for UISP here?: https://help.ui.com/hc/en-us/articles/115012196527-UISP-First-Time-Setup-Installation

rchac commented 1 month ago

Complete with v1.5