API ERROR 403 - Githubissues

TheGuardianLight commented 9 months ago

Hello,

I'm trying to access the Netdot API to process data via a python script but I'm getting 403 errors. Even when I enter my credentials in the script.

Do you have a solution?

candlerb commented 9 months ago

https://www.catb.org/~esr/faqs/smart-questions.html#intro

Without seeing your Python script (or a minimal Python script which reproduces the error), or the exact content of the error body and server logs, there is no way we can know what's going on with your system.

We don't even know if you're using a generic HTTP client like requests, or a Netdot-specific client library like netdot on pypi.

TheGuardianLight commented 9 months ago

I use the python requests library. As for my script, I can only give the anonymized version:

import requests
import xmltodict
import json

url = "https://servernetdot.com/netdot/rest/ipblock/"
username = "username"
password = "password"

response = requests.get(url, auth=requests.auth.HTTPBasicAuth(username, password), verify=False)  # Set verify to True in production environment

if response.status_code == 200:
    data = xmltodict.parse(response.text)  # Converts XML to Python dictionary
    json_data = json.dumps(data, indent=4)  # Converts Python dictionary to JSON
    print(json_data)
else:
    print(f"Failed to retrieve data. Error code: {response.status_code}")

candlerb commented 9 months ago

Try printing {response.text} as well. It may give a clue where the error is coming from.

Try looking in your Apache logs (typically access.log, error.log)

TheGuardianLight commented 9 months ago

Here is the answer given by {response.text} :

<html>
<head>
  <!-- Use the path as set in the Apache conf for the netdot root so
       the user can change the location with out editing any code.
       The path needs to be absolute here since the user could pass
       any url and get redirected to login (although right now the
       only files not at the root are the images, and possibly a
       request for /netdot which I think gets translated to /netdot/
       before AuthCookie takes over anyway).
  -->
  <link rel="StyleSheet" href="/netdot/css/style.css" type="text/css" />
  <link rel="StyleSheet" href="/netdot/css/datechooser.css" type="text/css" />
  <link rel="Shortcut Icon" href="/netdot/img/favicon.ico" type="image/x-icon" />
  <title>Netdot @ servernetdot.com: Netdot Login</title>
</head>
<script language="JavaScript" src="/netdot/java_script/dynamic_list.js"></script>
<script language="JavaScript" src="/netdot/java_script/select.js"></script>
<script language="JavaScript" src="/netdot/java_script/jsrsClient.js"></script>
<script language="JavaScript" src="/netdot/java_script/toggle.js"></script>
<script language="JavaScript" src="/netdot/java_script/datechooser.js"></script>
<body>
<div id="header" >
    <a href="../index.html" ><img border="0" class="headleft" SRC="/netdot/img/title.png"
        alt="Netdot: Network Documentation Tool" /></a>
    <span class="headright">
    </span>
    <hr />
    <span class="headleft">servernetdot.com</span>
    <span class="headright">Tue Feb  6 11:16:49 2024</span>
</div>

    <div id="content">

<div class="sectiondetail" align="center">

<!-- 
Use the path as set in the Apache conf for the netdot root so the user
can change the location with out editing any code.  The path needs to
be absolute here since the user could pass any url and get redirected
to login (although right now the only files not at the root are the
images, and possibly a request for /netdot which I think gets
translated to /netdot/ before AuthCookie takes over anyway).
-->

<div id="loginform" align="center" style="margin:0 auto;">

<div class="container">
    <div class="containerhead">
        Please enter your login and password to authenticate.
    </div>
    <div class="containerbody">
      <fieldset class="medium" id="sectiontools" style="border: none">
    <form method="post" action="/netdot/NetdotLogin" name="login">
    <input type="hidden" name="destination" value="/netdot/rest/ipblock" />
    <p>
      <label for="credential_0">Login:</label>
      <input type="text" name="credential_0" size="15" />
    </p>
    <p>
      <label for="credential_1">Password:</label>
      <input type="password" name="credential_1" size="15" />
    </p>
        <p>
      <lable for="permanent_session">Remember me:</lable>
      <input type="checkbox" name="permanent_session" value="1" />
        </p>
    <p>
          <input type="submit" value="Continue" />
    </p>
    </form>
    </fieldset> 
  </div>
</div>

</div>

</div>

<script type="text/javascript">
    document.login.credential_0.focus();
</script>

    </div> <!-- close content -->
    <div id="footer">
    <p>
        &copy GPL. 
        <a href="http://netdot.uoregon.edu">Netdot: NETwork DOcumentation Tool</a> 
        v.1.0.4 
    </p>
</div>
</body>
</html>

For Apache logs, I'm waiting for my manager to give me access to the machine itself.

candlerb commented 9 months ago

OK, so it wants you to login via a form. It looks like it's not happy to accept HTTP Basic Auth.

I don't have a running Netdot instance to test with, but I do note that the python 'netdot' library I referred you to before uses the login form to authenticate (in src/netdot/client.py):

    def _login(self, username, password):
        """Log into the NetDot API with provided credentials.
        Stores the generated cookies to be reused in future API calls.
        """
        params = {
            'destination': 'index.html',
            'credential_0': username,
            'credential_1': password,
            'permanent_session': 1,
        }
        try:
            response = self.http.post(self.login_url, data=params, timeout=self.timeout)
        except ConnectionError:
            raise exceptions.NetdotLoginError(
                f'Unable to reach to Netdot server: {self.server} (Maybe you need to use a VPN?)'
            )
        if response.status_code != 200:
            raise exceptions.NetdotLoginError(
                f'Login failed. Most likely caused by invalid credentials for user: {username}'
            )

If you want to login this way using the requests library, you'll want to use a Session so that the cookie is retained and passed to subsequent requests.

The other thing I note from that code is that it sets an Accept header asking for XML:

        self.http.headers.update(
            {
                'User_Agent': 'Netdot::Client::REST/self.version',
                'Accept': 'text/xml; version=1.0',
            }
        )

(Yes, it does appear to send the literal string Netdot::Client:Rest/self.version as the User-Agent!)

TheGuardianLight commented 9 months ago

I've updated my code to look like this:

import requests
import xmltodict
import json

class NetDotClient:
    def __init__(self, base_url='https://netdotserver.com/netdot/rest/', timeout=5):
        self.server = base_url
        self.timeout = timeout
        self.http = requests.Session()  # creates a session
        self.username = "username"
        self.password = "password"
        self.http.auth = (self.username, self.password)  # sets HTTP Basic Auth for the session

        self.http.headers.update(
            {
                'User_Agent': 'Netdot::Client::REST/1.0',  # assuming that 1.0 is your version
                'Accept': 'text/xml; version=1.0',
            }
        )

    def get_ipblock(self):
        url = self.server + 'ipblock'
        try:
            response = self.http.get(url, timeout=self.timeout)
            response.raise_for_status()
        except requests.RequestException as e:
            print(f"An error occurred while trying to retrieve ipblock: {e}")
            return None

        data = xmltodict.parse(response.text)  # Converts XML to Python dictionary
        json_data = json.dumps(data, indent=4)  # Converts Python dictionary to JSON
        print(json_data)

if __name__ == "__main__":
    client = NetDotClient()
    client.get_ipblock()

Despite this, it continues to give me a 403 error.

candlerb commented 9 months ago

You'll need to make a http.post with the login form details, like the code I showed you.

cvicente / Netdot

API ERROR 403 #181