search

cubewise-code / tm1py

TM1py is a Python package that wraps the TM1 REST API in a simple to use library.
http://tm1py.readthedocs.io/en/latest/
MIT License
190 stars 110 forks source link

Unauthorized error while accessing tm1 cube data with tm1py #124

Closed srinivas1703 closed 4 years ago

srinivas1703 commented 5 years ago

I am getting below error while accessing TM1 cube data using TM1py. Intigrated security mode:5 While accessing it is showing connection established and returning Application server name.

"TM1py.Exceptions.Exceptions.TM1pyException: Text: Status Code: 401 Reason: Unauthorized".

vviau commented 5 years ago

Hi @srinivas1703 ,

We will need more information, this message just tells us that TM1py can't connect.

What is your TM1 version?

Have you tried to test your credentials using Check.py as it is described here: https://code.cubewise.com/tm1py-help-content/check-connectivity-with-tm1

By the way, the CAM Namespace is case sensitive, just make sure it has the same case as it is configured in your Cognos configuration.

I hope this help.

Cheers,

Vincent

srinivas1703 commented 5 years ago

Hi Team,

please find below information for your reference.

  1. TM1 Version : 10.2.2
  2. I tried with "Check.py" and returned exact TM1 instance.
  3. for CAM namespace correctly entered.

Thanks & Regards, Srinivas.

On Wed, Apr 10, 2019 at 12:38 PM vviau notifications@github.com wrote:

Hi @srinivas1703 https://github.com/srinivas1703 ,

We will need more information, this message just tells us that TM1py can't connect.

What is your TM1 version?

Have you tried to test your credentials using Check.py as it is described here: https://code.cubewise.com/tm1py-help-content/check-connectivity-with-tm1

By the way, the CAM Namespace is case sensitive, just make sure it has the same case as it is configured in your Cognos configuration.

I hope this help.

Cheers,

Vincent

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cubewise-code/tm1py/issues/124#issuecomment-481563010, or mute the thread https://github.com/notifications/unsubscribe-auth/Ak7anVHKxtfe3kSBHJ5aurLtmHrWqvRjks5vfY3zgaJpZM4cmKXm .

-- Thanks and Regards,

christianthieme commented 5 years ago

@MariusWirtz, @vviau ,

This is the exact problem I am running into. I'm running 10.2.2 FP5. I'm running TM1py version 1.2.1 and Cognos BI 10.1.1. We are also using Intigrated security mode:5 with SSO. My config file looks like this:

[tm1admtest1] address= tm1admtest1.company.com port= 34100 user= password= namespace = TM1 8 Namespace gateway = https://C11.COMPANY.COM/ibmcognos/cgi-bin/cognosisapi.dll ssl=True

I opened a question on the other page with the samples here: https://github.com/cubewise-code/tm1py-samples/issues/44

All the information above has been added correctly. I've also tested this against a server without single sign on and it authenticates, although the test server is running a newer version of TM1 and Cognos BI - but I would think its a SSO issue.

MariusWirtz commented 5 years ago

Hi @christianthieme ,

most likely you get this error because the latest TM1py version at GitHub does not work with a gateway that uses HTTPS.

As discussed in #121, as a workaround for the moment you could do two things.

  1. Provide a ClientCAMURI that uses HTTP instead of HTTPS.

  2. On the python side you could switch off ssl verification for the request against the CAM gateway. You could change line 278 in the RestService to: response = requests.get(gateway, auth=HttpNegotiateAuth(), verify=False)

Let me know if it works.

We will provide a fix for TM1py so that could you could pass a certificate for the CAM gateway, when instantiating the TM1Service. I created a seperate issue for this feature here #123

christianthieme commented 5 years ago

Hi @MariusWirtz ,

Ok, a couple of notes first:

When I run the code without adding verify = False to line 278 I get: SSLError: HTTPSConnectionPool(host='smlogin.company.com', port=443): Max retries exceeded with url: /siteminderagent/forms/login.aspx?TYPE=33554433&REALMOID=06-000e7588-90aa-118a-9881-80a65e850000&GUID=&SMAUTHREASON=0&METHOD=GET&SMAGENTNAME=-SM-cognosprod.micron.com&TARGET=-SM-http%3a%2f%2fc10test.micron.com%2fibmcognos%2fcgi--bin%2fcognosisapi.dll (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)')))

So i went into the script and changed line 278 to include verify = False and I get:

RuntimeError: Failed to authenticate through CAM. HTTP response does not contain 'cam_passport' cookie

Again this is what my config file looks like: 

[comptm1admtest1]
address= comptm1admtest1.company.com
port= 65410
user= 
password= 
namespace = Cognos 7 
gateway = http://C1TEST.COMPANY.COM/ibmcognos/cgi-bin/cognosisapi.dll
ssl=True

We are using SSO, so i left user and password blank but tested it with my credentials and got the same error saying it failed to authenticate through CAM.

Not to rush anybody - really appreciate the great work you guys do - do you guys have a timeline on the fix? Our company is evaluating whether to just purchase a 3rd party connector to access TM1 data and I've been trying to show we can do the same thing with python, so I'd like to be able to give them a window of time for my experimenting. :)

Thanks again for all your help. Really appreciate it.

Best, Christian

srinivas1703 commented 5 years ago

Hi Team,

While getting the Cube data I am getting error(401 error). when i check the with "Check.py", i am getting server instance. please find below info for your reference.

TM1 User: Warning (from warnings module): File "C:\Python37\lib\getpass.py", line 100 return fallback_getpass(prompt, stream) GetPassWarning: Can not control echo on the terminal. Warning: Password input may be echoed. Password (cmd doesn't show input): CAM Namespace (leave empty if no CAM Security): Active_Directory Address (leave empty if localhost): Port: 8881 SSL (True or False): False Connection to TM1 established!! your Servername is: Test_serv

I was running TM1py 1.2.1 which doesn't have a line 278 in the RESTServices.py so i uninstalled that in downloaded the one on github and installed.. Can you send which dependencies modules I have to install. I installed "pypiwin32-223, pywin32-224, requests-negotiate-sspi-0.5.2" modules and again i am getting the module "pywintypes" not found.

Please help in this issue.

Thanks & Regards Srinivas.

Thanks & Regarards, Srinivas.

On Fri, Apr 19, 2019 at 12:53 AM Christian Thieme notifications@github.com wrote:

Hi @MariusWirtz https://github.com/MariusWirtz ,

Ok, a couple of notes first:

  • Apparently in some of my testing I changed the gateway to https, but the actual gateway is: http://C1TEST.COMPANY.COM/ibmcognos/cgi-bin/cognosisapi.dll I was running TM1py 1.2.1 which doesn't have a line 278 in the RESTServices.py so i uninstalled that in downloaded the one on github and installed the dependencies.

When I run the code without adding verify = False to line 278 I get: SSLError: HTTPSConnectionPool(host='smlogin.company.com', port=443): Max retries exceeded with url: /siteminderagent/forms/login.aspx?TYPE=33554433&REALMOID=06-000e7588-90aa-118a-9881-80a65e850000&GUID=&SMAUTHREASON=0&METHOD=GET&SMAGENTNAME=- SM-cognosprod.micron.com&TARGET=-SM-http%3a%2f%2fc10test.micron.com%2fibmcognos%2fcgi--bin%2fcognosisapi.dll (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)')))

So i went into the script and changed line 278 to include verify = False and I get:

RuntimeError: Failed to authenticate through CAM. HTTP response does not contain 'cam_passport' cookie

Again this is what my config file looks like: [comptm1admtest1] address= comptm1admtest1.company.com port= 65410 user= password= namespace = Cognos 7 gateway = http://C1TEST.COMPANY.COM/ibmcognos/cgi-bin/cognosisapi.dll ssl=True

We are using SSO, so i left user and password blank but tested it with my credentials and got the same error saying it failed to authenticate through CAM.

Not to rush anybody - really appreciate the great work you guys do - do you guys have a timeline on the fix? Our company is evaluating whether to just purchase a 3rd party connector to access TM1 data and I've been trying to show we can do the same thing with python, so I'd like to be able to give them a window of time for my experimenting. :)

Thanks again for all your help. Really appreciate it.

Best, Christian

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cubewise-code/tm1py/issues/124#issuecomment-484652648, or mute the thread https://github.com/notifications/unsubscribe-auth/AJHNVHLXJ4GSAZ3D6C4A6UDPRDDCLANCNFSM4HEYUXTA .

-- Thanks and Regards,

MariusWirtz commented 5 years ago

Hi @christianthieme,

OK. Good. So we have not officially released the SSO version of TM1py (1.2.2) currently you can only get it from GitHub. We will release it in the next weeks once we got all these issues around SSO solved.

Just to get this possibility out of the way: Is it perhaps an option for your case to simply not use SSO but authenticate with user, password and cam-namespace? This would of course simplify things. One way to implement this could be that people who run the python scripts could store their credentials (b64 encoded) as an environment variable on their local systems. This way they wouldn't need to type it in any time, and it would be (slightly) safer than storing credentials in a text file (assuming that it is done on an internal network, behind a firewall; etc.)

Regarding the error, I have seen the RuntimeError: Failed to authenticate through CAM. HTTP response does not contain 'cam_passport' cookie message before. I saw it when I was playing around with the URL for the gateway argument. Could it be that you are passing an incorrect URL? You need to assign the same value that is specified in the tm1s.cfg as ClientCAMURI. Can you doublecheck if the URL is correct?

Unfortunately, I can not commit to a timeline. We support TM1py in our spare time and sometimes the consulting work occupies more of our time. So far, we have always managed to fix bugs in a matter of a few days or weeks though!

Cheers,

Marius

MariusWirtz commented 5 years ago

Hi @srinivas1703,

you need to install: requests pandas pytz requests_negotiate_sspi

The last dependency was introcuded in version 1.2.2. You can install requests_negotiate_sspiwith pip. Just go to C:/Anaconda3/Scripts (or wherever you installed python) and run: pip install requests_negotiate_sspi

let me know how it goes!

Once we have released 1.2.2 and published it to PyPI you could simply install TM1py and would not need to worry about the dependencies.

srinivas1703 commented 5 years ago

Hi Team,

Thanks for your quick response. 1.First I installed tm1py 1.2.1 version and tested "check.py" and successfully retrieve the tm1 server instance. 2.when i was extracting tm1 cube view data I was getting 401 error(unauthorized user). 3.Next I replaced RESTService.py with github RestService.py and check with "Check.py" and successfully retrieve the tm1 server instance.

  1. Installed dependent modules requests pands pytz requests_negotiate_sspi

  2. I am using below parameters in config file to retrieve cube view data and getting the error (401 unauthorized)

    1. user 2.password 3.namespace 4.address 5.port 6.ssl Code: from TM1py.Services import TM1Service

    with TM1Service(*config['Test_Dev']) as tm1: all_dimensions=tm1.dimensions.get_all_names() print(all_dimensions) Error:* Traceback (most recent call last): File "C:\srinivas\26 03 2019\TM1DataExtract\getCube.py", line 12, in

    with TM1Service(address=,port=,user='',password='',ssl=False) as tm1: File "C:\Python37\lib\site-packages\TM1py\Services\TM1Service.py", line 12, in __init__ self._tm1_rest = RESTService(**kwargs) File "C:\Python37\lib\site-packages\TM1py\Services\RESTService.py", line 116, in __init__ decode_b64=self.translate_to_boolean(kwargs.get("decode_b64", False))) File "C:\Python37\lib\site-packages\TM1py\Services\RESTService.py", line 201, in _start_session response = self.GET(request=request) File "C:\Python37\lib\site-packages\TM1py\Services\RESTService.py", line 33, in wrapper self.verify_response(response=response) File "C:\Python37\lib\site-packages\TM1py\Services\RESTService.py", line 270, in verify_response raise TM1pyException(response.text, status_code=response.status_code, reason=response.reason) TM1py.Exceptions.Exceptions.TM1pyException: Text: Status Code: 401 Reason: Unauthorized

if i use ClientCAMURI what is the parameter to pass this value

Please help this issue. Thanks & Regards, Srinivas.

On Tue, Apr 23, 2019 at 12:14 AM Marius Wirtz notifications@github.com wrote:

Hi @srinivas1703 https://github.com/srinivas1703,

you need to install: requests pandas pytz requests_negotiate_sspi

The last dependency was introcuded in version 1.2.2. You can install requests_negotiate_sspi with pip. Just go to C:/Anaconda3/Scripts (or wherever you installed python) and run: pip install requests_negotiate_sspi

let me know how it goes!

Once we have released 1.2.2 and published it to PyPI you could simply install TM1py and would not need to worry about the dependencies.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cubewise-code/tm1py/issues/124#issuecomment-485509407, or mute the thread https://github.com/notifications/unsubscribe-auth/AJHNVHIRVVUUFZTYR6G7UQ3PRYBQTANCNFSM4HEYUXTA .

srinivas1703 commented 5 years ago

HI team, Any update regrading the Unauthorized issue.

Thanks & Regards, Srinivas.

srinivas1703 commented 5 years ago

HI team, Any update regrading the Unauthorized issue.

Thanks & Regards, Srinivas.

MariusWirtz commented 5 years ago

Hi, Sorry to delay this but im busy this week. Will look into your topic towards the end of the week or next week.

christianthieme commented 5 years ago

@MariusWirtz ,

Just wanted to give an update for anyone who may be working inside of a large company with lots of security. Many of these companies will have software that sits on top of your Cognos and TM1 servers that block API calls. If you are using CAM for security, you are trying to get a CAMPassport back from your API call so you can pass it to TM1 to authenticate. For me, this has been the issue. We have a software sitting on top of the Cognos server that blocks all API calls and so to get the CAMPassport from the cookies, you have to open a browser and extract it. I was able to get around this issue by using a combination of tools: Google's Headless Browser and Selenium's web driver. I'll share my code below for anyone who may face the same challenges. This essentially allows Google to open up an untraceable web page in the background routed to your specific CAMURI, at which point you can extract the CAMPassport from the cookies using Selenium. What I did was replace the code from the _build_authorization_token function starting on line 275 of the RESTService.py with the following:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--window-size=1920x1080")
driver = webdriver.Chrome(chrome_options = chrome_options, executable_path = r'C:\Users\xxxxx\Documents\xxxxxx\Python Stuff\chromedriver.exe')
driver.get('http://company.com/ibmcognos/cognosisapi.dll')
cookie_dictionary = driver.get_cookie('cam_passport')
cookie = cookie_dictionary['value']
return 'CAMPassport' + ' '  + cookie

To run this code, you'll need to download the google chrome web driver: https://sites.google.com/a/chromium.org/chromedriver/downloads

You'll also need to install Selenium, which can be done with the pip command.

This is an article I used that was helpful: https://medium.com/@pyzzled/running-headless-chrome-with-selenium-in-python-3f42d1f5ff1d

Best, Christian

MariusWirtz commented 5 years ago

Hi @christianthieme ,

happy to hear that you found a solution and thanks for sharing it with the community !

MariusWirtz commented 5 years ago

Hi @srinivas1703,

I understand from your response that the problem you have is that you don't know what value to provide for the gateway parameter.

In the config.ini file or during the instantiation of the TM1Service, you need to pass the gateway parameter. For the gateway you need to assign the same value that is specified in the tm1s.cfg as ClientCAMURI

If this is not your problem please post your code, the error message and the config.ini file.

srinivas1703 commented 5 years ago

Hi Team

Thanks for your support. Please find the Code and error log for you reference. Please find attached tm1s.cfg and RESTService.py file.

Code:

from TM1py.Services import TM1Service

with TM1Service(address='cognostm1.inedc.corpintra.net ',port=8881,user='dusrini',password='Password@12345',ssl=False,gateway=" http://cognostm1.inedc.corpintra.net:9300/bi/v1/disp") as tm1:

c = tm1.cubes.get('GL Summary Budget Reporting Currency')

print(c.name)

print(c.dimensions)

error:

Traceback (most recent call last):

File "C:\srinivas\26 03 2019\TM1DataExtract\getCube.py", line 12, in

with TM1Service(address='cognostm1.inedc.corpintra.net ',port=88981,user='dusrini',password='Password@12345',ssl=False, gateway=" http://cognostm1.inedc.corpintra.net:9300/bi/v1/disp") as tm1: File "C:\Python37\lib\site-packages\TM1py\Services\TM1Service.py", line 12, in __init__ self._tm1_rest = RESTService(**kwargs) File "C:\Python37\lib\site-packages\TM1py\Services\RESTService.py", line 116, in __init__ decode_b64=self.translate_to_boolean(kwargs.get("decode_b64", False))) File "C:\Python37\lib\site-packages\TM1py\Services\RESTService.py", line 201, in _start_session response = self.GET(request=request) File "C:\Python37\lib\site-packages\TM1py\Services\RESTService.py", line 33, in wrapper self.verify_response(response=response) File "C:\Python37\lib\site-packages\TM1py\Services\RESTService.py", line 270, in verify_response raise TM1pyException(response.text, status_code=response.status_code, reason=response.reason) TM1py.Exceptions.Exceptions.TM1pyException: Text: Status Code: 401 Reason: Unauthorized For Check.Py i am getting TM1 server instance. *TM1 User: dusrini* *Password (cmd doesn't show input): Password@12345* *CAM Namespace (leave empty if no CAM Security): Active_Directory* *Address (leave empty if localhost): cognostm1.inedc.corpintra.net * *Port: 8881* *SSL (True or False): False* *Connection to TM1 established!! your Servername is: Drishti_DEV* Thanks & Regards, Srinvias. On Mon, May 20, 2019 at 1:38 AM Marius Wirtz wrote: > Hi @srinivas1703 , > > I understand from your response that the problem you have is that you > don't know what value to provide for the gateway parameter. > > In the config.ini file or during the instantiation of the TM1Service, you > need to pass the gateway parameter. > For the gateway you need to assign the same value that is specified in > the tm1s.cfg as ClientCAMURI > > If this is not your problem please post your code, the error message and > the config.ini file. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > , > or mute the thread > > . > -- Thanks and Regards,
kolmssi commented 4 years ago

Hi @MariusWirtz & @christianthieme,

We still had the problem with integratedsecuritymode 5 and cam_passport cookie when having two domains configured to the gateway. Added parameter "CAMNamespace" as replaicing response (currently line 349) from _build_authorization_token_cam: response = requests.get(gateway, auth=HttpNegotiateAuth(), verify=verify) to response = requests.get(gateway, auth=HttpNegotiateAuth(), verify=verify, params={"CAMNamespace": namespace})

Best, Juho

MariusWirtz commented 4 years ago

Hi @kolmssi,

Thanks! I added your change to the master branch.

AlexanderDvoynev commented 2 years ago

Hi @christianthieme, thanks for sharing! This worked for me with 1 extra row of code: 

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--window-size=1920x1080")
driver = webdriver.Chrome(chrome_options = chrome_options, executable_path = r'C:\Users\xxxxx\Documents\xxxxxx\Python Stuff\chromedriver.exe')
driver.get('http://company.com/ibmcognos/cognosisapi.dll')
time.sleep(3)
cookie_dictionary = driver.get_cookie('cam_passport')
cookie = cookie_dictionary['value']
return 'CAMPassport' + ' '  + cookie

It looks like it takes time for Cognos to perform all the redirections, so without sleep it returns empty cookies

MariusWirtz commented 2 years ago

@AlexanderDvoynev Interesting finding. Thank you!

Do you think it makes sense to include this fallback login approach in the TM1py codebase? Perhaps as a utils function with the selenium import as an optional dependency.

macsir commented 2 years ago

Never realised selenium can be used here. Good catch!

Get Outlook for iOShttps://aka.ms/o0ukef

From: Marius Wirtz @.> Sent: Tuesday, February 8, 2022 9:01:18 PM To: cubewise-code/tm1py @.> Cc: Subscribed @.***> Subject: Re: [cubewise-code/tm1py] Unauthorized error while accessing tm1 cube data with tm1py (#124)

@AlexanderDvoynevhttps://github.com/AlexanderDvoynev Interesting finding. Thank you!

Do you think it makes sense to include this fallback login approach in the TM1py codebase? Perhaps as a utils function with the selenium import as an optional dependency.

— Reply to this email directly, view it on GitHubhttps://github.com/cubewise-code/tm1py/issues/124#issuecomment-1032480710, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAZQV7NYIS3YOHIZ6KX52HDU2DZX5ANCNFSM4HEYUXTA. You are receiving this because you are subscribed to this thread.Message ID: @.***>