Generated Token is not correct

pc-coholic commented 5 years ago

Hello!

Thank you very much for providing this - very useful piece of software :-)

I ran into an issue however, where the generated tokens don't work... In order to get them working, I had to:

Token is created as a (default) 8-digit token, even though it should be an 6-digit token
DefAddPIN is not placed in the template-file, causing the user to be asked to provide a PIN (or 0000 for no PIN).
Token-name is not included in the template

I'm assuming that this might be due to the data transmitted by the server not being exactly formatted as expected by the client... But generally speaking everything else is working after editing the resulting template-file before exporting/importing.

If you are interested, I can send you the -v output of me running the client - would that be helpful to look into why things didn't work as expected?

cemeyer commented 5 years ago

Token is created as a (default) 8-digit token, even though it should be an 6-digit token

This can probably be fixed; yes, -v output may be useful. But I think we already know how to parse that, and maybe just don't do it fully for now. E.g., in the final ProvisioningData, I see <OTPLength>8</OTPLength>. You would probably see 6 instead of 8. We should honor that.

DefAddPIN is not placed in the template-file, causing the user to be asked to provide a PIN (or 0000 for no PIN).

The PIN is created / shared out of band of the rsa_ct_kip protocol. Generally you will set the PIN after this protocol runs. So the generated token is just enough to start using stoken with the computed shared secret; you'll need to acquire / configure the PIN in stoken on your own.

Token-name is not included in the template

This can probably be fixed. Is the token name meaningful in your instance? For me, TokenId is just a decimal string that's mostly uninteresting.

pc-coholic commented 5 years ago

DefAddPIN is not placed in the template-file, causing the user to be asked to provide a PIN (or 0000 for no PIN).

The PIN is created / shared out of band of the rsa_ct_kip protocol. Generally you will set the PIN after this protocol runs. So the generated token is just enough to start using stoken with the computed shared secret; you'll need to acquire / configure the PIN in stoken on your own.

I'm not very well versed in the whole RSA-universe - so perhaps I'm misunderstanding how it's supposed to work... Please bear with me :)

If I'm importing a Token into the Android-App directly using the http://127.0.0.1/securid/ctkip?scheme=https&url=[...]-link, the token gets imported right away and I only need to open the app to get a new OTP.

When running stoken export --random --sdtid --template=template.xml > 838999658504.sdtid, stoken export --android and importing the resulting token back into the Android app, I'm asked to provide a PIN-number before being shown an OTP. Same thing also when just getting a fresh OTP with stoken.

I would assume, that somewhere in the ctkip-process, DefAddPIN is also transmitted. For testing purposes, I added <DefAddPIN>0</DefAddPIN> to the template-file before running the above mentioned two stoken export-commands. The resulting token, would then (correctly) not ask for a PIN before displaying a new OTP (in both stoken and the RSA Android app).

Token-name is not included in the template

This can probably be fixed. Is the token name meaningful in your instance? For me, TokenId is just a decimal string that's mostly uninteresting.

This is not particularly important, no... But it makes it easier to identify the token right away (especially when importing it into the official RSA-app).

In my case, the "Device Nickname" instead of a random string is displayed as the token's name when importing the token using the ctkip-link directly with the Android app. Said nickname was requested by the RSA Self-Service-Console when I created the token. But yeah, that's more of a decorative element ;-)

Is there a specific address I could send you the -v output to? I'm not sure how much "sensitive" data in in the output, so I haven't posted it here...

cemeyer commented 5 years ago

If I'm importing a Token into the Android-App directly using the http://127.0.0.1/securid/ctkip?scheme=https&url=[...]-link, the token gets imported right away and I only need to open the app to get a new OTP.

When running stoken export --random --sdtid --template=template.xml > 838999658504.sdtid, stoken export --android and importing the resulting token back into the Android app, I'm asked to provide a PIN-number before being shown an OTP. Same thing also when just getting a fresh OTP with stoken.

I would assume, that somewhere in the ctkip-process, DefAddPIN is also transmitted. For testing purposes, I added <DefAddPIN>0</DefAddPIN> to the template-file before running the above mentioned two stoken export-commands. The resulting token, would then (correctly) not ask for a PIN before displaying a new OTP (in both stoken and the RSA Android app).

Ah, now I understand better. Yes, the rsa_ct_kip protocol includes this as part of the ProvisioningData, in the form <AddPIN>1</AddPIN>. I guess your server is configured to send <AddPIN>0</AddPIN>, and that this little script doesn't honor that yet. (But I have not dug into the code yet.)

This can probably be fixed. Is the token name meaningful in your instance? For me, TokenId is just a decimal string that's mostly uninteresting.

This is not particularly important, no... But it makes it easier to identify the token right away (especially when importing it into the official RSA-app).

Sure, I expect that can be done (if stoken supports this concept).

In my case, the "Device Nickname" instead of a random string is displayed as the token's name when importing the token using the ctkip-link directly with the Android app. Said nickname was requested by the RSA Self-Service-Console when I created the token. But yeah, that's more of a decorative element ;-)

Hm, I don't see a nickname in the wire protocol logs I have, unless one of the KeyID/TokenID is the nickname and one is the decimal number. If you can share verbose logs with a nickname, that would be very helpful for figuring out how it is expressed.

Is there a specific address I could send you the -v output to? I'm not sure how much "sensitive" data in in the output, so I haven't posted it here...

Sure, please go ahead and send to cem AT freebsd.org. You're correct that the output has some sensitive data in it. Thanks!

cemeyer commented 5 years ago

Ok, here's my current understanding:

Token is created as a (default) 8-digit token, even though it should be an 6-digit token

DefAddPIN is not placed in the template-file, causing the user to be asked to provide a PIN (or 0000 for no PIN).

It looks like both of these should be honored, if they are present (and if the parsing logic is correct): https://github.com/dlenski/rsa_ct_kip/blob/master/rsa_ct_kip/client.py#L154-L188

However, they default to 8 and 1, respectively, if not found. I'll check out your logs and try to determine why we didn't find them. (Edit: it turns out the -v logs don't actually print out the raw final response, just a pointer to the Python object. I'll file a PR to fix that.)

Token-name is not included in the template

It looks like this should be added in TKNBatch/TKN/TokenAttributes/Nickname, once we figure out which field should populate it. But, caveat, stoken itself doesn't do anything with this field yet. You'd want to follow up with the stoken project about showing the nickname somewhere in the client or gui.

dlenski commented 5 years ago

Sorry I'm late to the (very useful) discussion. I'm glad this is getting used!

Token is created as a (default) 8-digit token, even though it should be an 6-digit token

DefAddPIN is not placed in the template-file, causing the user to be asked to provide a PIN (or 0000 for no PIN).

Token-name is not included in the template

The client script…

does extract the OTPLength parameter (8 vs 6 digits),
does extract the AddPIN parameter (0/1), and
does extract both the TokenID and KeyID parameters from the CT-KIP server's output

And the client script does populate all of these in the template output, except for the KeyID (which is always exactly the same as TokenID in the logs I have from real RSA servers), but we certainly could figure out a unique place to put that value like the nickname, as @cemeyer suggested.

Source for the above: https://github.com/dlenski/rsa_ct_kip/blob/master/rsa_ct_kip/client.py#L172-L190

So what am I missing? ¯\_(ツ)_/¯

Did you find any reason why the client script is failing to parse these parameters from the server response, while getting the important bits right (token secret)?

(@pc-coholic, if you can send your logs to me as well at the address on my profile, that'd be helpful.)

pc-coholic commented 5 years ago

Hey :)

I've reached my token limit for the time being, so until late October or so I won't be able to request a new token and log all the requested information - the log apparently does not contain the actually important bits of information... Sorry...

If any of my colleges is requesting a token in the meantime, I'll make sure to catch all the information :)

dlenski commented 5 years ago

Thanks, @pc-coholic.

I've reached my token limit for the time being, so until late October or so I won't be able to request a new token and log all the requested information

Argh. <sarcasm>Isn't RSA a fun black-box to play with?</sarcasm>

the log apparently does not contain the actually important bits of information... Sorry...

If any of my colleges is requesting a token in the meantime, I'll make sure to catch all the information :)

That was my fault. Derp. Fixed in #5, but too late. The latest version should log all the real contents of the exchange correctly, so let us know if you get more data.

cemeyer commented 5 years ago

I'm pretty happy at the progress we've made so far :-). Two years ago we couldn't negotiate a correct secret with the CT-KIP URL scheme, and for the last couple of months, we can at least do that.

Parsing XML correctly, pfff. I'm sorry it's broken (truly), but it is at least fixable by hand before stoken import. It'll be 50-50 some trivial bug in the Python client or a misbehaving RSA server (producing incorrect values for your org/key configuration), once we have the dump of that 2nd exchange. (The latter case we couldn't really fix; hopefully it's the former.)

dlenski commented 5 years ago

@cemeyer wrote:

It'll be 50-50 some trivial bug in the Python client or a misbehaving RSA server (producing incorrect values for your org/key configuration), once we have the dump of that 2nd exchange.

Basically, client.py assumes that the server's reply fits this XML structure in terms of tag and attribute naming and nesting (bits we want to extract in braces {}), because it's the only structure that I've seen emitted by a real server:

<?xml version="1.0" encoding="UTF-8"?><ServerFinished xmlns="http://www.rsasecurity.com/rsalabs/otps/schemas/2005/12/ct-kip#" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" SessionID="{sess}" Status="Success"><TokenID xmlns="">{token_id}</TokenID>
<KeyID xmlns="">{key_id}</KeyID>
<KeyExpiryDate xmlns="">{exp_date}</KeyExpiryDate>
<ServiceID xmlns="">{service_id}</ServiceID>
<UserID xmlns="">{user_id}</UserID>
<Extensions xmlns="" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Extension Critical="true" xmlns="" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><OTPFormat>{token_digit_type}</OTPFormat>
<OTPLength>{token_digit_length}</OTPLength>
<OTPMode xmlns="http://www.rsasecurity.com/rsalabs/otps/schemas/2005/12/ct-kip#" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Time TimeInterval="60" xmlns="http://www.rsasecurity.com/rsalabs/otps/schemas/2005/12/ct-kip#" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
</OTPMode>
</Extension>
</Extensions>
<Mac xmlns="" MacAlgorithm="http://www.rsasecurity.com/rsalabs/otps/schemas/2005/12/ct-kip#ct-kip-prf-aes" xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">{MAC}</Mac>
</ServerFinished>

Now, client.py handles this by actually parsing XML… but I have good reason to believe that the real RSA client doesn't really parse the "inner" de-SOAPified XML, but just divides things up by line endings and then extracts various bits using regexps or something similarly bogus (see 559b0eaf5025f9766f819e40b4f82ce4247aa6f0 commit message).

So, it's possible that some of the servers emit a slightly different XML struct, and the client doesn't even notice because it's so bogusly written. :man_facepalming:

… once we have the dump of that 2nd exchange.

We should be able to deal with whatever XML structure the real RSA servers are emitting, as long as we have a log of it.

cemeyer commented 5 years ago

@dlenski Yeah, exactly. The other option is the server is actually telling the client "no, your token is 8 digits and DefAddPin is 0," and it's just broken on that end.

Sacrin commented 5 years ago

Guys,

I'm trying to import token and getting the same issue with the length of the code and I suspect that response from the server parsed in a wrong way. I used the "-v" version and you clearly see in Server's response that it gives Interval = 30 seconds and OTPLength=6 , while the client.py uses values of 8 digits and 60 seconds interval.

Part of verbose output:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soapenv:Body>
    <ServerResponse xmlns="http://ctkipservice.rsasecurity.com">
      <AuthData>++ACTIVATION CODE++</AuthData>
      <ProvisioningData>++ OMITTED ++
      </ProvisioningData>
      <Response>PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz48U2VydmVyRmluaXNoZWQgeG1s bnM9Imh0dHA6Ly93d3cucnNhc2VjdXJpdHkuY29tL3JzYWxhYnMvb3Rwcy9zY2hlbWFzLzIwMDUv MTIvY3Qta2lwIyIgeG1sbnM6ZHM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvMDkveG1sZHNpZyMi
        IHhtbG5zOnhzaT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiIFNl c3Npb25JRD0iNDUwYmM0MWFiMTk1OWNiODk1MWI3ZDcwOGM4NzZmNmEzYi12ZkRQci8xTXd1RzA1 dzkwaXJDemRLa3Q1YWJNV1d4V1pkcU1KYnNha1dGdis1eERZSzBZR1BJSkFuY0x5bm9oS0lOY0d3
        K3FGeTIyIiBTdGF0dXM9IlN1Y2Nlc3MiPjxUb2tlbklEIHhtbG5zPSIiPk1EQXdOVEEyT1RZeE5U TXc8L1Rva2VuSUQ+CjxLZXlJRCB4bWxucz0iIj5NREF3TlRBMk9UWXhOVE13PC9LZXlJRD4KPEtl eUV4cGlyeURhdGUgeG1sbnM9IiI+MjAyMS0wNC0zMFQwMDowMDowMCswMDowMDwvS2V5RXhwaXJ5
        RGF0ZT4KPFNlcnZpY2VJRCB4bWxucz0iIj5SU0EgQ1QtS0lQPC9TZXJ2aWNlSUQ+CjxVc2VySUQg eG1sbnM9IiIvPgo8RXh0ZW5zaW9ucyB4bWxucz0iIiB4bWxuczpkcz0iaHR0cDovL3d3dy53My5v cmcvMjAwMC8wOS94bWxkc2lnIyIgeG1sbnM6eHNpPSJodHRwOi8vd3d3LnczLm9yZy8yMDAxL1hN
        TFNjaGVtYS1pbnN0YW5jZSI+PEV4dGVuc2lvbiBDcml0aWNhbD0idHJ1ZSIgeG1sbnM9IiIgeG1s bnM6ZHM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvMDkveG1sZHNpZyMiIHhtbG5zOnhzaT0iaHR0 cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiPjxPVFBGb3JtYXQ+RGVjaW1h
        bDwvT1RQRm9ybWF0Pgo8T1RQTGVuZ3RoPjY8L09UUExlbmd0aD4KPE9UUE1vZGUgeG1sbnM9Imh0 dHA6Ly93d3cucnNhc2VjdXJpdHkuY29tL3JzYWxhYnMvb3Rwcy9zY2hlbWFzLzIwMDUvMTIvY3Qt a2lwIyIgeG1sbnM6ZHM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvMDkveG1sZHNpZyMiIHhtbG5z
        OnhzaT0iaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiPjxUaW1lIFRp bWVJbnRlcnZhbD0iMzAiIHhtbG5zPSJodHRwOi8vd3d3LnJzYXNlY3VyaXR5LmNvbS9yc2FsYWJz L290cHMvc2NoZW1hcy8yMDA1LzEyL2N0LWtpcCMiIHhtbG5zOmRzPSJodHRwOi8vd3d3LnczLm9y
        ++OMITTED++
        YWxhYnMvb3Rwcy9zY2hlbWFzLzIwMDUvMTIvY3Qta2lwI2N0LWtpcC1wcmYtYWVzIiB4bWxuczpk cz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC8wOS94bWxkc2lnIyIgeG1sbnM6eHNpPSJodHRwOi8v d3d3LnczLm9yZy8yMDAxL1hNTFNjaGVtYS1pbnN0YW5jZSI+YXVuKzg1QVl4NWdwRHVxU1BhWkZu
        dz09PC9NYWM+CjwvU2VydmVyRmluaXNoZWQ+
      </Response>
    </ServerResponse>
  </soapenv:Body>
</soapenv:Envelope>
b'<ns0:ServerFinished xmlns:ns0="http://www.rsasecurity.com/rsalabs/otps/schemas/2005/12/ct-kip#" SessionID="++OMITTED++" Status="Success">
  <TokenID>MDAWNTA3OtYxNTMw</TokenID>\n<KeyID>MDAWNTA3OtYxNTMw</KeyID>\n<KeyExpiryDate>2021-04-30T00:00:00+00:00</KeyExpiryDate>\n<ServiceID>RSA CT-KIP</ServiceID>\n<UserID/>\n<Extensions>
    <Extension Critical="true">
      <OTPFormat>Decimal</OTPFormat>\n<OTPLength>6</OTPLength>\n<ns0:OTPMode><ns0:Time TimeInterval="30"/>\n</ns0:OTPMode>\n</Extension>\n</Extensions>\n<Mac MacAlgorithm="http://www.rsasecurity.com/rsalabs/otps/schemas/2005/12/ct-kip#ct-kip-prf-aes">aun+82AY45gpDuqSPaZFnw==</Mac>\n</ns0:ServerFinished>`

 Received ServerFinished response with token information:
  Service ID: RSA CT-KIP
  Key ID: 000506961530
  Token ID: 000506961530
  Token User:
  Expiration date: 2021-04-30T00:00:00+00:00
  OTP mode: 8 Decimal, every 60 seconds
  Token seed: 5b48a28bda2c1c6295185ce972aa9daa

**WARNING: Token has already been committed on server, even though you did not save it.**

================================================================ Any idea how to solve it? May I help with something maybe? Also this warning message bothers me or it should not?

DrDLL commented 5 years ago

@dlenski res2 should be replaced to otpext, in other case default ( 'Decimal', 8, 60) will be set every time

    if otpext:
        otpformat = get_text(res2.find('OTPFormat'), default='Decimal')
        otplength = get_text(res2.find('OTPLength'), int, 8)
        otptime = get_text(res2.find('otps:OTPMode/otps:Time', ns), int, 60, lambda n: n.attrib.get('TimeInterval'))
    else:
        otpformat, otplength, otptime = 'Decimal', 8, 60

Sacrin commented 5 years ago

@dlenski res2 should be replaced to otpext, in other case default ( 'Decimal', 8, 60) will be set every time

if otpext: otpformat = get_text(res2.find('OTPFormat'), default='Decimal') otplength = get_text(res2.find('OTPLength'), int, 8) otptime = get_text(res2.find('otps:OTPMode/otps:Time', ns), int, 60, lambda n: n.attrib.get('TimeInterval')) else: otpformat, otplength, otptime = 'Decimal', 8, 60

This helped to solve the issue with length and interval. But token is still incorrect - not accepted. I'm also confused with the PIN code requirement.

cemeyer commented 5 years ago

@dlenski res2 should be replaced to otpext, in other case default ( 'Decimal', 8, 60) will be set every time

Nice find!

This helped to solve the issue with length and interval. But token is still incorrect - not accepted. I'm also confused with the PIN code requirement.

How are you trying to use the token? Did you configure a PIN with your RSA administrator first? I think you might use an all-zero pin initially as part of the PIN-setting process, but I forget.

dlenski commented 5 years ago

@DrDLL, thanks, good catch. The client wasn't actually extracting the OTP format/length properties in a correct way, so it would always use the defaults. :man_facepalming:

I just pushed 4a44296, which should fix this.

dlenski commented 5 years ago

As for <PinType> and <AddPin>, I believe I am parsing those fields as the real RSA servers produce them… does anyone have a counterexample?

https://github.com/dlenski/rsa_ct_kip/blob/master/rsa_ct_kip/client.py#L155-L156

Currently, the client uses AddPin=1 as the default, if it's unspecified in the XML sent by the server. @cemeyer, do you think AddPin=0 should be the default? Have you seen any server that omits this field entirely, with the intended meaning of "no PIN required at all"?

dlenski commented 5 years ago

@Sacrin,

Any idea how to solve it? May I help with something maybe? Also this warning message bothers me or it should not?

I'm not clear on how this is related to the present issue at all.

If you run it like rsa_ct_kip URL ACTIVATION_CODE, and don't specify a third argument with the file in which to save the token… it won't save it. That's the intended behavior, as described with --help. :man_shrugging:

You should specify a third argument with the filename in which to save the token if you want to save it.

cemeyer commented 5 years ago

@cemeyer, do you think AddPin=0 should be the default? Have you seen any server that omits this field entirely, with the intended meaning of "no PIN required at all"?

Beats me. My n=1 datapoint is that my IT department's server seems to send those fields explicitly, so the default shouldn't matter much to me.

Sacrin commented 5 years ago

@Sacrin,

Any idea how to solve it? May I help with something maybe? Also this warning message bothers me or it should not?

I'm not clear on how this is related to the present issue at all.

If you run it like rsa_ct_kip URL ACTIVATION_CODE, and don't specify a third argument with the file in which to save the token… it won't save it. That's the intended behavior, as described with --help. 🤷‍♂

You should specify a third argument with the filename in which to save the token if you want to save it.

Yes, after I read the code all is clear. What I meant I tried to use it and see some warnings, so skip it.

dlenski / rsa_ct_kip

Generated Token is not correct #4