search

IN-CORE / pyincore

pyIncore is a component of IN-CORE. It is a python package consisting of two primary components: 1) a set of service classes to interact with the IN-CORE web services, and 2) IN-CORE analyses . The pyIncore allows users to apply various hazards to infrastructure in selected areas, propagating the effect of physical infrastructure damage and loss of functionality to social and economic impacts.
Mozilla Public License 2.0
24 stars 7 forks source link

Fixed JWT token validation #530

Closed ylyangtw closed 3 months ago

ylyangtw commented 3 months ago

In this PR, JWT token validation uses decoding the token itself since the token contains expiration info (the exp param). This way we don't need to talk to keycloak and have accurate expiration time.

For testing, you can run test_client.py

For testing whether it handles expired JWT token, here are the steps:

  1. Install the branch pip install . -U --no-dependencies
  2. cd ~/.incore
  3. Replace the most recent token file with expired token
  4. Use your python env and run these codes: 
    from pyincore import Client, IncoreClient
client=IncoreClient()

    It should ask your username and password. After you login, you will see a new token created under the folder cache_data

navarroc commented 3 months ago

Can you point this at the release?

ylyangtw commented 3 months ago

Can you point this at the release?

Ah right. Just did!

Vismayak commented 3 months ago

Thank you for the information on testing, was very helpful. All the tests passed for me and the token folder was created in cache_data!

longshuicy commented 3 months ago

Remove the most recent token using rmdir [token numer] under the folder cache_data For some reason I'm not seeing the token file? What have I missed?

# store the token file in the respective repository's directory
if token_file_name is None or len(token_file_name.strip()) == 0:
token_file_name = "." + self.hashed_service_url + "_token"

Here is what my hashed folder looks like:

image
longshuicy commented 3 months ago

Remove the most recent token using rmdir [token numer] under the folder cache_data For some reason I'm not seeing the token file? What have I missed?

 # store the token file in the respective repository's directory
            if token_file_name is None or len(token_file_name.strip()) == 0:
                token_file_name = "." + self.hashed_service_url + "_token"

Here is what my hashed folder looks like: image

Maybe I should test by creating a token file with expired token? @navarroc

image
longshuicy commented 3 months ago

The code looks good but I don't think the test method in description how we can test it?

  1. I don't have any token number file under cache_data folder. Could you double check? For me, my tokens are stored under ~/.incore e.g.

    image

  2. Deleting the token will of course triggering a prompt to type username/password, but it's not testing the "expiration" per se. To test expiration, I believe we need to modify the token to use an expired one. Here is mine example I happen to have it handy.

    bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJabHVyQldkLUhHV2ZxeUNkb28yc1FEbmd4RE9mbEVZeks3My05UXpnY0JBIn0.eyJleHAiOjE3MTAzODY1MzAsImlhdCI6MTcxMDM0MzMzMCwianRpIjoiZDIyODk3ZGItMjFjZC00YjM4LWE5YjgtNTBhYjFiNGNlZWViIiwiaXNzIjoiaHR0cHM6Ly9pbmNvcmUtZGV2Lm5jc2EuaWxsaW5vaXMuZWR1L3JlYWxtcy9Jbi1jb3JlIiwiYXVkIjpbIklOQ09SRSIsInJlYWxtLW1hbmFnZW1lbnQiLCJhY2NvdW50Il0sInN1YiI6ImQzYmRlMjJmLTBmOTYtNDRkNy04MjI5LTAxZDEzYWFjOGRmZiIsInR5cCI6IkJlYXJlciIsImF6cCI6InJlYWN0LWF1dGgiLCJzZXNzaW9uX3N0YXRlIjoiYTAwMTc0MzYtZTVmMi00ZDRlLTgzODctN2YxYjRmNjVmODQ4IiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyJodHRwOi8vbG9jYWxob3N0OjMwMDAiXSwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbImluY29yZV9hZG1pbiIsIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iLCJpbmNvcmVfdXNlcl9yb2xlIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsicmVhbG0tbWFuYWdlbWVudCI6eyJyb2xlcyI6WyJtYW5hZ2UtdXNlcnMiXX0sImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6InByb2ZpbGUgcm9sZXMgZW1haWwiLCJzaWQiOiJhMDAxNzQzNi1lNWYyLTRkNGUtODM4Ny03ZjFiNGY2NWY4NDgiLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwidWlkX251bWJlciI6NTYwMjEsIm5hbWUiOiJDaGVuIFdhbmciLCJncm91cHMiOlsiaW5jb3JlX2p1cHl0ZXIiLCJpbmNvcmVfc2xjX3VzZXIiLCJpbmNvcmVfZ2FsdmVzdG9uX3VzZXIiLCJpbmNvcmVfam9wbGluX3VzZXIiLCJpbmNvcmVfdXNlciIsImluY29yZV9jb2UiLCJpbmNvcmVfbmNzYSJdLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJjd2FuZzEzOCIsImdpdmVuX25hbWUiOiJDaGVuIiwiZmFtaWx5X25hbWUiOiJXYW5nIiwiZW1haWwiOiJjd2FuZzEzOEBpbGxpbm9pcy5lZHUifQ.n9_Ny_MpmS9dAicRz-to1c4Mnh4aweKFk8AUr4gdF6LRvfnmh_WR1Ty_YCxen8SqorXf8zgEIx4S8u_2vAZaNPQ9KiU29DlD1lX-8bkkFHGj0Ma7Gpe-8yLYsWSj5CrSDBqaEI8Kzx12UBg6a-3H7pIYEWs4AzU4rO0VG0F7ODU1wbT6kBK1ROqLuDldiJRNXtBcnw9pWcsq_kdivTv_ow4Z20fnXdFS9iBraRC1KWYM9j25Oh3ZKvMnXac0AwEbGWOy4tWIhpfV4y-qSR40jE-JuQ3KX5SZLEYsfyKqAUDPLN8T24Ep_lTsiyRBcpZcp-533vDEPMR5OBon2bujhQ
longshuicy commented 3 months ago

Last thing, I'm not sure why the pytest are failing. Could you merge the latest release-1.18.0 to this branch see if it got fixed. Other PR pointing to release seems to be fine. Thanks!

navarroc commented 3 months ago

Last thing, I'm not sure why the pytest are failing. Could you merge the latest release-1.18.0 to this branch see if it got fixed. Other PR pointing to release seems to be fine. Thanks!

Yes release has unit test fixes

ylyangtw commented 3 months ago

The code looks good but I don't think the test method in description how we can test it?

  1. I don't have any token number file under cache_data folder. Could you double check? For me, my tokens are stored under ~/.incore e.g.
image
  1. Deleting the token will of course triggering a prompt to type username/password, but it's not testing the "expiration" per se. To test expiration, I believe we need to modify the token to use an expired one. Here is mine example I happen to have it handy.
bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJabHVyQldkLUhHV2ZxeUNkb28yc1FEbmd4RE9mbEVZeks3My05UXpnY0JBIn0.eyJleHAiOjE3MTAzODY1MzAsImlhdCI6MTcxMDM0MzMzMCwianRpIjoiZDIyODk3ZGItMjFjZC00YjM4LWE5YjgtNTBhYjFiNGNlZWViIiwiaXNzIjoiaHR0cHM6Ly9pbmNvcmUtZGV2Lm5jc2EuaWxsaW5vaXMuZWR1L3JlYWxtcy9Jbi1jb3JlIiwiYXVkIjpbIklOQ09SRSIsInJlYWxtLW1hbmFnZW1lbnQiLCJhY2NvdW50Il0sInN1YiI6ImQzYmRlMjJmLTBmOTYtNDRkNy04MjI5LTAxZDEzYWFjOGRmZiIsInR5cCI6IkJlYXJlciIsImF6cCI6InJlYWN0LWF1dGgiLCJzZXNzaW9uX3N0YXRlIjoiYTAwMTc0MzYtZTVmMi00ZDRlLTgzODctN2YxYjRmNjVmODQ4IiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyJodHRwOi8vbG9jYWxob3N0OjMwMDAiXSwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbImluY29yZV9hZG1pbiIsIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iLCJpbmNvcmVfdXNlcl9yb2xlIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsicmVhbG0tbWFuYWdlbWVudCI6eyJyb2xlcyI6WyJtYW5hZ2UtdXNlcnMiXX0sImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6InByb2ZpbGUgcm9sZXMgZW1haWwiLCJzaWQiOiJhMDAxNzQzNi1lNWYyLTRkNGUtODM4Ny03ZjFiNGY2NWY4NDgiLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwidWlkX251bWJlciI6NTYwMjEsIm5hbWUiOiJDaGVuIFdhbmciLCJncm91cHMiOlsiaW5jb3JlX2p1cHl0ZXIiLCJpbmNvcmVfc2xjX3VzZXIiLCJpbmNvcmVfZ2FsdmVzdG9uX3VzZXIiLCJpbmNvcmVfam9wbGluX3VzZXIiLCJpbmNvcmVfdXNlciIsImluY29yZV9jb2UiLCJpbmNvcmVfbmNzYSJdLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJjd2FuZzEzOCIsImdpdmVuX25hbWUiOiJDaGVuIiwiZmFtaWx5X25hbWUiOiJXYW5nIiwiZW1haWwiOiJjd2FuZzEzOEBpbGxpbm9pcy5lZHUifQ.n9_Ny_MpmS9dAicRz-to1c4Mnh4aweKFk8AUr4gdF6LRvfnmh_WR1Ty_YCxen8SqorXf8zgEIx4S8u_2vAZaNPQ9KiU29DlD1lX-8bkkFHGj0Ma7Gpe-8yLYsWSj5CrSDBqaEI8Kzx12UBg6a-3H7pIYEWs4AzU4rO0VG0F7ODU1wbT6kBK1ROqLuDldiJRNXtBcnw9pWcsq_kdivTv_ow4Z20fnXdFS9iBraRC1KWYM9j25Oh3ZKvMnXac0AwEbGWOy4tWIhpfV4y-qSR40jE-JuQ3KX5SZLEYsfyKqAUDPLN8T24Ep_lTsiyRBcpZcp-533vDEPMR5OBon2bujhQ

Hmm seems like different OS works differently, I will mark it in the description

ylyangtw commented 3 months ago

just modified the test step and merged the release branch

navarroc commented 3 months ago

Remove the most recent token using rmdir [token numer] under the folder cache_data For some reason I'm not seeing the token file? What have I missed?

 # store the token file in the respective repository's directory
            if token_file_name is None or len(token_file_name.strip()) == 0:
                token_file_name = "." + self.hashed_service_url + "_token"

Here is what my hashed folder looks like: image

Maybe I should test by creating a token file with expired token? @navarroc image

That's what I did, I used an old token from my chrome rest client and put that in the .incore folder and to verify, I also asked it to print the time the token expired and it all looks right.

navarroc commented 3 months ago

The code looks good but I don't think the test method in description how we can test it?

  1. I don't have any token number file under cache_data folder. Could you double check? For me, my tokens are stored under ~/.incore e.g.
image
  1. Deleting the token will of course triggering a prompt to type username/password, but it's not testing the "expiration" per se. To test expiration, I believe we need to modify the token to use an expired one. Here is mine example I happen to have it handy. 
    bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJabHVyQldkLUhHV2ZxeUNkb28yc1FEbmd4RE9mbEVZeks3My05UXpnY0JBIn0.eyJleHAiOjE3MTAzODY1MzAsImlhdCI6MTcxMDM0MzMzMCwianRpIjoiZDIyODk3ZGItMjFjZC00YjM4LWE5YjgtNTBhYjFiNGNlZWViIiwiaXNzIjoiaHR0cHM6Ly9pbmNvcmUtZGV2Lm5jc2EuaWxsaW5vaXMuZWR1L3JlYWxtcy9Jbi1jb3JlIiwiYXVkIjpbIklOQ09SRSIsInJlYWxtLW1hbmFnZW1lbnQiLCJhY2NvdW50Il0sInN1YiI6ImQzYmRlMjJmLTBmOTYtNDRkNy04MjI5LTAxZDEzYWFjOGRmZiIsInR5cCI6IkJlYXJlciIsImF6cCI6InJlYWN0LWF1dGgiLCJzZXNzaW9uX3N0YXRlIjoiYTAwMTc0MzYtZTVmMi00ZDRlLTgzODctN2YxYjRmNjVmODQ4IiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyJodHRwOi8vbG9jYWxob3N0OjMwMDAiXSwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbImluY29yZV9hZG1pbiIsIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iLCJpbmNvcmVfdXNlcl9yb2xlIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsicmVhbG0tbWFuYWdlbWVudCI6eyJyb2xlcyI6WyJtYW5hZ2UtdXNlcnMiXX0sImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6InByb2ZpbGUgcm9sZXMgZW1haWwiLCJzaWQiOiJhMDAxNzQzNi1lNWYyLTRkNGUtODM4Ny03ZjFiNGY2NWY4NDgiLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwidWlkX251bWJlciI6NTYwMjEsIm5hbWUiOiJDaGVuIFdhbmciLCJncm91cHMiOlsiaW5jb3JlX2p1cHl0ZXIiLCJpbmNvcmVfc2xjX3VzZXIiLCJpbmNvcmVfZ2FsdmVzdG9uX3VzZXIiLCJpbmNvcmVfam9wbGluX3VzZXIiLCJpbmNvcmVfdXNlciIsImluY29yZV9jb2UiLCJpbmNvcmVfbmNzYSJdLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJjd2FuZzEzOCIsImdpdmVuX25hbWUiOiJDaGVuIiwiZmFtaWx5X25hbWUiOiJXYW5nIiwiZW1haWwiOiJjd2FuZzEzOEBpbGxpbm9pcy5lZHUifQ.n9_Ny_MpmS9dAicRz-to1c4Mnh4aweKFk8AUr4gdF6LRvfnmh_WR1Ty_YCxen8SqorXf8zgEIx4S8u_2vAZaNPQ9KiU29DlD1lX-8bkkFHGj0Ma7Gpe-8yLYsWSj5CrSDBqaEI8Kzx12UBg6a-3H7pIYEWs4AzU4rO0VG0F7ODU1wbT6kBK1ROqLuDldiJRNXtBcnw9pWcsq_kdivTv_ow4Z20fnXdFS9iBraRC1KWYM9j25Oh3ZKvMnXac0AwEbGWOy4tWIhpfV4y-qSR40jE-JuQ3KX5SZLEYsfyKqAUDPLN8T24Ep_lTsiyRBcpZcp-533vDEPMR5OBon2bujhQ

Hmm seems like different OS works differently, I will mark it in the description

Are you sure your tokens are in the cache_data folder? It should be one level up in .incore so clear_cache doesn't wipe out the tokens. This should not be OS dependent at all.

ylyangtw commented 3 months ago

The code looks good but I don't think the test method in description how we can test it?

  1. I don't have any token number file under cache_data folder. Could you double check? For me, my tokens are stored under ~/.incore e.g.
image
  1. Deleting the token will of course triggering a prompt to type username/password, but it's not testing the "expiration" per se. To test expiration, I believe we need to modify the token to use an expired one. Here is mine example I happen to have it handy. 
    bearer eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJabHVyQldkLUhHV2ZxeUNkb28yc1FEbmd4RE9mbEVZeks3My05UXpnY0JBIn0.eyJleHAiOjE3MTAzODY1MzAsImlhdCI6MTcxMDM0MzMzMCwianRpIjoiZDIyODk3ZGItMjFjZC00YjM4LWE5YjgtNTBhYjFiNGNlZWViIiwiaXNzIjoiaHR0cHM6Ly9pbmNvcmUtZGV2Lm5jc2EuaWxsaW5vaXMuZWR1L3JlYWxtcy9Jbi1jb3JlIiwiYXVkIjpbIklOQ09SRSIsInJlYWxtLW1hbmFnZW1lbnQiLCJhY2NvdW50Il0sInN1YiI6ImQzYmRlMjJmLTBmOTYtNDRkNy04MjI5LTAxZDEzYWFjOGRmZiIsInR5cCI6IkJlYXJlciIsImF6cCI6InJlYWN0LWF1dGgiLCJzZXNzaW9uX3N0YXRlIjoiYTAwMTc0MzYtZTVmMi00ZDRlLTgzODctN2YxYjRmNjVmODQ4IiwiYWNyIjoiMSIsImFsbG93ZWQtb3JpZ2lucyI6WyJodHRwOi8vbG9jYWxob3N0OjMwMDAiXSwicmVhbG1fYWNjZXNzIjp7InJvbGVzIjpbImluY29yZV9hZG1pbiIsIm9mZmxpbmVfYWNjZXNzIiwidW1hX2F1dGhvcml6YXRpb24iLCJpbmNvcmVfdXNlcl9yb2xlIl19LCJyZXNvdXJjZV9hY2Nlc3MiOnsicmVhbG0tbWFuYWdlbWVudCI6eyJyb2xlcyI6WyJtYW5hZ2UtdXNlcnMiXX0sImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJ2aWV3LXByb2ZpbGUiXX19LCJzY29wZSI6InByb2ZpbGUgcm9sZXMgZW1haWwiLCJzaWQiOiJhMDAxNzQzNi1lNWYyLTRkNGUtODM4Ny03ZjFiNGY2NWY4NDgiLCJlbWFpbF92ZXJpZmllZCI6dHJ1ZSwidWlkX251bWJlciI6NTYwMjEsIm5hbWUiOiJDaGVuIFdhbmciLCJncm91cHMiOlsiaW5jb3JlX2p1cHl0ZXIiLCJpbmNvcmVfc2xjX3VzZXIiLCJpbmNvcmVfZ2FsdmVzdG9uX3VzZXIiLCJpbmNvcmVfam9wbGluX3VzZXIiLCJpbmNvcmVfdXNlciIsImluY29yZV9jb2UiLCJpbmNvcmVfbmNzYSJdLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJjd2FuZzEzOCIsImdpdmVuX25hbWUiOiJDaGVuIiwiZmFtaWx5X25hbWUiOiJXYW5nIiwiZW1haWwiOiJjd2FuZzEzOEBpbGxpbm9pcy5lZHUifQ.n9_Ny_MpmS9dAicRz-to1c4Mnh4aweKFk8AUr4gdF6LRvfnmh_WR1Ty_YCxen8SqorXf8zgEIx4S8u_2vAZaNPQ9KiU29DlD1lX-8bkkFHGj0Ma7Gpe-8yLYsWSj5CrSDBqaEI8Kzx12UBg6a-3H7pIYEWs4AzU4rO0VG0F7ODU1wbT6kBK1ROqLuDldiJRNXtBcnw9pWcsq_kdivTv_ow4Z20fnXdFS9iBraRC1KWYM9j25Oh3ZKvMnXac0AwEbGWOy4tWIhpfV4y-qSR40jE-JuQ3KX5SZLEYsfyKqAUDPLN8T24Ep_lTsiyRBcpZcp-533vDEPMR5OBon2bujhQ

Hmm seems like different OS works differently, I will mark it in the description

Are you sure your tokens are in the cache_data folder? It should be one level up in .incore so clear_cache doesn't wipe out the tokens. This should not be OS dependent at all.

Sorry I checked it again it's in .incore and I modified the test step too