Closed jjkoehorst closed 5 months ago
It's not available in the current version. Is it supported in iCommands? If so, do environment variables (system variables) have higher priority than configuration file?
Yes, environment variables have higher priority than the configuration file.
That code is from iRODS 4.2.12, but it's identical to what's in 4.3.1.
Oh I got it. Thanks for the info. I'll try to add this feature soon.
The commit 64058a8 will read config from environment variables if they are set with the highest priority.
@jjkoehorst can you confirm the commit fixes your issue?
{
"irods_authentication_scheme": "pam",
"irods_client_server_negotiation": "request_server_negotiation",
"irods_client_server_policy": "CS_NEG_REQUIRE",
"irods_host": ...
./bin/gocmd -c ~/.irods/irods_environment.json ls
Authentication failed (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock')!
For some reason it is not picking up the pam authentication scheme?
Also using environmental variables
echo $IRODS_AUTHENTICATION_SCHEME
pam
(base) ➜ gocommands git:(main) ./bin/gocmd -c ~/.irods/irods_environment.json ls
Authentication failed (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock')!
It looks like something is going wrong with PAM auth. I'll take a look.
PAM authentication uses native authentication as its base. The password for the account is PAM token.
Can you rerun it with -d
flag to show debug logs?
(base) ➜ gocommands git:(main) ./bin/gocmd -d -c ~/.irods/irods_environment.json ls
DEBU[2024-03-13 13:28:06.711] use sessionID - 51756 function=ProcessCommonFlags package=flag
DEBU[2024-03-13 13:28:06.711] reading config file/dir - /Users/koeho006/.irods/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-03-13 13:28:06.711] reading icommands environment file - /Users/koeho006/.irods/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-03-13 13:28:06.712] reading environment file /Users/koeho006/.irods/irods_environment.json function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-03-13 13:28:06.714] reading environment password file /Users/koeho006/.irods/.irodsA function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-03-13 13:28:06.715] reading config from environment variables function=LoadAndOverwriteConfigFromEnv package=commons
DEBU[2024-03-13 13:28:06.716] Connecting to unlock-icat.irods.surfsara.nl:1247 function=Connect package=connection struct=IRODSConnection
DEBU[2024-03-13 13:28:06.729] Start up a connection with CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-03-13 13:28:06.804] Start up CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-03-13 13:28:06.804] Client policy - CS_NEG_REQUIRE, server policy - CS_NEG_REQUIRE function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-03-13 13:28:06.814] Start up SSL function=sslStartup package=connection struct=IRODSConnection
WARN[2024-03-13 13:28:06.814] failed to read CA cert, ignoring... error="ca certificate file is not set" function=sslStartup package=connection struct=IRODSConnection
DEBU[2024-03-13 13:28:06.864] Logging in using native authentication method function=loginNative package=connection struct=IRODSConnection
ERRO[2024-03-13 13:28:06.889] failed to login to irods:
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).Connect
/Users/koeho006/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.13.2/irods/connection/connection.go:253
- received irods authentication error (CAT_INVALID_AUTHENTICATION):
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).login
/Users/koeho006/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.13.2/irods/connection/connection.go:468
- authentication error (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock') function=Connect package=connection struct=IRODSConnection
ERRO[2024-03-13 13:28:06.890] failed to perform ls .:
github.com/cyverse/gocommands/cmd/subcmd.processLsCommand
/Volumes/Git/other/gocommands/cmd/subcmd/ls.go:110
- failed to get connection:
github.com/cyverse/gocommands/cmd/subcmd.listOne
/Volumes/Git/other/gocommands/cmd/subcmd/ls.go:125
- failed to connect to irods server:
github.com/cyverse/go-irodsclient/irods/session.(*ConnectionPool).Get
/Users/koeho006/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.13.2/irods/session/pool.go:202
- failed to login to irods:
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).Connect
/Users/koeho006/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.13.2/irods/connection/connection.go:253
- received irods authentication error (CAT_INVALID_AUTHENTICATION):
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).login
/Users/koeho006/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.13.2/irods/connection/connection.go:468
- authentication error (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock') function=main package=main
Authentication failed (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock')!
Error Trace:
- failed to perform ls .:
github.com/cyverse/gocommands/cmd/subcmd.processLsCommand
/Volumes/Git/other/gocommands/cmd/subcmd/ls.go:110
- failed to get connection:
github.com/cyverse/gocommands/cmd/subcmd.listOne
/Volumes/Git/other/gocommands/cmd/subcmd/ls.go:125
- failed to connect to irods server:
github.com/cyverse/go-irodsclient/irods/session.(*ConnectionPool).Get
/Users/koeho006/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.13.2/irods/session/pool.go:202
- failed to login to irods:
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).Connect
/Users/koeho006/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.13.2/irods/connection/connection.go:253
- received irods authentication error (CAT_INVALID_AUTHENTICATION):
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).login
/Users/koeho006/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.13.2/irods/connection/connection.go:468
- authentication error (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock')
Can you check if it fails to login right after gocmd init
? This might be related to the same PAM TTL issue.
https://github.com/cyverse/gocommands/issues/29
./bin/gocmd iinit
iRODS Host [unlock-icat.irods.surfsara.nl]:
iRODS Port [1247]:
iRODS Zone [unlock]:
iRODS Username [jkoehorst]:
iRODS Password:
Authentication failed (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock')!
Error Trace:
- failed to connect to iRODS server:
github.com/cyverse/gocommands/cmd/subcmd.processInitCommand
/Volumes/Git/other/gocommands/cmd/subcmd/init.go:58
- failed to connect:
github.com/cyverse/gocommands/commons.GetIRODSConnection
/Volumes/Git/other/gocommands/commons/irods_client.go:49
- failed to login to irods:
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).Connect
/Users/koeho006/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.13.2/irods/connection/connection.go:253
- received irods authentication error (CAT_INVALID_AUTHENTICATION):
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).login
/Users/koeho006/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.13.2/irods/connection/connection.go:468
- authentication error (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock')
Do you mean this? @iychoi I can give you access to our system if that would make debugging easier?
When you do gocmd init
, it overwrites ~/.irods/irods_environment.json
file. It will reset your authentication method to native
. You will need to edit the file to set to PAM.
Yes, it will help greatly! Can you send a user account info and server info to this email, "iychoi@arizona.edu"?
Account invite sent if you have questions about the procedure feel free to respond via email...
I already have
"irods_authentication_scheme": "pam",
so not sure what you mean exactly.
Are you using the latest release v0.9.0? It looks like you are getting different error from mine.
My error below is due to empty CA certificate file path. It looks like iCommands have a default one. I'll fix this soon.
(base) iychoi@iychoi-SER5:~/Projects/gocommands/bin$ ./gocmd init -d DEBU[2024-03-20 11:56:02.932] use sessionID - 63771 function=ProcessCommonFlags package=flag DEBU[2024-03-20 11:56:02.932] reading config file/dir - /home/iychoi/.irods function=LoadConfigFromFile package=commons DEBU[2024-03-20 11:56:02.932] reading icommands environment file - /home/iychoi/.irods/irods_environment.json function=LoadConfigFromFile package=commons DEBU[2024-03-20 11:56:02.932] reading environment file /home/iychoi/.irods/irods_environment.json function=Load package=icommands struct=ICommandsEnvironmentManager DEBU[2024-03-20 11:56:02.932] reading environment password file /home/iychoi/.irods/.irodsA function=Load package=icommands struct=ICommandsEnvironmentManager DEBU[2024-03-20 11:56:02.932] reading config from environment variables function=LoadAndOverwriteConfigFromEnv package=commons iRODS Host [unlock-icat.irods.surfsara.nl]: iRODS Port [1247]: iRODS Zone [unlock]: iRODS Username [ichoi]: iRODS Password: DEBU[2024-03-20 11:56:07.997] Connecting to unlock-icat.irods.surfsara.nl:1247 function=Connect package=connection struct=IRODSConnection DEBU[2024-03-20 11:56:08.362] Start up a connection with CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection DEBU[2024-03-20 11:56:08.583] Start up CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection DEBU[2024-03-20 11:56:08.584] Client policy - CS_NEG_REQUIRE, server policy - CS_NEG_REQUIRE function=connectWithCSNegotiation package=connection struct=IRODSConnection DEBU[2024-03-20 11:56:08.738] Start up SSL function=sslStartup package=connection struct=IRODSConnection WARN[2024-03-20 11:56:08.738] failed to read CA cert, ignoring... error="ca certificate file is not set" function=sslStartup package=connection struct=IRODSConnection DEBU[2024-03-20 11:56:09.059] Logging in using pam authentication method function=loginPAMWithToken package=connection struct=IRODSConnection DEBU[2024-03-20 11:56:09.372] Disconnecting the connection function=Disconnect package=connection struct=IRODSConnection
Indeed with -d
.......
DEBU[2024-03-21 08:47:20.861] Start up SSL function=sslStartup package=connection struct=IRODSConnection
WARN[2024-03-21 08:47:20.861] failed to read CA cert, ignoring... error="ca certificate file is not set" function=sslStartup package=connection struct=IRODSConnection
DEBU[2024-03-21 08:47:20.887] Logging in using native authentication method function=loginNative package=connection struct=IRODSConnection
ERRO[2024-03-21 08:47:20.903] failed to login to irods:
Hi @jjkoehorst
I think I fixed the issue. I was able to init
and ls
to your iRODS server.
Can you upgrade gocommands (re-download or use update
command) and try again?
Be sure to check the version of gocommands. The latest one is v0.9.1.
It might be just settings from my side now but with
gocmd -v
{
"clientVersion": "v0.9.1",
"gitCommit": "67c61ec89a8a973dd47dff0eed96cb0d80e3228c",
"buildDate": "2024-03-21T16:51:24Z",
"goVersion": "go1.18.6",
"compiler": "gc",
"platform": "darwin/amd64"
}
and
{
"irods_authentication_scheme": "PAM",
"irods_client_server_negotiation": "request_server_negotiation",
"irods_client_server_policy": "CS_NEG_REQUIRE",
"irods_host": "unlock-icat.irods.surfsara.nl",
"irods_port": 1247,
"irods_user_name": "jkoehorst",
"irods_user_password": "REDACTED",
"irods_zone_name": "unlock",
"irods_encryption_algorithm": "AES-256-CBC",
"irods_encryption_key_size": 32,
"irods_encryption_num_hash_rounds": 16,
"irods_encryption_salt_size": 8,
"irods_ssl_verify_server": "cert"
}
gocmd -d -c ~/.irods/irods_environment.json init
DEBU[2024-03-22 07:41:49.353] use sessionID - 57371 function=ProcessCommonFlags package=flag
DEBU[2024-03-22 07:41:49.353] reading config file/dir - /Users/koeho006/.irods/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-03-22 07:41:49.353] reading icommands environment file - /Users/koeho006/.irods/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-03-22 07:41:49.353] reading environment file /Users/koeho006/.irods/irods_environment.json function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-03-22 07:41:49.354] reading environment password file /Users/koeho006/.irods/.irodsA function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-03-22 07:41:49.354] reading config from environment variables function=LoadAndOverwriteConfigFromEnv package=commons
iRODS Host [unlock-icat.irods.surfsara.nl]:
iRODS Port [1247]:
iRODS Zone [unlock]:
iRODS Username [jkoehorst]:
iRODS Password:
DEBU[2024-03-22 07:41:59.691] Connecting to unlock-icat.irods.surfsara.nl:1247 function=Connect package=connection struct=IRODSConnection
DEBU[2024-03-22 07:41:59.716] Start up a connection with CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-03-22 07:41:59.804] Start up CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-03-22 07:41:59.805] Client policy - CS_NEG_REQUIRE, server policy - CS_NEG_REQUIRE function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-03-22 07:41:59.814] Start up SSL function=sslStartup package=connection struct=IRODSConnection
DEBU[2024-03-22 07:41:59.939] Logging in using native authentication method function=loginNative package=connection struct=IRODSConnection
ERRO[2024-03-22 07:41:59.969] failed to login to irods:
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).Connect
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:289
- received irods authentication error (CAT_INVALID_AUTHENTICATION):
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).login
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:504
- authentication error (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock') function=Connect package=connection struct=IRODSConnection
ERRO[2024-03-22 07:41:59.969] failed to connect to iRODS server:
github.com/cyverse/gocommands/cmd/subcmd.processInitCommand
/github/workspace/cmd/subcmd/init.go:64
- failed to connect:
github.com/cyverse/gocommands/commons.GetIRODSConnection
/github/workspace/commons/irods_client.go:49
- failed to login to irods:
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).Connect
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:289
- received irods authentication error (CAT_INVALID_AUTHENTICATION):
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).login
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:504
- authentication error (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock') function=main package=main
Authentication failed (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock')!
Error Trace:
- failed to connect to iRODS server:
github.com/cyverse/gocommands/cmd/subcmd.processInitCommand
/github/workspace/cmd/subcmd/init.go:64
- failed to connect:
github.com/cyverse/gocommands/commons.GetIRODSConnection
/github/workspace/commons/irods_client.go:49
- failed to login to irods:
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).Connect
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:289
- received irods authentication error (CAT_INVALID_AUTHENTICATION):
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).login
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:504
- authentication error (auth scheme: 'native', username: 'jkoehorst', zone: 'unlock')
-
After removing all my irods related system variables which as requested precedes the normal variables...
gocmd -d -c ~/.irods/irods_environment.json init
DEBU[2024-03-22 07:43:50.660] use sessionID - 48799 function=ProcessCommonFlags package=flag
DEBU[2024-03-22 07:43:50.660] reading config file/dir - /Users/koeho006/.irods/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-03-22 07:43:50.661] reading icommands environment file - /Users/koeho006/.irods/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-03-22 07:43:50.661] reading environment file /Users/koeho006/.irods/irods_environment.json function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-03-22 07:43:50.661] reading environment password file /Users/koeho006/.irods/.irodsA function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-03-22 07:43:50.662] reading config from environment variables function=LoadAndOverwriteConfigFromEnv package=commons
iRODS Host [unlock-icat.irods.surfsara.nl]:
iRODS Port [1247]:
iRODS Zone [unlock]:
iRODS Username [jkoehorst]:
iRODS Password:
DEBU[2024-03-22 07:44:01.609] Connecting to unlock-icat.irods.surfsara.nl:1247 function=Connect package=connection struct=IRODSConnection
DEBU[2024-03-22 07:44:01.632] Start up a connection with CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-03-22 07:44:01.723] Start up CS Negotiation function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-03-22 07:44:01.724] Client policy - CS_NEG_REQUIRE, server policy - CS_NEG_REQUIRE function=connectWithCSNegotiation package=connection struct=IRODSConnection
DEBU[2024-03-22 07:44:01.734] Start up SSL function=sslStartup package=connection struct=IRODSConnection
DEBU[2024-03-22 07:44:01.874] Logging in using pam authentication method function=loginPAMWithToken package=connection struct=IRODSConnection
ERRO[2024-03-22 07:44:01.903] failed to login to irods:
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).Connect
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:289
- received irods authentication error (CAT_INVALID_AUTHENTICATION):
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).login
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:504
- authentication error (auth scheme: 'pam', username: 'jkoehorst', zone: 'unlock') function=Connect package=connection struct=IRODSConnection
ERRO[2024-03-22 07:44:01.903] failed to connect to iRODS server:
github.com/cyverse/gocommands/cmd/subcmd.processInitCommand
/github/workspace/cmd/subcmd/init.go:64
- failed to connect:
github.com/cyverse/gocommands/commons.GetIRODSConnection
/github/workspace/commons/irods_client.go:49
- failed to login to irods:
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).Connect
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:289
- received irods authentication error (CAT_INVALID_AUTHENTICATION):
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).login
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:504
- authentication error (auth scheme: 'pam', username: 'jkoehorst', zone: 'unlock') function=main package=main
Authentication failed (auth scheme: 'pam', username: 'jkoehorst', zone: 'unlock')!
Error Trace:
- failed to connect to iRODS server:
github.com/cyverse/gocommands/cmd/subcmd.processInitCommand
/github/workspace/cmd/subcmd/init.go:64
- failed to connect:
github.com/cyverse/gocommands/commons.GetIRODSConnection
/github/workspace/commons/irods_client.go:49
- failed to login to irods:
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).Connect
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:289
- received irods authentication error (CAT_INVALID_AUTHENTICATION):
github.com/cyverse/go-irodsclient/irods/connection.(*IRODSConnection).login
/go/pkg/mod/github.com/cyverse/go-irodsclient@v0.14.1/irods/connection/connection.go:504
- authentication error (auth scheme: 'pam', username: 'jkoehorst', zone: 'unlock')
it works!!! Had to remove .irodsA
Now the next step is to authenticate using a system variable. Currently as far as I can see even with a system password set the gocmd ls still asks for the password
export irods_user_password="....REDACTED...."
echo $irods_user_password
Baseuh...REDACTED..."
gocmd -d -c ~/.irods/irods_environment.json ls
DEBU[2024-03-22 07:48:52.517] use sessionID - 50339 function=ProcessCommonFlags package=flag
DEBU[2024-03-22 07:48:52.517] reading config file/dir - /Users/koeho006/.irods/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-03-22 07:48:52.517] reading icommands environment file - /Users/koeho006/.irods/irods_environment.json function=LoadConfigFromFile package=commons
DEBU[2024-03-22 07:48:52.517] reading environment file /Users/koeho006/.irods/irods_environment.json function=Load package=icommands struct=ICommandsEnvironmentManager
DEBU[2024-03-22 07:48:52.518] reading config from environment variables function=LoadAndOverwriteConfigFromEnv package=commons
iRODS Password:
<this should not be requested right?>
it is also set in the irods_environment.json so that does not help either?
Okay. Great to hear that the patch worked.
Providing password via irods_user_password
variable is not implemented in gocommands. I wasn't able to find the field in iCommands and irods source code. Is irods_user_password
variable supported in iCommands?
Not sure but I tried the example file at https://github.com/cyverse/gocommands/blob/main/examples/config_pam.yaml
Ah.. the environment variable for the password is `IRODS_USER_PASSWORD". Name must be in capital.
I added an example of this to the repo. Please check it out. https://github.com/cyverse/gocommands/blob/main/examples/config_pam_env.sh
That works like a charm!
Thanks for the wonderful GO implementation of iCommands. Is it possible to mix and match the pam configuration with system variables?
This way we can set the PAM skeleton but can push the username, password and host through system variables upon deployment.