Closed NhlamuloChauke closed 2 years ago
$ ierror 893000
irods error: -893000 CAT_TICKET_USER_EXCLUDED
Does this user have access to this ticket?
@trel setting up a zoom session with CSIR DIRISA Team would be very helpful.
Can you confirm that the ticket is in good standing - iticket ls
?
And that the user has access via iget -t
with the same ticket?
@trel please see the details below:
[14:16] Claude Fortune Confirmed ticket is available and valid
cfortune@irods_dev:~$ iticket ls
id: 10253
string: d51Eo9vlIPyIINZ
ticket type: read
obj type: data
owner name: newguy
owner zone: dirisa.ac.za
uses count: 0
uses limit: 0
write file count: 0
write file limit: 0
write byte count: 0
write byte limit: 0
expire time: none
data-object name: ENMT ActionList 19JanuaryV1.docx
data collection: /dirisa.ac.za/home/newguy
No host restrictions
No user restrictions
No group restrictions
[14:19] Claude Fortune Checking if i can download the ticket fails as it does not see the object.
Ticket string: d51Eo9vlIPyIINZ
cfortune@irods_dev:~$ iget -t ticket d51Eo9vlIPyIINZ
remote addresses: 192.168.100.100 ERROR: resolveRodsTarget: srcPath /dirisa.ac.za/home/newguy/d51Eo9vlIPyIINZ does not exist
remote addresses: 192.168.100.100 ERROR: getUtil: resolveRodsTarget status = -317000 USER_INPUT_PATH_ERR
<https://teams.microsoft.com/l/message/19:3744aa6ee3bd45699bbfcd9b30452c9e@thread.tacv2/1652703416599?tenantId=2fd3c5d5-ddb2-4ed3-9803-f89675928df4&groupId=e2c15fb6-95c5-424a-b542-9a489a83afe1&parentMessageId=1652678274706&teamName=Dirisa - technical&channelName=Software development&createdTime=1652703416599>
Okay, so the ticket is valid...
Please try
$ iget -t d51Eo9vlIPyIINZ "/dirisa.ac.za/home/newguy/ENMT ActionList 19JanuaryV1.docx"
Please see the following
iget -t d51Eo9vlIPyIINZ "/dirisa.ac.za/home/newguy/ENMT ActionList 19JanuaryV1.docx"
download works on CLI
Ah, very good. We'll try to reproduce.
@trel Thank you
@trel update: Downloading from the shared links and tickets only works when you are logged in as admin
We will look into this today.
What version of iRODS are you running Metalnx against?
I believe we've reproduced this in Metalnx.
Can you confirm the following:
metalnx.enable.tickets
is set to true
in your configuration file500 - Internal Server Error
when you attempt to download the data object via the Tickets
UI (steps to reproduce are below):
This was done against iRODS 4.2.11.
What version of iRODS are you running?
@korydraughn We are running IRODS Version 4.2.10
metalnx.enable.tickets
is set to true
. I am getting the following attached when downloading from the tickets
Thanks. I'll test against 4.2.10 then.
There is a ticket compatibility issue with 4.2.11, but that is specific to Jargon.
I will post what I find here.
What version of Metalnx are you running?
We are running Metalnxversion 2.6.0
Have you set/modified any additional options when launching the container?
@korydraughn I am not following
Have you changed the configuration in other ways that I should know about?
I want to make sure my setup matches yours as much as possible. We've already confirmed that you've enabled support for tickets. Have you made other changes to that config file or any other file?
metalnx.properties
irods.host=xxxxxx
irods.port=1247
irods.zoneName=xxxxx
irods.admin.user=xxxxx
irods.admin.password=xxxxxx
# may be left blank this is the default email for any messages generated from metalnx
irods.admin.email=
# STANDARD | PAM AUTH
irods.auth.scheme=STANDARD
default.storage.resource=storage1
# sets jargon ssl negotiation policy for the client. Leaving to DONT_CARE defers to the server, and is recommended
# NO_NEGOTIATION, CS_NEG_REFUSE, CS_NEG_REQUIRE, CS_NEG_DONT_CARE
ssl.negotiation.policy=CS_NEG_DONT_CARE
##########################################################
# jargon properties settings
utilize.packing.streams=true
# jargoni now supports checksum calculation for streaming uploads. This does not currently verify, but does store if set to true
compute.checksum=true
##########################################################
# mail settings
# configures an smtp server for various mail functions that are only enabled by setting mail.enable to true.
# This includes the initial implementation of access requests
mail.enabled=false
mail.default-encoding=UTF-8
mail.host=example.com
mail.username=
mail.password=
# mail 'to' field is set by default to the irods email above, but can be overridden in code
mail.from=
mail.port=25
mail.smtp.starttls.enable=true
mail.smtp.auth=false
mail.transport.protocol=smtp
mail.debug=true
##########################################################
# metalnx database settings
db.driverClassName=org.postgresql.Driver
db.url=jdbc:postgresql://xxxxx:5432/xxxx
db.username=xxxx
db.password=xxxxx
hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
# Debugging options
hibernate.show_sql=false
hibernate.format_sql=false
# Automatic schema generation
hibernate.hbm2ddl.auto=update
# Connection parameters
connection.pool_size=10
######################################
# other irods environment
# Jobs parameters
jobs.irods.username=xxxxx
jobs.irods.password=xxxxx
jobs.irods.auth.scheme=STANDARD
runSyncJobs=true
# HTTP connection for RMD
rmd.connection.timeout=500
rmd.connection.port=8000
# Reverse DNS lookup on dashboard
reverse.dns.lookup=false
populate.msi.enabled=false
illumina.msi.enabled=true
# MSI API version supported by this application
msi.api.version=1.X.X
msi.metalnx.list=libmsiget_illumina_meta.so,libmsiobjget_microservices.so,libmsiobjget_version.so,libmsiobjjpeg_extract.so,libmsiobjput_mdbam.so,libmsiobjput_mdbam.so,libmsiobjput_mdmanifest.so,libmsiobjput_mdvcf.so,libmsiobjput_populate.so
msi.irods.list=libmsisync_to_archive.so,libmsi_update_unixfilesystem_resource_free_space.so,libmsiobjput_http.so,libmsiobjput_irods.so,libmsiobjget_irods.so,libmsiobjget_http.so,libmsiobjput_slink.so,libmsiobjget_slink.so
msi.irods.42.list=libmsisync_to_archive.so,libmsi_update_unixfilesystem_resource_free_space.so
msi.other.list=
metalnx.enable.tickets=true
metalnx.enable.upload.rules=false
metalnx.download.limit=6000
metalnx.enable.dashboard=false
access.proxy=false
irodsext.dataprofiler.retrieve.tickets=true
irodsext.dataprofiler.retrieve.starred=true
irodsext.dataprofiler.retrieve.shared=true
irodsext.dataprofiler.retrieve.tags.and.comments=false
irodsext.dataprofiler.retrieve.metadata.templates=false
irodsext.datatyper.persist.data.types=false
irodsext.datatyper.detailed.determination=false
metalnx.authtype.mappings=PAM:PAM|STANDARD:Standard
jwt.issuer=metalnx
jwt.secret=thisisasecretthatisverysecretyouwillneverguessthiskey
jwt.algo=HS384
#############################
# Pluggable search configuration. Turn on and off pluggable search globally, and configure search endpoints.
# N.B. pluggable search also requires provisioning of the jwt.* information above
#############################
# configured endpoints, comma delimited in form https://host.com/v1
# Note the commented out URL which matches up to the irods-contrib/file-and-metadata-indexer docker compose assembly. In order to
# utilize this assembly you need to uncomment the URL and set pluggablesearch.enabled to true
pluggablesearch.endpointRegistryList=
# enable pluggable search globally and show the search GUI components
pluggablesearch.enabled=false
# display the older file and properties search in the menu, if you are running the elasticsearch standard plugin this is probably
# a menu item to turn off
classicsearch.enabled=true
# JWT subject claim used to access search endpoint for data gathering. User searches will utilize the name of the individual
pluggablesearch.endpointAccessSubject=metalnx
# timeout for info/attribute gathering, set to 0 for no timeout
pluggablesearch.info.timeout=0
# timeout for actual search, set to 0 for no timeout
pluggablesearch.search.timeout=0
#############################
# Pluggable shopping cart and export plugin configuration.
# Turn on and off pluggable shopping cart globally, and configure export endpoints.
# N.B. plugins also requires provisioning of the jwt.* information above
#############################
# enable pluggable export globally and show the export GUI components
pluggableshoppingcart.enabled=false
# configured endpoints, comma delimited in form https://host.com/v1
pluggablepublishing.endpointRegistryList=
# timeout for info/attribute gathering, set to 0 for no timeout
pluggablepublishing.info.timeout=0
# timeout for actual publishing, set to 0 for no timeout
pluggablepublishing.publishing.timeout=0
# server rule engine instance that will provide the galleryview listing
gallery_view.rule_engine_plugin.instance_name=irods_rule_engine_plugin-irods_rule_language-instance
docker-compose.yml
version: '3'
services:
metalnx_db: container_name: metalnx_db image: postgres:11 restart: always environment: POSTGRES_PASSWORD: xxxx POSTGRES_USER: xxx POSTGRES_DB: xxxxx
metalnx: container_name: metalnx_web image: irods/metalnx:latest restart: always volumes:
@korydraughn please the configs above
Thanks.
Are there any other stacktraces in the log related to the ticket-based download attempt?
I get several stacktraces and it would be helpful if you can post all stacktraces in your log following the ticket-based download operation.
Please make sure to filter out any sensitive info before posting.
Please see the logs below.
2022-07-13 17:33:13 INFO TicketClientController:107 - Getting files using ticket: 3P8xlU8XjY9iAAP
2022-07-13 17:33:13 WARN TicketClientServiceImpl:209 - Accessing files and collections with tickets as authenticated user
2022-07-13 17:33:13 INFO IRODSFileImpl:263 - setting file name, given path = /xxxxxx/home/xxxxx/OpenVPN profile update for 2FA How To Guide.pdf
2022-07-13 17:33:13 INFO IRODSFileImpl:264 - detected local separator = /
2022-07-13 17:33:13 INFO IRODSFileImpl:293 - file name was set as: OpenVPN profile update for 2FA How To Guide.pdf
2022-07-13 17:33:13 INFO TicketClientOperationsImpl:109 - getFileFromIRODSUsingTicket()
2022-07-13 17:33:13 INFO TicketClientOperationsImpl:117 - initializing session with ticket:3P8xlU8XjY9iAAP
2022-07-13 17:33:13 INFO TicketClientSupport:51 - initializeSessionWithTicket()
2022-07-13 17:33:13 INFO TicketClientSupport:57 - ticketString:3P8xlU8XjY9iAAP
2022-07-13 17:33:13 INFO TicketClientOperationsImpl:120 - session initialized, doing get operation
2022-07-13 17:33:13 INFO DataTransferOperationsImpl:253 - getOperation()
2022-07-13 17:33:13 INFO DataTransferOperationsImpl:819 - creating default transfer control block, none was supplied and a callback listener is set
2022-07-13 17:33:13 INFO DataTransferOperationsImpl:259 - no transferStatusCallbackListener set for getOperation()
2022-07-13 17:33:13 INFO DataTransferOperationsImpl:270 - get operation, irods source file is: /xxxxxxx/home/xxxxxx/OpenVPN profile update for 2FA How To Guide.pdf
2022-07-13 17:33:13 INFO DataTransferOperationsImpl:271 - local file for get: /usr/local/tomcat/tmp-ticket-files
2022-07-13 17:33:13 INFO DataTransferOperationsImpl:279 - file name normalized:/usr/local/tomcat/tmp-ticket-files/OpenVPN profile update for 2FA How To Guide.pdf
2022-07-13 17:33:13 INFO DataTransferOperationsImpl:285 - am I rerouting?
2022-07-13 17:33:13 INFO DataTransferOperationsImpl:372 - get objStat..
2022-07-13 17:33:13 INFO CollectionListingUtils:1278 - ObjStat [absolutePath=/xxxxx/home/xxxxxx/OpenVPN profile update for 2FA How To Guide.pdf, objectPath=, objectType=DATA_OBJECT, dataId=10270, checksum=sha2:CD0KZoxoByUP7xb9gknXJDTUnYsXKBkJCqXTCAkEIis=, ownerName=xxxxx ownerZone=xxxxx, objSize=251741, createdAt=Thu Apr 21 06:42:10 GMT 2022, modifiedAt=Thu Apr 21 06:42:10 GMT 2022, specColType=NORMAL, collectionPath=, cacheDir=, cacheDirty=false, replNumber=0, standInGeneratedObjStat=false]
2022-07-13 17:33:13 INFO TransferOperationsHelper:206 - processGetOfSingleFile()
2022-07-13 17:33:13 INFO TransferOperationsHelper:232 - get of single file...filtered?
2022-07-13 17:33:13 INFO DefaultTransferControlBlock:132 - filtering: /xxxxx/home/xxxxx/OpenVPN profile update for 2FA How To Guide.pdf
2022-07-13 17:33:13 INFO DefaultTransferControlBlock:135 - no filter
2022-07-13 17:33:13 INFO TransferOperationsHelper:257 - filter passed, process...
2022-07-13 17:33:13 INFO DataObjectAOImpl:988 - getDataObjectFromIrods()
2022-07-13 17:33:13 INFO DataObjectAOImpl:998 - irodsFileToGet:/xxxxx/home/xxxxx/OpenVPN profile update for 2FA How To Guide.pdf
2022-07-13 17:33:13 INFO DataObjectAOImpl:999 - localFileToHoldData:/usr/local/tomcat/tmp-ticket-files/OpenVPN profile update for 2FA How To Guide.pdf
2022-07-13 17:33:13 INFO DataObjectAOImpl:1024 - checking to see if this is a restart...
2022-07-13 17:33:13 INFO DataObjectAOImpl:1600 - retrieveRestartInfoIfAvailable()
2022-07-13 17:33:13 INFO IRODSFileImpl:779 - length() for path:/xxxxx/home/xxxxxOpenVPN profile update for 2FA How To Guide.pdf
2022-07-13 17:33:13 INFO IRODSFileSystemAOImpl:427 - getObjStat(final String irodsAbsolutePath)
2022-07-13 17:33:13 INFO CollectionAndDataObjectListAndSearchAOImpl:1628 - retrieveObjectStatForPathWithHeuristicPathGuessing()
2022-07-13 17:33:13 INFO CollectionListingUtils:1278 - ObjStat [absolutePath=/xxxxx/home/xxxxx/OpenVPN profile update for 2FA How To Guide.pdf, objectPath=, objectType=DATA_OBJECT, dataId=10270, checksum=sha2:CD0KZoxoByUP7xb9gknXJDTUnYsXKBkJCqXTCAkEIis=, ownerName=xxxxx ownerZone=xxxxx, objSize=251741, createdAt=Thu Apr 21 06:42:10 GMT 2022, modifiedAt=Thu Apr 21 06:42:10 GMT 2022, specColType=NORMAL, collectionPath=, cacheDir=, cacheDirty=false, replNumber=0, standInGeneratedObjStat=false]
2022-07-13 17:33:13 INFO DataObjectAOImpl:1054 - testing file length to set parallel transfer options
2022-07-13 17:33:13 INFO DataObjectAOImpl:1064 - target local file: /usr/local/tomcat/tmp-ticket-files/OpenVPN profile update for 2FA How To Guide.pdf
2022-07-13 17:33:13 INFO DataObjectAOImpl:1065 - from source file: /xxxxx/home/xxxxx/OpenVPN profile update for 2FA How To Guide.pdf
2022-07-13 17:33:13 INFO DataObjectAOImpl:1342 - process get after resource determined
2022-07-13 17:33:13 INFO DataObjectAOImpl:1345 - no transfer status callback listener provided
2022-07-13 17:33:13 INFO LocalFileUtils:233 - local file does not exist, will attempt to create local file: /usr/local/tomcat/tmp-ticket-files/OpenVPN profile update for 2FA How To Guide.pdf
2022-07-13 17:33:13 ERROR TransferOperationsHelper:295 - exception in transfer, will abandon the connection and rethrow
org.irods.jargon.core.exception.JargonException: Unknown iRODS exception code recieved:-893000
at org.irods.jargon.core.connection.IRODSErrorScanner.inspectAndThrowIfNeeded(IRODSErrorScanner.java:105)
at org.irods.jargon.core.connection.IRODSMidLevelProtocol.processMessageInfoLessThanZero(IRODSMidLevelProtocol.java:1542)
at org.irods.jargon.core.connection.IRODSMidLevelProtocol.readMessage(IRODSMidLevelProtocol.java:1046)
at org.irods.jargon.core.connection.IRODSMidLevelProtocol.readMessage(IRODSMidLevelProtocol.java:1014)
at org.irods.jargon.core.connection.IRODSMidLevelProtocol.irodsFunction(IRODSMidLevelProtocol.java:427)
at org.irods.jargon.core.connection.IRODSMidLevelProtocol.irodsFunction(IRODSMidLevelProtocol.java:553)
at org.irods.jargon.core.connection.IRODSMidLevelProtocol.irodsFunction(IRODSMidLevelProtocol.java:909)
at org.irods.jargon.core.pub.DataObjectAOImpl.processGetAfterResourceDetermined(DataObjectAOImpl.java:1355)
at org.irods.jargon.core.pub.DataObjectAOImpl.getDataObjectFromIrods(DataObjectAOImpl.java:1076)
at org.irods.jargon.core.pub.TransferOperationsHelper.processGetOfSingleFile(TransferOperationsHelper.java:292)
at org.irods.jargon.core.pub.DataTransferOperationsImpl.processGetAfterAnyConnectionRerouting(DataTransferOperationsImpl.java:476)
at org.irods.jargon.core.pub.DataTransferOperationsImpl.getOperation(DataTransferOperationsImpl.java:300)
at org.irods.jargon.ticket.TicketClientOperationsImpl.getOperationFromIRODSUsingTicket(TicketClientOperationsImpl.java:121)
at com.emc.metalnx.services.irods.TicketClientServiceImpl.getFileFromIRODSUsingTicket(TicketClientServiceImpl.java:142)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
Please post the following:
ils -A
)iticket ls <ticket_string>
)We've determined that this is due to a checksum operation being requested following the transfer. To fix this issue, we have to make Metalnx override the Jargon checksum properties on startup.
The change introduced in DICE-UNC/jargon-irods-ext affects all transfers. To disable checksum operations following a transfer, set compute.checksum
to false
in metalnx.properties.
This change will require restarting the Metalnx server.
Hi we are having issues with Metalnx, we cannot download from the Tickets. Please see the errors below: