Problem creating s3Ressources on IRODS provider server on a virtual machine

[AnthonySt-G]

Hello,

I followed the documentation (https://docs.irods.org/4.3.3/) to install IRODS 4.3.3 and the pluggin irods_resource_plugin_s3 on a specific professional environment. I add the information that we use a solution AWS s3 API like, but is not AWS. We have our own data center.

After setting the set_up, I tried to create an S3 resource using the command (https://github.com/irods/irods_resource_plugin_s3) : irods@hostname $ iadmin mkresc s3resc s3 $(hostname)://prefix/in/bucket "S3_DEFAULT_HOSTNAME=s3.us-east-1.amazonaws.com;S3_AUTH_FILE=/var/lib/irods/s3.keypair;S3_REGIONNAME=us-east-1;S3_RETRY_COUNT=1;S3_WAIT_TIME_SECONDS=3;S3_PROTO=HTTP;ARCHIVE_NAMING_POLICY=consistent;HOST_MODE=cacheless_attached"

That seems to have work, and when I list my resources, s3resc appears.

Now when i try to put an object in it with the command: irods@hostname $ iput -R s3resc foo.txt

i get this error : remote addresses: x.x.x.x ERROR: putUtil: put error for /zone/home/rods/myfile, status = -66000 status = -66000 SYS_NOT_SUPPORTED

But i can see the object directly here '/' when i do ils.

i tryed to remove it => impossible I tryed to see where was the problem, but my irods service didnt write logs, while i have 'info' level in my /etc/irods/server_config.json

As you can see, i m not really familiar with IRODS, but at each step of the installation/configuration i encounter(ed) problems that i must overcome with low informations an non explicit return error code.

So can you help me find an issue on this s3 ressource creation ? Have you some tips that you can give me for help me get better informations on the server/services IRODS errors ? Where are the important informations used by the irods_resource_plugin_s3 code for interact with the s3 data storage ? Do you eventually know how to get logs from the 4.3+ IRODS version (i already read the official doc that didn't help) ?

I don't really know how have support/help, so i created this issue. Help me please, i worked weeks on this technologie without results

Cordially, Anthony

[trel]

Hi Anthony,

Welcome - sorry you're having trouble.

You never told iRODS about your local s3 service... you only pointed at AWS (s3.us-east-1.amazonaws.com) with your command...

iadmin mkresc s3resc s3 $(hostname):/<s3BucketName>/prefix/in/bucket "S3_DEFAULT_HOSTNAME=s3.us-east-1.amazonaws.com;S3_AUTH_FILE=/var/lib/irods/s3.keypair;S3_REGIONNAME=us-east-1;S3_RETRY_COUNT=1;S3_WAIT_TIME_SECONDS=3;S3_PROTO=HTTP;ARCHIVE_NAMING_POLICY=consistent;HOST_MODE=cacheless_attached"

You need to update the various settings to point to your own S3 service...

iadmin mkresc s3resc s3 $(hostname):/<bucket_name>/prefix "S3_DEFAULT_HOSTNAME=<your_s3_host>;S3_AUTH_FILE=/var/lib/irods/s3.keypair;S3_REGIONNAME=us-east-1;S3_RETRY_COUNT=1;S3_WAIT_TIME_SECONDS=3;S3_PROTO=HTTP;ARCHIVE_NAMING_POLICY=consistent;HOST_MODE=cacheless_attached"

And make sure you populated /var/lib/irods/s3.keypair correctly with your credentials for that s3 service.

...

Additionally, you can see the error messages in the iRODS server log itself - that should help you understand what it is trying to do and WHY it did not succeed.

[AnthonySt-G]

Hello Terrell,

Obviously the requests used in my code have the right parameters, instead of the documentation command given in the issue to explain my problem, I am not in a position to provide public information regarding our use of irods.

I am also a computer scientist, I know how to read documentation, as said in the issue. Also the answers given do not provide me with any food for thought since they have all already been explored without success.

The outcome as I wrote it is a formal request for help from someone who might be able to interact with me in more detail and in a non-public manner, possibly by email exchange.

I remind you that I followed the various documentation available step by step without success when using the s3 plugin. I also have other problems with the service launched on the provider which does not generate logs. In addition, the error report during the execution of the code does not give any information as to the blocking step; it is impossible for me to move forward.

Please help me, what information do you need to investigate my case ?

Regarding your recommendations : You never told iRODS about your local s3 service... you only pointed at AWS (s3.us-east-1.amazonaws.com http://s3.us-east-1.amazonaws.com) with your command... => I used the good server storage dns of our own You need to update the various settings to point to your own S3 service... => I updated all this arguments : , , S3_AUTH_FILE=/var/lib/irods/auth_file, And make sure you populated /var/lib/irods/s3.keypair correctly with your credentials for that s3 service. => I named it "auth_file" => I used s3 key (line 1) and secret s3 key (line 2) like it is recommended in the documentation Cordially, Anthony

Le ven. 4 oct. 2024 à 22:19, Terrell Russell @.***> a écrit :

Hi Anthony,

Welcome - sorry you're having trouble.

You never told iRODS about your local s3 service... you only pointed at AWS (s3.us-east-1.amazonaws.com) with your command...

iadmin mkresc s3resc s3 $(hostname)://prefix/in/bucket "S3_DEFAULT_HOSTNAME=s3.us-east-1.amazonaws.com;S3_AUTH_FILE=/var/lib/irods/s3.keypair;S3_REGIONNAME=us-east-1;S3_RETRY_COUNT=1;S3_WAIT_TIME_SECONDS=3;S3_PROTO=HTTP;ARCHIVE_NAMING_POLICY=consistent;HOST_MODE=cacheless_attached"

You need to update the various settings to point to your own S3 service...

iadmin mkresc s3resc s3 $(hostname)://prefix "S3_DEFAULT_HOSTNAME=;S3_AUTH_FILE=/var/lib/irods/s3.keypair;S3_REGIONNAME=us-east-1;S3_RETRY_COUNT=1;S3_WAIT_TIME_SECONDS=3;S3_PROTO=HTTP;ARCHIVE_NAMING_POLICY=consistent;HOST_MODE=cacheless_attached"

And make sure you populated /var/lib/irods/s3.keypair correctly with your credentials for that s3 service.

...

Additionally, you can see the error messages in the iRODS server log itself - that should help you understand what it is trying to do and WHY it did not succeed.

— Reply to this email directly, view it on GitHub https://github.com/irods/irods_resource_plugin_s3/issues/2219#issuecomment-2394508424, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL74W4VI3IR4VNVT5L2LEF3ZZ3Z4ZAVCNFSM6AAAAABPMDYT72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJUGUYDQNBSGQ . You are receiving this because you authored the thread.Message ID: @.***>

[trel]

We are happy to provide paid support. Please contact info@irods.org.

The server writes messages to syslog.

https://docs.irods.org/4.3.3/system_overview/troubleshooting/#rsyslog-configuration