quota set by the plugin cannot block Metalnx user uploading data

[mstfdkmn]

We're running iRODS 4.2.10 on CentOS 7. Metalnx version is 2.4.0

Quotas set by the logical quotas for a collection can be over exceeded by using Metalnx. In other saying, the data upload process through Metalnx cannot be blocked by the logical quotas.

Whereas the maximum_number_of_data_objects operation can block exceeds, the logical_quotas::maximum_size_in_bytes operation cannot so. And this issue occurs for rodsusers.

AVUs defined for collection /u0137480/home/lt1_d_pilot_003:
attribute: irods::logical_quotas::maximum_size_in_bytes
value: 200
units:
----
attribute: irods::logical_quotas::total_number_of_data_objects
value: 10
units:
----
attribute: irods::logical_quotas::total_size_in_bytes
value: 159345
units:

[korydraughn]

If Metalnx is streaming (open, write, close) the bytes to iRODS, then this behavior is explained.

See https://github.com/irods/irods_rule_engine_plugin_logical_quotas#stream-operations

My guess is that Metalnx is using the streaming operations.

[mstfdkmn]

Regarding with the streaming operation, If I am interpreting the quoted part below correctly :

These changes have the following effects:

The plugin allows stream-based writes to violate the maximum bytes quota once. Subsequent stream-based creates and writes will be denied until the quotas are out of violation.

the following points are correct, right?

• The first streaming based upload can exceed the quotas set even if there is still only one byte exists before being over quoted.
• From the second upload on, the streaming based uploads should be rejected because the first streaming based upload already exceeded the quotas set.

If these are correct, then there seems the second point still not working properly. Because via Metalnx, we can continuously upload data.

Also, I couldnt test this for other clients using Jargon api because of the limit of our dev environment. So might it be related to the api that Metalnx use?

[korydraughn]

That is surprising. We'll take a look.

[korydraughn]

Looking at this with fresh eyes again, I want to confirm we're interpreting the documentation about streaming in the same way.

What that documentation is actually saying is ... a violation for streaming operations is only considered during a call to open. That means, when the REP sees the open PEPs fire, it will check to see if there are any violations. If there are violations, the REP blocks the operation. If there aren't any violations at the time of the open, the open succeeds and any number of write operations can be executed (which can result in violations).

@mstfdkmn Does that align with your understanding/expectations?

I've confirmed that Metalnx 2.6.1 honors those rules when run against iRODS 4.2.12 and what will be iRODS 4.3.2.

@mstfdkmn Are you still seeing an issue? If not, then I'm going to close this.

[mstfdkmn]

I guess there were different things in this thread. Apart from how to interpret the documentation, the other thing was about what discussed in the following thread https://github.com/irods/python-irodsclient/issues/379#issuecomment-1344649766. Please see that also Daniel created another issue while debugging this.

Anyway, we are no longer using the Metalnx client nor the logical quota. So I am not be able to easily test this again. Sorry! And yes this can be closed on my end.

[trel]

Please see that also Daniel created another issue while debugging this.

Dan created https://github.com/irods/irods_rule_engine_plugin_logical_quotas/issues/84 and it was fixed/merged in March 2023 - and then included in the 4.2.12.0 and 4.3.1.0 releases of this plugin.

[korydraughn]

All good.

I think things are working. Closing.