"directory name not allowed" error in StorageInsert API

[nataliaratnikova]

Error reported by Beijing site, where one of the users directories names contains a colon, and does not pass the directory name check implemented in PHEDEX/perl_lib/PHEDEX/Web/API/StorageInsert.pm:

$k =~ m%^/[A-Za-z0-9_.-/]*$% || return PHEDEX::Web::Util::http_error(400,'directory name not allowed');

[nataliaratnikova]

The regex used in the API check corresponds to the POSIX standard. Most LINUX systems do not place any restriction on directory names, except for a slash and null characters. The proper fix would be to check for a leading /. That's all. Here are the considerations:

• since it is our goal to track space usage, we should not reject site info just because users create non-posix compliant directory names
• if certain names cause problems in applications like CRAB, the latter have to deal with it. In SpaceMon we could do a sanity check as a "courtesy" option.
• if we check for non-standard names in SpaceMon, it should be done in the dump parsing step. The upload API handles aggregated information, which does not include all directory names.

[nataliaratnikova]

Fix will require a website release. Until then site can use additional option --level 5 (or less) as a workaround. This will force a higher level of aggregation (default=6), and the offending directory won't be included in the upload record.

[nataliaratnikova]

Assigned to myself. The check for a leading slash in directory names already exists in the parsing. Strict posix check on the server side is removed, but I still leave a check for a leading slash on the server side as we foresee that sites may use their own aggregation procedure in the future. Change is pushed to master branch, commit: 69357cdf0a154bbfedff3d8bff3259e0a60982bd