mar-file-system / GUFI

Grand Unified File-Index
Other
46 stars 23 forks source link

Issues with folders with spaces #101

Closed brockpalen closed 1 year ago

brockpalen commented 2 years ago

We're having issues with folders with spaces in names:

eg

Project_RNA-Seq 5-AZA_2

Will create error messages like:

input-dir '/tmp/GUFI/umms-cspeers//Project_RNA-Seq' is not a directory
input-dir '/tmp/GUFI/umms-cspeers//5-AZA_2' is not a directory

The query being run is located: https://github.com/umich-arc/gufi-archive/blob/master/reports/dirsum.sh

I can confirm that the index does have the path correctly. The error appears to be in the query but simpler queries like https://github.com/umich-arc/gufi-archive/blob/master/reports/totals.sh

Using GUFI Index located in: /tmp/GUFI/umms-cspeers/
Reporting on data older than 180 days last accessed

count                     sizeGB  oldsize  percent  
137661                    20173   5992     29%   

Work just fine.

garygrider commented 2 years ago

Interesting,

-I "CREATE TABLE sument (username text, name text, size int64, atime int64, oldsize int64);" $1/$dir

will the $dir really work in any command that is walking with spaces in the dir name

There might be an easier way, sweep through and make the db’s and sweep again to add records, in other words use gufi to do the walk, not find Like bfq -e “create db…” (not in a find)

Then bfw -e ‘insert into” (not in a find)

Or there used to be a -S “sql statement” and even a -T “sql statement” And there used to be an order Run -T then based on a flag run -S or stop and then based on a flag run -E based on a flag It was like and and/or flag

So in one swoop

Bfq -n x -S “create table …..’ -E “populate table. And don’t use find, just let gufi traverse

And

There use to be a way to run more than one sql on the same -E

I think it was -E ‘create table blah; insert …;’ and it would stack them

And if you need conditionals, there are conditionals in sql itself too.

I honestly don’t know how much of that function is still in the gufi-query anymore

But doing things in shell’s gets you into these odd character string issues, but just doing them gufi its all in C and it’s a real string with a null at the end 😊

if gufi-query still has all this function, I think there is a far more elegant way to do what you are trying to do (or at least what I think you are trying to do).

just guessing, I should leave this to the smarter people

From: Brock Palen @.> Sent: Wednesday, October 19, 2022 9:28 AM To: mar-file-system/GUFI @.> Cc: Subscribed @.***> Subject: [EXTERNAL] [mar-file-system/GUFI] Issues with folders with spaces (Issue #101)

We're having issues with folders with spaces in names:

eg

Project_RNA-Seq 5-AZA_2

Will create error messages like:

input-dir '/tmp/GUFI/umms-cspeers//Project_RNA-Seq' is not a directory

input-dir '/tmp/GUFI/umms-cspeers//5-AZA_2' is not a directory

The query being run is located: https://github.com/umich-arc/gufi-archive/blob/master/reports/dirsum.shhttps://urldefense.com/v3/__https:/github.com/umich-arc/gufi-archive/blob/master/reports/dirsum.sh__;!!Bt8fGhp8LhKGRg!HWyigz6cAKS9Hpe-c6dzjeMgrzsLOILcI7y4l6eX2d5tbu1nOvtoqHO7W_6nLPw3604tII1KL-rPc7ktKDK5O2lV$

I can confirm that the index does have the path correctly. The error appears to be in the query but simpler queries like https://github.com/umich-arc/gufi-archive/blob/master/reports/totals.shhttps://urldefense.com/v3/__https:/github.com/umich-arc/gufi-archive/blob/master/reports/totals.sh__;!!Bt8fGhp8LhKGRg!HWyigz6cAKS9Hpe-c6dzjeMgrzsLOILcI7y4l6eX2d5tbu1nOvtoqHO7W_6nLPw3604tII1KL-rPc7ktKNNRyOwU$

Using GUFI Index located in: /tmp/GUFI/umms-cspeers/

Reporting on data older than 180 days last accessed

count sizeGB oldsize percent

137661 20173 5992 29%

Work just fine.

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/mar-file-system/GUFI/issues/101__;!!Bt8fGhp8LhKGRg!HWyigz6cAKS9Hpe-c6dzjeMgrzsLOILcI7y4l6eX2d5tbu1nOvtoqHO7W_6nLPw3604tII1KL-rPc7ktKKBDeG0f$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ACZXDGKIKWRGCGYCMNO2QE3WEAHO3ANCNFSM6AAAAAARJH3RHI__;!!Bt8fGhp8LhKGRg!HWyigz6cAKS9Hpe-c6dzjeMgrzsLOILcI7y4l6eX2d5tbu1nOvtoqHO7W_6nLPw3604tII1KL-rPc7ktKIFq4tuu$. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

calccrypto commented 2 years ago

Hi. This print comes from the main function, meaning the path is not being passed into gufi_query properly . Please try quoting your input paths: $1/$dir -> "$1/${dir}"

brockpalen commented 5 months ago

Sorry old topic but closing the looop this was the issue and is now updated for all the reports in gufi-archive https://github.com/umich-arc/gufi-archive/blob/master/reports/dirsum.sh