ssec / polar2grid

Tools for reading, remapping, and writing satellite instrument data.
http://www.ssec.wisc.edu/software/polar2grid/
GNU General Public License v3.0
72 stars 34 forks source link

Strange permissions of the conda-unpack file in Geo2Grid release 1.1.beta #444

Closed jpnIII closed 2 years ago

jpnIII commented 2 years ago

Hi all -- Dave is already familiar with this problem, but I had not filed an issue on it, as Dave had requested. Sorry Dave. So for accounting purposes, here is my issue:

After unpacking and setting up Geo2Grid release 1.1.beta as user "gops" on imaginator.ssec.wisc.edu, another user on imaginator ("tims") tried to execute geo2grid.sh (Tim just adjusted his $PATH via a ksh alias to have the proper Geo2Grid bin file included). Tim then ran into the following problem:

python: can't open file '/home/shared/bin/geo2grid_v_1_1_b/geo2grid-swbundle-20220308-201316/libexec/python_runtime/bin/conda-un pack': [Errno 13] Permission denied

-rwx--x--x 1 gops domain users 4266700 Mar 8 20:17 conda-unpack

I thought that this was a very strange set of permissions for a file to have, so I ran: chmod 755 conda-unpack

Tim is now able to run the Geo2Grid package.

Truthfully, I do not recall the order of steps taken to allow Tim to execute the 1.1.beta Geo2Grid. I have reason to believe though that changing permissions for activate, deactivate and ksu had no effect on the situation. I guess I would say that the bottom line would be that all of the files in the bin directory (especially conda-unpack, activate, deactivate and ksu) should have the same permissions as all of the other files (rwxr-xr-x). I think that would be cleaner.

djhoese commented 2 years ago

@jpnIII As mentioned in my email, the installation process for Geo2Grid requires the "installing" account to perform some operation (ex. run geo2grid.sh -h) at least once before any other user can execute any commands. The one-time install process (potentially) requires write permission to the entire extracted directory. It may not need it in the current revision, but may in the future depending on what packages are installed in the bundle. It at least needs permission to create a file in one specific location inside the directory. Lastly, in my own tests even though it complains about conda-unpack it seems to still continue executing.

Could you please remove the extracted directory for 1.1beta, re-extract it, run geo2grid.sh -h, and then have Tim try executing code. In general Geo2Grid was not designed/tested to be used by multiple users, but besides this one-time installation step of running any command, it shouldn't not be possible.

As for the files you've listed that you think should have changed permissions, none of those have executable permissions (except conda-unpack) because they aren't meant to be called directly. Conda-unpack does have execute permissions for all (user, group, other) but not read permissions for group/other, but it doesn't need read permissions. It is executed directly by my initialization scripts. So what I'm saying is I think the permissions are fine the way they are, we (the Geo2Grid team) just need to document that geo2grid.sh -h should be run by the installing user before any other users try to work with the scripts.

jpnIII commented 2 years ago

Thank you, Dave!

1.) I reinstalled 1.1.beta on both imaginator.ssec.wisc.edu and rci.ssec.wisc.edu as user “gops”. 2.) I ran: geo2grid.sh -v # The first message to appear was: “Running one-time initialization of environment...” 3.) After #2 was run, I noticed that: /home/shared/bin/geo2grid_v_1_1_b/geo2grid-swbundle-20220308-201316/libexec/python_runtime/bin/conda-unpack had rwx—x—x permissions (not saying that anything changed – just making this observation). 4.) I reinstalled /home/shared/bin/geo2grid_v_1_1_b/geo2grid-swbundle-20220308-201316/etc/polar2grid/colormaps/abi_l2_modified_cloud_top.cmap

That was all I did.

So now we wait until Mr. Schmit has a chanced to try things out sometime Monday.

Talk to you soon!

Sincerely,

Jim From: David Hoese @.> Sent: Saturday, April 2, 2022 3:33 PM To: ssec/polar2grid @.> Cc: James P. Nelson @.>; Mention @.> Subject: Re: [ssec/polar2grid] Strange permissions of the conda-unpack file in Geo2Grid release 1.1.beta (Issue #444)

@jpnIIIhttps://github.com/jpnIII As mentioned in my email, the installation process for Geo2Grid requires the "installing" account to perform some operation (ex. run geo2grid.sh -h) at least once before any other user can execute any commands. The one-time install process (potentially) requires write permission to the entire extracted directory. It may not need it in the current revision, but may in the future depending on what packages are installed in the bundle. It at least needs permission to create a file in one specific location inside the directory. Lastly, in my own tests even though it complains about conda-unpack it seems to still continue executing.

Could you please remove the extracted directory for 1.1beta, re-extract it, run geo2grid.sh -h, and then have Tim try executing code. In general Geo2Grid was not designed/tested to be used by multiple users, but besides this one-time installation step of running any command, it shouldn't not be possible.

As for the files you've listed that you think should have changed permissions, none of those have executable permissions (except conda-unpack) because they aren't meant to be called directly. Conda-unpack does have execute permissions for all (user, group, other) but not read permissions for group/other, but it doesn't need read permissions. It is executed directly by my initialization scripts. So what I'm saying is I think the permissions are fine the way they are, we (the Geo2Grid team) just need to document that geo2grid.sh -h should be run by the installing user before any other users try to work with the scripts.

— Reply to this email directly, view it on GitHubhttps://github.com/ssec/polar2grid/issues/444#issuecomment-1086717684, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AARELFY7IJ5ERLGKYPC6ZLLVDCVG5ANCNFSM5SKKQJWA. You are receiving this because you were mentioned.Message ID: @.**@.>>

jpnIII commented 2 years ago

Hi again, Dave –

1.) Neither Tim nor I have run into anything regarding conda-unpack. I take that as a win. Tim has run 1.1.b as user “tims”, and I have run 1.1.b as both user “gops” and user “jimn”. 2.) Probably not surprising to you, both Tim and I encountered the same problem when trying to generate band 02 (i.e., C02) imagery from RadC (i.e., CONUS) GOES-16 imagery: “ValueError: Aggregation factors are not integers.” was the error message returned. This error occurred when nothing regarding --method was specified on the geo2grid.sh commandline. 3.) When we added --method nearest to the geo2grid.sh commandline, things then seemed to work properly.

Do you need any other information from us?

Thank you for your help!

Sincerely,

Jim

From: James P. Nelson Sent: Monday, April 4, 2022 12:21 AM To: ssec/polar2grid @.>; ssec/polar2grid @.> Cc: Mention @.>; James P. Nelson @.>; Tim J Schmit - NOAA Federal @.***> Subject: RE: [ssec/polar2grid] Strange permissions of the conda-unpack file in Geo2Grid release 1.1.beta (Issue #444)

Thank you, Dave!

1.) I reinstalled 1.1.beta on both imaginator.ssec.wisc.edu and rci.ssec.wisc.edu as user “gops”. 2.) I ran: geo2grid.sh -v # The first message to appear was: “Running one-time initialization of environment...” 3.) After #2 was run, I noticed that: /home/shared/bin/geo2grid_v_1_1_b/geo2grid-swbundle-20220308-201316/libexec/python_runtime/bin/conda-unpack had rwx—x—x permissions (not saying that anything changed – just making this observation). 4.) I reinstalled /home/shared/bin/geo2grid_v_1_1_b/geo2grid-swbundle-20220308-201316/etc/polar2grid/colormaps/abi_l2_modified_cloud_top.cmap

That was all I did.

So now we wait until Mr. Schmit has a chanced to try things out sometime Monday.

Talk to you soon!

Sincerely,

Jim From: David Hoese @.**@.>> Sent: Saturday, April 2, 2022 3:33 PM To: ssec/polar2grid @.**@.>> Cc: James P. Nelson @.**@.>>; Mention @.**@.>> Subject: Re: [ssec/polar2grid] Strange permissions of the conda-unpack file in Geo2Grid release 1.1.beta (Issue #444)

@jpnIIIhttps://github.com/jpnIII As mentioned in my email, the installation process for Geo2Grid requires the "installing" account to perform some operation (ex. run geo2grid.sh -h) at least once before any other user can execute any commands. The one-time install process (potentially) requires write permission to the entire extracted directory. It may not need it in the current revision, but may in the future depending on what packages are installed in the bundle. It at least needs permission to create a file in one specific location inside the directory. Lastly, in my own tests even though it complains about conda-unpack it seems to still continue executing.

Could you please remove the extracted directory for 1.1beta, re-extract it, run geo2grid.sh -h, and then have Tim try executing code. In general Geo2Grid was not designed/tested to be used by multiple users, but besides this one-time installation step of running any command, it shouldn't not be possible.

As for the files you've listed that you think should have changed permissions, none of those have executable permissions (except conda-unpack) because they aren't meant to be called directly. Conda-unpack does have execute permissions for all (user, group, other) but not read permissions for group/other, but it doesn't need read permissions. It is executed directly by my initialization scripts. So what I'm saying is I think the permissions are fine the way they are, we (the Geo2Grid team) just need to document that geo2grid.sh -h should be run by the installing user before any other users try to work with the scripts.

— Reply to this email directly, view it on GitHubhttps://github.com/ssec/polar2grid/issues/444#issuecomment-1086717684, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AARELFY7IJ5ERLGKYPC6ZLLVDCVG5ANCNFSM5SKKQJWA. You are receiving this because you were mentioned.Message ID: @.**@.>>

djhoese commented 2 years ago

No this is good information. Thanks @jpnIII!

I can give you another G2G tarball soon (this week) that should have the native/nearest thing fixed. I think the original issue discussed here has been determined.

Just curious, do you guys process AHI data with Geo2Grid at all? I have a change that should clean up the true_color composite for it, but if you don't use AHI data then I can give you a tarball sooner.

jpnIII commented 2 years ago

Thanks again, Dave!

Scott Bachmeier has processed AHI data in the past using Geo2Grid. Do you know when the AHI true_color composite fix will be incorporated into the 1.1.b tarball? If it is not too far into the future (2 weeks?), then we will wait until both the native/nearest and AHI true_color composite fixes are incorporated before fiddling with the next iteration of the 1.1.b tarball.

Thank you.

Sincerely,

Jim From: David Hoese @.> Sent: Monday, April 4, 2022 1:12 PM To: ssec/polar2grid @.> Cc: James P. Nelson @.>; Mention @.> Subject: Re: [ssec/polar2grid] Strange permissions of the conda-unpack file in Geo2Grid release 1.1.beta (Issue #444)

No this is good information. Thanks @jpnIIIhttps://github.com/jpnIII!

I can give you another G2G tarball soon (this week) that should have the native/nearest thing fixed. I think the original issue discussed here has been determined.

Just curious, do you guys process AHI data with Geo2Grid at all? I have a change that should clean up the true_color composite for it, but if you don't use AHI data then I can give you a tarball sooner.

— Reply to this email directly, view it on GitHubhttps://github.com/ssec/polar2grid/issues/444#issuecomment-1087862166, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AARELF5TBIZPFQSJPX334GTVDMWFFANCNFSM5SKKQJWA. You are receiving this because you were mentioned.Message ID: @.**@.>>

djhoese commented 2 years ago

Yes. I would hope that it is this week. The changes are all in place they just have to be reviewed by other Satpy maintainers and merged. Then I just need to generate a tarball which is relatively easy.

jpnIII commented 2 years ago

Very good, Dave. Thank you.

Sincerely,

Jim

From: David Hoese @.> Sent: Monday, April 4, 2022 2:21 PM To: ssec/polar2grid @.> Cc: James P. Nelson @.>; Mention @.> Subject: Re: [ssec/polar2grid] Strange permissions of the conda-unpack file in Geo2Grid release 1.1.beta (Issue #444)

Yes. I would hope that it is this week. The changes are all in place they just have to be reviewed by other Satpy maintainers and merged. Then I just need to generate a tarball which is relatively easy.

— Reply to this email directly, view it on GitHubhttps://github.com/ssec/polar2grid/issues/444#issuecomment-1087923929, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AARELF54OL3H6V636AHVXZLVDM6JJANCNFSM5SKKQJWA. You are receiving this because you were mentioned.Message ID: @.**@.>>