search

HEPCloud / decisionengine

HEPCloud Decision Engine framework
Apache License 2.0
6 stars 25 forks source link

Certain debug inputs from channels now only visible in systemctl status output in DE 2.0rc2 #635

Open StevenCTimm opened 2 years ago

StevenCTimm commented 2 years ago

There are some debug outputs from the Amazon sources which previously were going to standard out (or possibly standard error) and now seem to be visible only via the systemctl status command . since I believe that systemctl status can be run by a non-privileged user we need to do something about this. Note that one of those outputs is an Amazon Secret Access Key which I've redacted.

Some of this may have also been going in on 1.7.

Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'date': 'Thu, 24 Mar 2022 11:58:47 GMT', Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'x-amzn-requestid': '8bd107c7-d35e-44fe-a1ca-f68d3fd6c7c2'}, Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'HTTPStatusCode': 200, Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'RequestId': '8bd107c7-d35e-44fe-a1ca-f68d3fd6c7c2', Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'RetryAttempts': 0}} Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: {'AssumedRoleUser': {'Arn': 'arn:aws:sts::159067897602:assumed-role/CalculateBill/rol...ssion', Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'AssumedRoleId': 'AROAI3T6KYFHZFV7446KK:roleSwitchSession'}, Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'Credentials': {'AccessKeyId': 'ASIASKCJPP4BMLZZYAL2', Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'Expiration': datetime.datetime(2022, 3, 24, 14, 0, 30, tzinfo=tzlocal()), Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'SecretAccessKey': 'xxxxxxx',

mambelli commented 1 year ago

@vitodb could you check if the latest changes to master (#670) fixed this?

StevenCTimm commented 1 year ago

(This certainly was also going on in 1.7 it turns out, it's been happening for quite a while).

goodenou commented 11 months ago

I am curious about what the status is here. Has the problem been fixed?

StevenCTimm commented 11 months ago

Haven't tested a 2.0 series DE for a while but I'm not aware of any fix that has been done. just start up a 2.0 de and do de-client --status and see all the junk that spews out, especially if you have debug enabled. Certainly it is still happening in the 1.7.4 debug issue.

goodenou commented 10 months ago

The PR that (I believe) fixed these issues #670 was merged in on December 9th 2022, which is after the 1.7.4 and 2.0 DE tags were cut. Can we try running something that is dated post December 9th as a test?

StevenCTimm commented 9 months ago

The verbose output is still available in systemctl status as of DE 1.7.5 And I have verified it can be seen by a non-privileged user either running systemctl status or journalctl.

[decisionengine@hepcsvc03 ~]$ journalctl _SYSTEMD_UNIT=decisionengine.service -S '2023-10-03'

goodenou commented 9 months ago

OK. I will look into this. Is it ONLY the Amazon sources, or are there other sources that exhibit this behavior?

StevenCTimm commented 9 months ago

There are others.. on hepcsvc03 you can do journalctl -xe and see the history of the other sources that are there.

mambelli commented 3 months ago

@StevenCTimm does #702 fix all the remaining ones or are there others?