In https://github.com/California-Planet-Search/KPF-Pipeline/issues/375, Cindy mentioned that the logs from running the pipeline for a particular spectrum are in /code/KPF-Pipeline/pipeline.log. Presumably, this is a relative path from running the DRP in Docker. We don't have a /code directory on shrek.caltech.edu. I'm not sure where to find those logs.
To me, it makes sense to have the logs for the "standard" output of the DRP (which goes into subdirectories of /data/kpf/ be stored in a way that users can inspect them. For example, we could create a file like /data/kpf/L1/YYYYMMDD/KP.YYYYMMDD.NNNNN.NN_L1.log, or something that would be clearly identified with the output file (/data/kpf/L1/YYYYMMDD/KP.YYYYMMDD.NNNNN.NN_L1.fits). Or maybe that clutters the L1 directory too much and it should we should have a parallel directory structure for logs like /data/kpf/log/L1/YYYYMMDD/.
I'm not sure. The main point of this issue is that I suggest that we should have a standard way for users to inspect the logs. A secondary point is that the logs should be overwritten during reprocessing runs that overwrite data products. If the logs are stored in a standard place, they could be displayed on Jump (this is a lower priority).
bjfultn
commented
1 year ago
In https://github.com/California-Planet-Search/KPF-Pipeline/issues/375, Cindy mentioned that the logs from running the pipeline for a particular spectrum are in
/code/KPF-Pipeline/pipeline.log. Presumably, this is a relative path from running the DRP in Docker. We don't have a/codedirectory onshrek.caltech.edu. I'm not sure where to find those logs.To me, it makes sense to have the logs for the "standard" output of the DRP (which goes into subdirectories of
/data/kpf/be stored in a way that users can inspect them. For example, we could create a file like/data/kpf/L1/YYYYMMDD/KP.YYYYMMDD.NNNNN.NN_L1.log, or something that would be clearly identified with the output file (/data/kpf/L1/YYYYMMDD/KP.YYYYMMDD.NNNNN.NN_L1.fits). Or maybe that clutters the L1 directory too much and it should we should have a parallel directory structure for logs like/data/kpf/log/L1/YYYYMMDD/.I'm not sure. The main point of this issue is that I suggest that we should have a standard way for users to inspect the logs. A secondary point is that the logs should be overwritten during reprocessing runs that overwrite data products. If the logs are stored in a standard place, they could be displayed on Jump (this is a lower priority).