Inspect New Pipeline Products

[acviana]

Dylan, while you were away @walyssonBarbosa ran the pipeline over some new files (new in the sense we haven't processed them with our pipeline before). We made some changes to the pipeline, they shouldn't change how the outputs are looking though. I'd like you to check out some of those files out to make sure everything looks normal.

The first thing I'd like you to do is use the command line to head over to /astro/mtpipeline/logs/run_imaging_pipeline/. Here you will find our log files from the script that runs the pipeline. These log files are just text files. So, do a little googling and see if you can figure out how to open text files from the command line. Have fun! :sunglasses:

[acviana]

Actually, see if you can find 5 different ways of open a text file from the command line. Hint, avoid emacs and vim or any other text editor.

[ghost]

cat less head tail nano

[ghost]

either more or nano

[acviana]

:+1:

Let's get together now and discuss what you can so with you new-found skill at opening text files.

Just FYI you can make a list like this:

 - thing 1
 - thing 2
 - thing 3

This will end up looking like this:

• thing 1
• thing 2
• thing 3

This uses a syntax called Markdown which a type of markup language. There are several markup languages such as Hypertext Markup Language (HTML) which is what everything on the web is built around. You can learn more about all the fun things you can do by clicking on the Parsed as Markdown button above the "Leave a comment" box.

[ghost]

[tac-osx09:mtpipeline/logs/run_imaging_pipeline] dhazlett% grep -c "Running cr_reject" run_imaging_pipeline_2014-06-24-10-41.log 137 [tac-osx09:mtpipeline/logs/run_imaging_pipeline] dhazlett% grep -c "Running png" run_imaging_pipeline_2014-06-24-10-41.log 144 [tac-osx09:mtpipeline/logs/run_imaging_pipeline] dhazlett% grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-24-10-41.log 127 [tac-osx09:mtpipeline/logs/run_imaging_pipeline] dhazlett% grep -c "Running Drizzle" run_imaging_pipeline_2014-06-24-10-41.log 500 [tac-osx09:mtpipeline/logs/run_imaging_pipeline] dhazlett% grep -c Completed run_imaging_pipeline_2014-06-24-10-41.log 145

[acviana]

Try editing your comment with code blocks to make it a little more readable, for example taking this first line:

$ grep -c "Running cr_reject" run_imaging_pipeline_2014-06-24-10-41.log 137

[ghost]

grep -c "Running png" run_imaging_pipeline_2014-06-23-20-37.log 132 grep -c "Running cr_reject" run_imaging_pipeline_2014-06-23-20-37.log 121 grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-23-20-37.log 121 grep -c "Running Drizzle" run_imaging_pipeline_2014-06-23-20-37.log 482 grep -c Completed run_imaging_pipeline_2014-06-23-20-37.log 138

[ghost]

grep -c "Running cr_reject" run_imaging_pipeline_2014-06-23-14-34.log 103 grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-23-14-34.log 104 grep -c "Current File" run_imaging_pipeline_2014-06-23-14-34.log 115 grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-23-14-34.log 103 grep -c Completed run_imaging_pipeline_2014-06-23-14-34.log 11 grep -c "Running Drizzle" run_imaging_pipeline_2014-06-23-14-34.log 0 grep -c "Running png" run_imaging_pipeline_2014-06-23-14-34.log 11

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-23-14-33.log 143 grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-23-14-33.log 143 grep -c "Running cr_reject" run_imaging_pipeline_2014-06-23-14-33.log 143 grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-23-14-33.log 143 grep -c "CRITICAL" run_imaging_pipeline_2014-06-23-14-33.log 143

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-12-05-55.log 108 grep -c "Running cr_reject" run_imaging_pipeline_2014-06-12-05-55.log 108 grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-12-05-55.log 108 grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-12-05-55.log 108 grep -c "CRITICAL" run_imaging_pipeline_2014-06-12-05-55.log 108

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-23-20-36.log 126 grep -c "Running cr_reject" run_imaging_pipeline_2014-06-23-20-36.log 91 grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-23-20-36.log 91 grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-23-20-36.log 116 grep -c "CRITICAL" run_imaging_pipeline_2014-06-23-20-36.log 5 grep -c "Completed" run_imaging_pipeline_2014-06-23-20-36.log 120

[acviana]

That works, so does this though :)

grep -c "Current File" run_imaging_pipeline_2014-06-23-20-36.log  126
grep -c "Running cr_reject" run_imaging_pipeline_2014-06-23-20-36.log  91
grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-23-20-36.log  91
grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-23-20-36.log  116
grep -c "CRITICAL" run_imaging_pipeline_2014-06-23-20-36.log  5
grep -c "Completed" run_imaging_pipeline_2014-06-23-20-36.log  120

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-23-20-34.log 68 grep -c "Running cr_reject" run_imaging_pipeline_2014-06-23-20-34.log 68 grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-23-20-34.log 68 grep -c "Running Drizzle" run_imaging_pipeline_2014-06-23-20-34.log 136 grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-23-20-34.log 68 grep -c "CRITICAL" run_imaging_pipeline_2014-06-23-20-34.log 68

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-23-20-33.log 60 grep -c "Running cr_reject" run_imaging_pipeline_2014-06-23-20-33.log 60 grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-23-20-33.log 60 grep -c "Running Drizzle" run_imaging_pipeline_2014-06-23-20-33.log 240 grep -c "Completed" run_imaging_pipeline_2014-06-23-20-33.log 60 grep -c "Running png" run_imaging_pipeline_2014-06-23-20-33.log 60

[ghost]

^grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-23-20-33.log 60

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-23-20-32.log  60
grep -c "Running png" run_imaging_pipeline_2014-06-23-20-32.log  60
grep -c "Running cr_reject" run_imaging_pipeline_2014-06-23-20-32.log  60
grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-23-20-32.log  60
grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-23-20-32.log  60
grep -c "Running Drizzle" run_imaging_pipeline_2014-06-23-20-32.log  240
grep -c "Completed" run_imaging_pipeline_2014-06-23-20-32.log  60

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-23-14-35.log  66
grep -c "Running cr_reject" run_imaging_pipeline_2014-06-23-14-35.log  66
grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-23-14-35.log  66
grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-23-14-35.log  66
grep -c "CRITICAL" run_imaging_pipeline_2014-06-23-14-35.log  66

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-11-09-29.log  38
grep -c "Running cr_reject" run_imaging_pipeline_2014-06-11-09-29.log  38
grep -c "CRITICAL" run_imaging_pipeline_2014-06-11-09-29.log  37

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-24-17-48.log  196
grep -c "Running cr_reject" run_imaging_pipeline_2014-06-24-17-48.log  21
grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-24-17-48.log  21
grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-24-17-48.log  23
grep -c "Running Drizzle" run_imaging_pipeline_2014-06-24-17-48.log  88
grep -c "Running png" run_imaging_pipeline_2014-06-24-17-48.log  21
grep -c "Completed" run_imaging_pipeline_2014-06-24-17-48.log  194

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-20-13-27.log  54
grep -c "Running cr_reject" run_imaging_pipeline_2014-06-20-13-27.log  26
grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-20-13-27.log  25
grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-20-13-27.log  53
grep -c "Running Drizzle" run_imaging_pipeline_2014-06-20-13-27.log  212
grep -c "Running png" run_imaging_pipeline_2014-06-20-13-27.log  53
grep -c "Completed" run_imaging_pipeline_2014-06-20-13-27.log  53

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-19-10-38.log  20
grep -c "Running cr_reject" run_imaging_pipeline_2014-06-19-10-38.log  20
grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-19-10-38.log  20
grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-19-10-38.log  20
grep -c "CRITICAL" run_imaging_pipeline_2014-06-19-10-38.log  20

[ghost]

grep -c "Current File" run_imaging_pipeline_2014-06-19-09-07.log  18
grep -c "CRITICAL" run_imaging_pipeline_2014-06-19-09-07.log  18
grep -c "Running cr_reject" run_imaging_pipeline_2014-06-19-09-07.log  18
grep -c "Running Astrodrizzle" run_imaging_pipeline_2014-06-19-09-07.log  18
grep -c "Done running cr_reject" run_imaging_pipeline_2014-06-19-09-07.log  18

[acviana]

Here is a link with an example of using the AND operator in grep http://www.thegeekstuff.com/2011/10/grep-or-and-not-operators/

[ghost]

File name: ua1c7101m_c0m_center_single_sci_linear.png Proposal #: 11292 This image is just one of the many png files that contain this ring-like obstruction. The original pictures also contain this. We do not why it is covering Uranus or in the image.

[ghost]

In the middle of the top right quadrant, there is a detector artifact, a fault in the in the instrument. This is supported by the constant position of the grey dot throughout multiple frames while the moon around Uranus are changing position in their orbits.