Changes to make file: by the end of the week? (5/27)

[stephlj]

(1) How to deal with multiple data sets to be analyzed separately: different make analysis in each subdir? one make file and a parameter that tells it which subdir to analyze?

(2) Have make load from and save to my current data structure

(3) Add a parameter to each traces's params.yml file called discard that can be 1 or 0. If discard: 0, it shouldn't be included in the analysis (and it would be great if make would ignore it entirely, but it's ok if make continues to update it).

[mattjj]

For (1), I just pushed a simple solution to master (b5d4889). Say your pwd (in Matlab or your shell) is ~/Desktop/analysis/2015-12-04_traces so that the directory structure (including the pwd at the top) looks like

2015-12-04_traces/
    data/
        params.yml
        SNF2h103nMATP1mMHighFPS_2015_Dec04_Spot1_106_Results.mat
        SNF2h103nMATP1mMHighFPS_2015_Dec04_Spot1_106_Results.params.yml

Now in that directory if you run

make -f ~/packages/slopey/Makefile

(assuming the slopey repository is in ~/packages/slopey), you'll get

2015-12-04_traces/
    data/
        params.yml
        SNF2h103nMATP1mMHighFPS_2015_Dec04_Spot1_106_Results.mat
        SNF2h103nMATP1mMHighFPS_2015_Dec04_Spot1_106_Results.params.yml
    results/
        SNF2h103nMATP1mMHighFPS_2015_Dec04_Spot1_106_Results.results.pkl
        all_results.mat
    figures/
        SNF2h103nMATP1mMHighFPS_2015_Dec04_Spot1_106_Results.pdf
        SNF2h103nMATP1mMHighFPS_2015_Dec04_Spot1_106_Results_prior.pdf

So basically this strategy is to keep the Makefile as it is, nice and simple, but to handle running it on different datasets by using your current working directory to determine the dataset.

You can also wrap this in a trivial Bash script like this:

#!/bin/bash
cd $1 && make -f ~/packages/slopey/Makefile

so that if that text is in a file analyze_traces.sh you can just run this command from anywhere

analyze_traces.sh ~/Desktop/analysis/2015-12-04_traces

[mattjj]

Can you explain (2)? I don't remember what that means.

[mattjj]

For (3), the commit I just pushed to master (45ae127) reads discard from a params.yml file. Write discard: true to discard, and either write discard: false or just don't have a discard field to use the trace.

The only remaining item in this issue is (2).

[stephlj]

I think you fixed (2) already; the idea was to have make save results and figures to the directory that it loads data from, which it looks like is what you're doing now.

[mattjj]

Woo closing it! Maybe premature but I like to close.

[stephlj]

Alrighty, running into some trouble ...

First I made a shell script like this:

!/bin/bash

cd "$1" && make -f "~/Documents/UCSF/Narlikar lab/HMM analysis Slopey/slopey/Makefile"

and called it "Analyze_Slopey.sh" and saved it in the same directory as the Makefile and my Matlab analysis scripts.

Next I edited the Makefile slightly because I wanted the subdirectories that make makes to be results_slopey and figures_slopey instead of results and figures.

Lastly in the directory where the .sh file is (using Terminal) I tried running: ./Analyze_Slopey.sh "/Users/Steph/Documents/UCSF/Narlikar lab/smFRET data analysis/HMM results/Ino8015nMATP1mMTestMake"

(clearly Luke helped me out with this :P)

I got this output: usage: dirname path usage: dirname path usage: dirname path usage: dirname path usage: dirname path usage: dirname path make: *\ No rule to make target /scripts/collect_results.py', needed byresults_slopey/all_results.mat'. Stop.

I tried looking through the Makefile to see if I could figure out what's wrong but it looks right to me ...

I just pushed my changes to github, perhaps you can take a look?

[stephlj]

Fixed!

New bash script Analyze_Slopey.sh (in the folder above the git repo):

!/bin/bash

cd $1 && make $2 -f ~/Documents/symlink/HMM_analysis_Slopey/slopey/Makefile

And some notes from Luke's debugging that I'm sticking here for my future reference:

symlink required for Makefile because it uses $(lastword $(MAKEFILE_LIST)), which picks

out the last, space-delimited element; unless you never want this makefile to be

include-able by another, there's no easy way around the space-delimited functionality

cd ~/Documents mkdir symlink cd symlink ln -s "/Users/Steph/Documents/UCSF/Narlikar lab/HMM analysis Slopey" HMM_analysis_Slopey

cd "~/Documents/UCSF/Narlikar lab/HMM analysis Slopey/Symlinks_Data" ln -s "/Users/Steph/Documents/UCSF/Narlikar lab/smFRET data analysis/HMM results/Ino8015nMATP1mMTestMake" Ino8015nMATP1mMTestMake

cd ~/Documents/symlink/slopey ./Analyze_Slopey.sh Symlinks_Data/Ino8015nMATP1mMTestMake ./Analyze_Slopey.sh Symlinks_Data/Ino8015nMATP1mMTestMake clean

Makefile debugging (found that we needed abspath instead of realpath):

REALP = $(realpath $(lastword $(MAKEFILE_LIST))) ABSP = $(abspath $(lastword $(MAKEFILE_LIST))) $(info $$ROOT is [${ROOT}]) $(info $$MAKEFILE_LIST is [${MAKEFILE_LIST}]) $(info $$REALP is [${REALP}]) $(info $$ABSP is [${ABSP}])

[mattjj]

Woohoo! Nice fixes, Luke!