Add ability to send user raw data

[ministerofsillywalks]

I added a checkbox to observe.php, made it so that plot,py and plot_hi.py save csv's for each graph, and added those csv's to the message in observe.py

[ministerofsillywalks]

To be clear, I did find a way to test plot_hi, and did so.

[0xCoto]

I think it would be nice (e.g. more "Pythonic") to not have to import redundant libraries (and force the user to install pandas too etc.). You might want to check this out. It's very easy to store the data as a .csv file in numpy directly with np.savetxt and delimiter=','.

Also, I haven't had much time to look at the changes made to observe.py yet. Does the user also get sent the .dat, or some .csv file?

[ministerofsillywalks]

I played around with np.savetxt a bit and figured out a way to to pretty much the same things that I did with pandas, so I'll change plot.py and plot_hi.py to remove that dependency.

Observe.py sends the .dat file, and two .csv's for frequency and time data. I.e. one has freq, zmean, andestimate_S_N_simple(spectrum,mask) if plot_hi is used, and the other .csv has t and w. I included the csv files in case someone doesn't want to or can't run plot.py, but still wants to have data.

[0xCoto]

Hey, sorry, just got the time to take a look at your new PR. Can you please rename things like zmean to something else to not confuse the user as to what 'z' means? Maybe averaged_spectrum.csv etc. would be a bit more clear. 🙂

[ministerofsillywalks]

Yes, I'll do that.

[ministerofsillywalks]

I changed the name of the .csv's and the names of the columns in the csv.

[0xCoto]

Please re-submit observe.php with the new file as the base. It was slightly outdated, so I pushed the updated version to git. observe.py, plot.py and plot_hi.py look fine, so you don't need to change anything about it.

Also, for UI purposes, please add the field under the email option, and use an dropdown menu instead of a checkbox/radio button, so it looks better, like so:

            Please enter an email address to get notified once the observation is complete.<p style="font-size:3px;">&emsp;</p>
            <div class="wrap-input100 validate-input" data-validate = "A valid email is required.">
                <label class="label-input100" for="email">Email Address</label>
                <input id="email" class="input100" type="text" name="email" placeholder="Enter your email..." required>
                <span class="focus-input100"></span>
            </div>
            <div class="wrap-input100">
                <div class="label-input100">Would you like to recieve your raw data as a .csv file?</div>
                <div>
                    <select class="js-select2" name="raw_data">
                        <option value="1">Yes</option>
                        <option value="0" selected="selected">No</option>
                    </select>
                    <div class="dropDownSelect2"></div>
                </div>
                <span class="focus-input100"></span>
            </div>

EDIT: I also applied some changes to observe.py, plot.py and plot_hi.py, so please base your changes on these files, otherwise there will be problems (because the directories are hardcoded, and the root username is no longer pictor, but pi).

[0xCoto]

@ministerofsillywalks Can you fix the directories in plot.py and plot_hi.py as well from pictor to pi please?

[ministerofsillywalks]

That should be everything.

[0xCoto]

Sorry for the late implementation, but looks like it's working fine! Only thing I had to change is remove the sending of the observation.dat file. My only concern with this is on large-size observations (e.g. 1 hour with 100 bins at 2048 channels), the mail server will have issues sending the attachment, and completely crash on trying to send the observation to the user. But I think the .csv files are sufficient.