How to Access of Ancillary, ql, sitl data sets?

neetasha commented 3 years ago

Step 1: Latest code?

[x] This issue is present in the most recent code (i.e. after running git pull this is still a problem).

Step 2: Describe your environment

irfu-matlab branch (i.e. "master", "devel"): devel
Matlab version used (i.e. "Matlab R2017a"): _R2020a___
Operating system (ie "Windows 7", "Mac OS X 10.12.3", "Linux Ubuntu 16.04"): Windows10 pro
[x] A 64-bit installation of Matlab was used.

Step 3: Describe the problem

Hello sir,

I am using irfu-matlab package and trying to run mms examples given in the /plots/mms_examples/Example_MMS_probe_align_times/ Here, I am unable to download the data for "mms?_ancillary_defatt'',''zphase" from mms-sdc and cdaweb. Also and few mms_examples, parameters such as 1] E2d_dsl_edp_fast_l2pre; 2]E_sscedp; 3]mms?_dfg_srvy_ql 4]mms?_fpi_fast_sitl 5]mms?_edp_fast_ql_dce2d

thomas-nilsson-irfu commented 3 years ago

Dear @neetasha,

thank you bringing this to our attention.

As for the ancillary defatt files they are publicly available https://lasp.colorado.edu/mms/sdc/public/data/ (in the sub-folders ancillary/mms[1-4]/defatt/) and should be placed in the corresponding sub-folder structure in your root MMS data folder have created locally for the scripts to works. In this script in particular it is set with the line: https://github.com/irfu/irfu-matlab/blob/3772085427d6de2b1dccf06e2e33b2a26fa5c62a/plots/mms/Example_MMS_probe_align_times.m#L6 (which you should of course adopt to your paths).

As for the other data-products, that are processed only to the datalevel of QL and SITL respectively, and these files are not typically released to the general MMS community which is why you have been unable to locate them at the SDC or NASA servers. These files are lower level data products used by the SITL and is used to produce the quicklook plots at SDC https://lasp.colorado.edu/mms/sdc/public/plots/#/quicklook. The data at this level has generally speaking NOT really been properly calibrated by any of the instrument teams and as the SDC states; Users should use QuickLook data plots for event browsing, and not for scientific analysis or publication without express consent of the PI and relevant Lead Co-Investigators.

I would suggest someone [exactly who is TBD] rewrites all the example plot scripts for MMS to only use data products which are available to the more general public, that it to say L2 or higher.

neetasha commented 3 years ago

Hello Sir, That means I cannot access ql and SITl data sets. Thanks for the information of SITl and ql data sets.

As far as ancillary data is concerned, I am not able to download it i.e. the data set is visible after visiting /data/ancillary/mms?/defatt/ , but there is no download option available. Thus I am not able to download it in my system.

thomas-nilsson-irfu commented 3 years ago

Dear @neetasha,

as for downloading the ancillary data I would typically suggest tools like wget but as you are on Windows I'm not sure if that is available.

One alternative could be to simply use Matlab (the following is not tested but should basically* work, if executed in the folder where it is to download the files to):

% read the URL in question
url = 'https://lasp.colorado.edu/mms/sdc/public/data/ancillary/mms1/defatt/';
webList = websave(url);
% get a list of all file links on this page matching the expected filename
fList = regexp(webList, 'a href="(?<fName>MMS1_DEFATT_\d{7,7}_\d{7,7}.V\d{2,2})', 'names');
% now download each found match in succession 
for iFile=1:length(fList)
  outFilname = websave(fList(iFile).fName, [url, fList(iFile).fName]);
  display(['Saved: ', outFilname]);
end

*I write "basically work" but sometimes the SDC have problems listing this directory as it contains a lot of files (2205 at time of writing). If SDC experience such problems it in most cases simply return an error page, which does not have any of the links that the regexp looks for.

Note: There is a lot of files in this folder so you may want to refine the script to download only the files you are interested in.

neetasha commented 3 years ago

Hello Sir, I am using WHAMP dispersion solver for plotting dispersion characteristics of Lower Hybrid Wave (LHW). I have referred Ronmark,1982 report on WHAMP and vaivads,2015 paper and I am stuck with few questions.

1] Does WHAMP plots dispersion characteristics of electrostatic waves such as LHW? 2] what should be the possible start frequency and wave vector values to give as an input? 3] The distribution function can take 10 species and the normalization is always with respect to the first component of the plasma model. Does the output (frequency, perpendicular and parallel wave vector, parallel and perpendicular growth rate, etc) are already normalized values/matrix?

Please help me out with this issue. Regards, Neetasha Arya.

irfu / irfu-matlab