file directory and file naming

[bgchristensen]

We should think a least a little bit how we want the directory system to look. For example, I've been frequently taking overnight measurements. It's rather inconvenient to have the files saved in folders linked to the date in this case, since I then have to rename files and move them into the same folder to keep all of the data grouped. However, during normal bringup stuff, it's very nice having the date of the folder, because then I know which day certain scans were taken.

Also, I'd like to have control over the file name for saving, since sometimes I'd like to add a note in the file name to distinguish two different measurements in the same folder. For example, Ivan's current saving would name files something like Calibration_000 and Calibration_001, but I would rather have control and name them Calibration_I, and Calibration_X.

[amopremcak]

I agree @bgchristensen. The way that dates are handled right now confuses me. The folder structure is: Data/ProjectName/System+StartDate/User/SampleDescription/TimeOfScan/ScanDescription/datasets

It seems that Brad is in favor of eliminating the TimeOfScan folder in this path and just using the StartDate to determine what datasets end up where. I don't understand why we group System and StartDate into one concatenated string. @patzinak , we should all talk about why it was done this way as I am sure you had your reasons.

[patzinak]

First of all, the current system is slightly different. The measurement code forces only the /User/Sample/Day/Experiment/datasets part of the path. (We use Date, not TimeOfScan, and ExperimentName, not ScanDecription — different scans could be saved under the same name.) Everything above the User level is totally arbitrary. I use Data/Project/System - CooldownDate - CooldownDescription/.

I do not know the reasons behind the current directory naming. It is what it is for the historical reasons and since I hadn't had a better idea I just preserved what some other people used before. Keeping the things the same way is better than changing them to something equally bad or good. In other words, I asked @bgchristensen to open this thread so that people can share their opinion. Once we have settled on something, it can be easily implemented. I do not have a strong opinion here. My suggestion is that it would be really useful to have one directory level to provide the cooldown description with the date (or only cooldown date but no other dates in the path), another level should include the sample label and there should be a level for the experiment name.

[patzinak]

System - CooldownDate - CooldownDescription is one string because it works very well for me. It is not enforced by the code. The reason I'd like to keep CooldownDate and CooldownDescription is because it extremely convenient when you are looking through datasets generated within one a half year on the same project. You kind of have some general ideas of the date when the data was taken and some general ideas on what was the goal of that set of measurements. In other words, suppose everything starts with the CooldownDate. After a few cooldowns you will end up with a directory filled with dates and you have to click deeper to see what is inside. If you name everything by purpose of the cooldown you may end up having folders like "Resonator Test" or "Testing Resonators" and both of them will contain the comparable sets of data.

Let me start with a suggestion, so that everybody can share their opinion on this and/or improve it:

• Data
• \Project
• \CooldownDate - CooldownDescription (measurement purpose or general description of the enclosed datasets, like "Date - Resonator Tests" or "Date - Inverted Layout Resonators", or "Date - Mock-up Device Dip Test")
• \System (Fridge, Dip Station, etc.)
• \Sample (ABC294769-UJ-25)
• \Experiment
• \datasets

[roboguy222]

It seems like I am alone in this, but I would rather have the system as a parameter of the data file than part of the path. It is rare that we cool a sample and measure it in multiple systems in one day, so it is just an extra click. Otherwise this path structure sounds good to me.

[patzinak]

@roboguy222 you have a good point. That is why I currently has the system information as a part of System - CooldownDate - CooldownDescription. The system information was helpful when we were measuring JPMs from the same wafer in ADR3 and in Leiden simultaneously. I'm fine with eliminating the system information. Just want to provide the context why the things are the way they are now.

[amopremcak]

I think system information is important in the file path. In some sense, the system name determines the type of experiments that you would be doing on a particular device for a particular experiment. When you move from ADR to the Leiden, you aren't going to want to sort through a shit load of dates to determine your first leiden cooldown, @bgchristensen how do you feel about system name in the path to experimental data?

[amopremcak]

I like the folder structure suggested by @patzinak by the way. One thing you may want to add however is the User.

[bgchristensen]

I like Ivan's suggestion, but having System is strange having before the Sample. The only reason I would care which system I cooled something down with, is because it is the same sample. If it were different samples, I wouldn't care about data from a dip vs adr vs dr. So I suggest if we keep system, then put it after sample.

Although, I think it is pretty clear from Cooldown Description what system is used anyway, so not sure this information is necessary? I'm fine with it in there, but should be after sample.

[roboguy222]

Haha I was going to say the opposite. I would say if we have system, we should put it before the date.