search

LiberTEM / LiberTEM

Open pixelated STEM framework
https://libertem.github.io/LiberTEM/
GNU General Public License v3.0
109 stars 68 forks source link

File dialog: support for file stacks #498

Open sk1p opened 4 years ago

sk1p commented 4 years ago

This could be, in the easiest version, opening files via entering a glob (should present a file list as second step, when entering the other parameters). Could also consist of having a selection interface in the file dialog.

Also needs "natural" sorting, for example via the natsort package, as many data sets sadly don't use leading zeros in the filename.

Iamshankhadeep commented 4 years ago

Can you tell me more about this issue? Like what should I need to do and what is glob?

sk1p commented 4 years ago

what is glob?

https://en.wikipedia.org/wiki/Glob_(programming) https://docs.python.org/3.7/library/glob.html

Can you tell me more about this issue?

Opening a "stack of files" refers to having a dataset that consists not of a single file, but of many smaller files. These are then "stacked" together to create a dataset of more dimensions than a single file. For example, you could have 256 files, each containing a single 2D image of size (128, 128). You could then stack them to create a single 3D dataset of shape (256, 128, 128), or a 4D dataset of (16, 16, 128, 128). This is often used together with the DM file format, but could later also be useful for other formats, for example raw, tif or edf.

Now, if you have a look at the current file browser in the GUI, only a single file can be selected, which works for most formats. The task is to improve the file browsing interface and add the possibility to select multiple files. This includes (not necessarily in this order):

1) Thinking about a user-friendly way to select multiple files (how should the UI look like and behave?) 2) Implementing the results of step 1 in the file browser 3) Adding a parameter form for the DM format 4) Changing the web API backend to support not just a single path, but multiple files and/or glob patterns (depending on what the result of step 1 was)

Now to get started, I would suggest to keep it very simple and think about the MVP, the minimum viable product. How can we support opening multiple files in the most simple way? Even writing only a prototype implementation would be a good preparation for your GSoC project.

For example data in DM format and some additional discussion, see also https://github.com/LiberTEM/LiberTEM/issues/291

Iamshankhadeep commented 4 years ago

But what if we select multiple file of different types, then what happens?

Adding a parameter form for the DM format

I think I did not get it. Can you provide me with more details?

sk1p commented 4 years ago

But what if we select multiple file of different types, then what happens?

Good question! In my opinion, the best thing to do would be to display an error, as it is the least surprising behavior.

Adding a parameter form for the DM format

I think I did not get it. Can you provide me with more details?

Please have a detailed look at the "workflow" of opening files: observe each single step you have to take. After selecting a file in the file browser, you will get a parameter form. This does not exist for the DM format yet, and needs to be added.

If you want to know how the existing parameter forms are implemented, you can use the react devtools and inspect the GUI when a parameter form is open to see which components are used, and then have a look at the corresponding source files.

Iamshankhadeep commented 4 years ago

I think I have an idea about how to tackle this issue.

Thanks for additional information.