search

paulscherrerinstitute / pyzebra

GNU General Public License v3.0
3 stars 2 forks source link

Trouble accessing hdf files under certain proposal numbers #30

Closed cblarsen1 closed 3 years ago

cblarsen1 commented 3 years ago

I'm having trouble loading hdf files from some experiments into pyzebra.

When I type in proposal number 20202268, pyzebra looks in a path ending with 2020/zebra/20202268/, but the files are actually in 2021/zebra/20202268/. I guess the issue is that pyzebra assumes that the first four numbers of the proposal id always corresponds to the year where the experiment was carried out?

Last part of error output: FileNotFoundError: [Errno 2] No such file or directory: '/afs/psi.ch/project/sinqdata/2020/zebra/20202268'

It is however still possible to load them using .cami files

Edit: Have similar issue when trying to run spind tab. I put in prefix: "/afs/psi.ch/project/sinqdata/2021/zebra/20202268/zebra2021n00"

and get error:

FileNotFoundError: [Errno 2] No such file or directory: '/afs/psi.ch/project/sinqdata/2020/zebra/20202268'

so also in this case it seems to look for files under year 2020

ivan-usov commented 3 years ago

Hi @cblarsen1 , yes, this is exactly correct how you explained it. The first 4 digits of the proposal are used as a year in a search path. In fact, we had a similar issue with @zaharko for one of the previous experiment, and as a understood, the quick solution was to move the files to a folder with the "right" year in the path, or a proposal number was changed, so it starts with 2021.

But we can change the way pyzebra looks for files. If it's not a good assumption to expect 4 first digits of a proposal to correspond to a year in the path, we can either include all paths to the search, or explicitly list all folders where to search. Either way, your input on a proposal number lifecycle (e.g. how they are created, and to which path the data is then copied to afs) would be really helpful.

cc @zaharko

zaharko commented 3 years ago

Dear Ivan,

Thank you for these suggestions. I think the solution: ' we can either include all paths to the search’ would be optimal. It will be great, if you can implement this as fast as possible and let us know that it works.

Also another ‘hot' problem is that we cannot get projections for the files written in directory (proposal ID) /afs/psi.ch/project/sinqdata/2021/zebra/20211324http://psi.ch/project/sinqdata/2021/zebra/20211324 We can however leaf frame by frame. I send you the corresponding cami file

Best wishes Oksana

On 20 Jun 2021, at 18:25, Ivan Usov @.**@.>> wrote:

we can either include all paths to the search