Open Calpas opened 5 years ago
takluyver
commented
5 years ago The ID_83 sounds like you're using data on beegfs. We don't maintain that data or indexamajig, so you'll probably have to talk to people at CFEL if you think something has changed.
Have you tried inspecting the data in the cxi file, e.g. using hdfsee? Or even looking at the raw numbers in hdfview? This would help narrow it down whether it's a problem with the data or with indexamajig.
Calpas
commented
5 years ago The problem is that a week a go it was running fine several times, and now it does not. I'm running the same command on the same data. The data are located here (1). I run the script and it says that the data are available. So I do not see what could have change from my side.
(1) /beegfs/desy/group/it/ReferenceData/cxidb/ID-83/proc/r0250
takluyver
commented
5 years ago I don't know what could have changed. But both the /beegfs/desy folder and indexamajig are managed by people at DESY, not us. The only bit you're using our tools for is generating the cxi file. If you think something's wrong with that step, please point us to the file so we can investigate.
Calpas
commented
5 years ago Dear experts,
takluyver
commented
5 years ago In (1), I suspect it's Q2M4 that isn't working - the data there looks like noise. hdfsee scales the colours depending on the highest pixel value, so high values in one module can make the rest of the image invisible. There's a way to 'boost' the intensity in hdfsee so you can see the data in the rest of the image.
For (2), I don't know. I'm not a crystallographer. The images that have been pointed out to me as good data had fewer peaks than that, especially around the edges. But I haven't seen that many, and I find it hard to spot a real pattern until someone like Tom White points it out.
It might be a good idea to ask someone who's worked a lot with this kind of analysis if they could spare an hour or two to discuss it with you in person. Some things are easier to grasp that way. If he's got time, Tom White would be a good choice - you're using tools he's written, and he gave us some really useful tutorial sessions a couple of months ago. But if not, there are other people both at DESY and XFEL who know about crystallography.
Calpas
commented
5 years ago Dear Takluyver, is there a plot which show the cumulative among of data register by the SFX/SPB experiment? Regards
takluyver
commented
5 years ago I'm not aware of any such plot.
If it helps, the data from one proposal may be on the order of 100 TB, though this could increase as data can be collected more continuously. Of course, proposals can also record much less data, depending on what they're doing.
Calpas
commented
5 years ago Dear Takluyver, I saw that the crystfel indexing had been processed on the file ID-83 (*) which is great! But I can't access the cxi data in this file. Can I have right? Regards
(*) https://www.cxidb.org/data/83/blac_new_v0_nomulti.stream
takluyver
commented
5 years ago I'm not exactly clear what you're trying. But the analysis shared on CXIDB uses Cheetah as a hitfinder producing CXI files for CrystFEL. The CXI files produced by Cheetah are not quite the same layout as the 'virtual' CXI files produced by our tool. So things like the geometry files can't be directly shared between the two analyses.
Dear experts, since yesterday when I run "indexamajig" (1) to find peaks, crysfel can't find any of them (2). Did something change recently? A few days ago I was able to find peaks in this dataset: run_250 and ID_83. The file.lst has a file.cxi and I checked that it was accessible. Regards
(1) indexamajig -i file.lst -g agipd_mar18_v11.geom --peaks=peakfinder8 --threshold=300 --min-gradient=500000 --min-snr=5 --int-radius=3,4,5 --indexing=mosflm -o log
(2) ...
0 indexable out of 6087 processed ( 0.0%), 0 crystals so far. 23 images processed since the last message. 0 indexable out of 6097 processed ( 0.0%), 0 crystals so far. 10 images processed since the last message. ...