Vivek's Requests

[stuwilkins]

@stuwilkins This works great for the most part so far. One issue that needs to be addressed is dealing with scans where the number of images per event are different. This can arise either if a scan is aborted, or if you want to process two different scans together which have different number of images per event. The second issue can be addressed by processing the scans separately and concatenating the image stacks. But that won't work in the first scenario.

At the moment, I deal with this by returning a list of 3D numpy arrays (one for each event), instead of a 4D numpy array. These are converted to a 3D stack using this function:

def convert_to_3d(images): if isinstance(images, list): ims = images[0] for im in images[1:]: ims = np.vstack((ims, im)) return ims images = np.asarray(images) if images.ndim < 4: return images else: shape = images.shape return np.reshape(images, (shape[0]*shape[1], shape[2], shape[3])) There might be a more efficient way to do this. But something like this is required to pass a 3D stack of images to a function which calculates G2 for instance.

Another useful function that you might want to add is specifying an ROI for the images. This can actually be implemented in the get_fastccd_images function to only process the specified ROI instead of the whole image and can allow processing much bigger stacks of images. For instance, I use the function below to truncate the images before passing it to the correct_images function (the "rotated" option is to account for the ROI being specified in the proper image orientation and not the rotated raw data format.).

def get_roi(images, roi=[0,0,960,960], default_roi=[0,0,960,960], rotated=False): if roi == default_roi: return images

if rotated:
    roi = [960-roi[3], roi[0], 960-roi[1], roi[2]]

if images.ndim == 3:
    images = images[:,roi[0]:roi[2], roi[1]:roi[3]]
else:
    images = images[roi[0]:roi[2], roi[1]:roi[3]]

return images

Also, it will be good to have a few simple functions like ccdtotal_per_event, ccdmean_per_event, stackmean_per_event and stacksum_per_event for quick plotting of motor scans etc.

[stuwilkins]

@vivekthampy @tacaswell Moving off wishlist not to spam everyone.

[stuwilkins]

@vivekthampy Your request is handled in csxtools now. It does not return a 4d array, but an interable over all events which has a pims object for each event. This means that you CAN do a different number of points per event ok

The ROI issue can be handled both by streaming (which is now implemented thanks to @tacaswell and @danielballan) and (@sameera2004 has done a really nice job on the multi-tau code) or by taking an ROI which can be done over the stack as indicated above.

Why don't you put in a pull request to the things that you think should be there - then we can discuss and review.

[vivekthampy]

I get this error with the current CSX (stable) kernel:

https://gist.github.com/vivekthampy/5a23f53efbf08a419ec1

[danielballan]

I believe you may be using the wrong value for the tag argument. Is it now simply 'fccd_image'? Also, you'll be glad to know that that error message has been made less cryptic in a soon-to-be-released update to databroker. It means that no data was found and the array has size 0.

[vivekthampy]

This is for old LBCO data, where the tag argument was 'fccd_image_lighfield'. I also tried with the 'fccd_image' tag, but that doesn't work either (this time with databroker crashing).

The problem here is, as far as I can tell, stacking multiple 3D arrays into a 4D array if the 3 dimensions are not identical.

See attached notebook example for both these problems:

https://gist.github.com/vivekthampy/d49b48086f6087ffbb08

[stuwilkins]

@vivekthampy which versions are you running... not able to reproduce here.

[stuwilkins]

Also @vivekthampy is is fccd_image_lightfield

[stuwilkins]

No problem here:

https://nbviewer.jupyter.org/gist/stuwilkins/1a1651eb75e5dd24aa61

[tacaswell]

You can always look at head.descriptors to see what the data keys are.

On Thu, Feb 25, 2016 at 3:37 PM Stuart Wilkins notifications@github.com wrote:

No problem here:

https://nbviewer.jupyter.org/gist/stuwilkins/1a1651eb75e5dd24aa61

— Reply to this email directly or view it on GitHub https://github.com/NSLS-II-CSX/csxtools/issues/32#issuecomment-188972201 .

[vivekthampy]

That was a typo. I did mean 'fccd_image_lightfield'.

It must be a version error. I am using the csxtools version in the default CSX (stable) kernel. What version are you using, @stuwilkins ?

[stuwilkins]

Let me know what version you are running and I will compare. I have 0.1.4. I thought you were running a custom kernel anyway?

[vivekthampy]

Version '0.1.0'

This is the version tagged in the CSX stable kernel, I suppose. I do have a custom kernel which I normally use. But I am trying to test the standard kernel as well, to have easily sharable notebooks.

[stuwilkins]

@vivekthampy let's make sure that it works with the latest tag (stable release) on master and then we can work on getting the correct version on Jupyterhub.

[stuwilkins]

@vivekthampy I just tagged version 0.1.5. Are you putting any pull requests in to expand the code? Attn @mpmdean

[vivekthampy]

Yes. I'll put in a pull request later today.

[stuwilkins]

Still waiting .... @vivekthampy (@mpmdean)

[vivekthampy]

I'm testing a few things out. Will commit and make pull request by end of day @stuwilkins