search

hqucms / weaver-core

Streamlined neural network training.
MIT License
44 stars 54 forks source link

Problem in _clip #2

Closed dietrichliko closed 1 year ago

dietrichliko commented 1 year ago

Hi!

I am using waever-core on running a sample of flat ntuples (no point clouds), so my data are numbers not vectors.

weaver is crashing, when using automatic scaling. The reason is that the values are numpy arrays dressed up as awkward arrays. In subroutine _clip in utis/data/tools.py is calling the method flatten on the awkward array, which does not work for 1-dim arrays and raises an exception.

One could argue that flatten should be implemented as noop on 1 dim arrays ... but that's not the case ...

I am suggesting to add a.ndim == 1 to the if statement in _clip, as flatten is not required.

def _clip(a, a_min, a_max): if isinstance(a, np.ndarray) or a.ndim == 1: <====== return np.clip(a, a_min, a_max) else: return ak.unflatten(np.clip(ak.to_numpy(ak.flatten(a)), a_min, a_max), ak.num(a))

In case you prefer, I could also send you a PR.

Cheers, Dietrich

hqucms commented 1 year ago

Hi @dietrichliko -- thanks a lot for reporting this! The proposal makes sense to me. Before opening a pull request, can you also share an example of your data config yaml to illustrate when it crashes?

hqucms commented 1 year ago

It's fixed in https://github.com/hqucms/weaver-core/commit/6a74ce6b86d98015601fcd2ad53a7710588fd252 now, and a new release v0.4.2 is made.

dietrichliko commented 1 year ago

Hi @hqucms ! Thx for the fast fix.

I am just feeding sub jet variables into a standard network for comparison with ParticleNet ... There is not lot one can learn. If you think its useful for you, I still can send you the confit file and data, but I do not see any need for that.

Assuming its fine for you, I am closing the issue.