search

NOAA-OWP / t-route

Tree based hydrologic and hydraulic routing
Other
44 stars 50 forks source link

single numpy array with 2 dimensions per segment. #109

Closed jhrehanoaa closed 4 years ago

jhrehanoaa commented 4 years ago

An alternative idea would be to use a single numpy array with 2 dimensions per segement. This would have the added benefit that since the array is contiguous, you could update all the values in a timestep in a single operation. I would suggest a (nts, 7) shaped array. Then your update operation is simply

flowveldepth[current_segment][ts] = [ts*dt, qlatCum, qlat, qdc, velc, depthc, volumec]

_Originally posted by @groutr in https://github.com/NOAA-OWP/t-route/pull/107#discussion_r479458485_

jhrehanoaa commented 4 years ago

jameshalgren 3 days ago Member

Is there any benefit in access speed to having it be (nts, 7) vs. (7, nts)? (i.e., having the timesteps as the column dimension?) I can see that the single-line assignment would be more complicated...

@groutr groutr 3 days ago Contributor

It boils down to C-contiguous vs Fortran-contiguous. Numpy arrays are by default C-contiguous so a shape of (nts, 7) means that the values for each time step are contiguous. If you have a (7, nts) array, unless the array is Fortran-contiguous, the values for the timestep are strided across the array. Performance wise, it would be very difficult to pin down the effect because of so much else going on, but it can make a difference.

Here are some timings:

%timeit C[500]  # shape=(1440, 7) C-contig. The resulting view is contiguous.
179ns
%timeit C1[:, 500] #shape (7, 1440) C-contig.  The resulting view is noncontiguous.
381ns
%timeit F[:, 500] # shape (7, 1440) F-contig. Resulting view is contiguous
277ns
%timeit F1[500] #shape (1440, 7), F-contig. Resulting view is noncontiguous.
197ns
jhrehanoaa commented 4 years ago

columns 0-6 are time, qlatCumval, qlatval, flowval, velval, depthval, storageval respectively. The row corresponds to the timestep