Closed maddyscientist closed 10 months ago
I've reviewed every file not called interface_quda.cpp
and I'm a big fan of all of the cleanup. I'll need to go through interface_quda.cpp
with a fine-tipped comb before I feel good about it!
Why was timing QUDA_PROFILE_INIT
removed in a handful of interface functions? (I know I mentioned this in one review comment, but it seems to appear in multiple places.)
I've completed my visual review of interface_quda.cpp
and left a handful of comments/questions, most of which probably only beg clarification. I'll start doing some explicit tests soon.
@weinbe2 Ok, all your comments should be addressed now.
This has now been tested with MILC RHMC, and things are looking good.
Tentative approve, I noticed a few cosmetic things, and this needs a fresh merge of develop
+ clang-format
. After that it's good to go!
@SaltyChiang I have merged in your changes into this branch, that will shortly be merged into develop.
One line I have not included is this one:
gauge[d] = (param.gauge) ? ((void **)param.gauge)[d] : nullptr;
as this line doesn't make sense to me. If it's a reference to a pre-existing field, why should the value be a nullptr
? Can you explain why this change is needed?
Thanks
Hi @maddyscientist, there are some if
statements about the input gauge link, like https://github.com/lattice/quda/blob/6e90b49a2d12891f2f68e35b7cfa7defbd506526/lib/interface_quda.cpp#L3873 and https://github.com/lattice/quda/blob/6e90b49a2d12891f2f68e35b7cfa7defbd506526/lib/interface_quda.cpp#L3903. I believe the logic here is originally used for QUDA_MILC_GAUGE_ORDER, but this will fail if QUDA_QDP_GAUGE_ORDER is used here. For example, if the ulink
is nullptr
here, the unitarized fat7 link will not be calculated, which is right; but cpuUnitarizedLink
defined in https://github.com/lattice/quda/blob/6e90b49a2d12891f2f68e35b7cfa7defbd506526/lib/interface_quda.cpp#L3847 fails now as you cannot get ((void **)nullptr)[d]
. A null QUDA_QDP_GAUGE_ORDER
field should be something like void *ulink[4] = {nullptr, nullptr, nullptr, nullptr}
but now the if (ulink)
will return true which we do not expect.
So I made all four reference fields nullptr
when the input value itself is nullptr
and I think this will not affect the existing logic.
@SaltyChiang thanks for the explanation. I have fixed my branch accordingly (https://github.com/lattice/quda/pull/1384/commits/4c308f6f72a2239e97f3423137459e049dc56b03), so it should work for your usecase. If you could test this branch, that would be appreciated, thanks.
Hi @maddyscientist, very sorry for the late feedback. I think it should be !unitarizedLink.StaggeredPhaseApplied() && param->staggered_phase_applied
here. https://github.com/lattice/quda/blob/0c016af709ad6a3c249540fdad809c8d18abc226/lib/interface_quda.cpp#L3748-L3749
Edit: I believe gauge_param.location
should be changed to QUDA_CUDA_FIELD_LOCATION
here.
https://github.com/lattice/quda/blob/0c016af709ad6a3c249540fdad809c8d18abc226/lib/interface_quda.cpp#L765-L774
@SaltyChiang thanks for pointing put my bug. I've pushed a fix to my next PR that will be merged (#1393). This is a fairly simple PR, so will likely be merged soon.
@maddyscientist thanks for the fix. You might miss another issue that will happen while saving the smeared gauge through saveGaugeQuda(). I updated the comment above and I repeat it here.
Edit: I believe
gauge_param.location
should be changed toQUDA_CUDA_FIELD_LOCATION
here.
Thanks again @SaltyChiang. Fix is here 44c41006158e493517452210a7a1d1e55412d63f (in #1393).
This PR's primary purpose is to unify the
cpuGaugeField
andcudaGaugeField
classes, along with an assortment of other cleanups.cpuGaugeField
andcudaGaugeField
classes, we have now onlyGaugeField
ColorSpinorField
the communiciation routines for host and device are still separate within theGaugeField
class, unification of these is deferred until laterGaugeField
to facilitate significant cleanupdata()
method. This is to align with STL design. Though here we also have an optional template parameter that can be used to perform a cast to the desired type. (Also applied toColorSpinorField
andCloverField
)quda_ptr
, which provides heterogenous memory allocationGaugeField
,ColorSpinorField
andCloverField
qudaMemcpy
andqudaMemset
functions that accept aquda_ptr
pushProfile
andpopProfile
functions, together withgetProfile
for getting a reference to the current top of stack profile.Outstanding items
quda_ptr
for ROCm