Allow for odd-sized CPU grids regardless of calling application

[maddyscientist]

I believe MILC supports this, but Chroma does not (at least the interface doesn't). We need to support this in Chroma as well. Guochun, what do you do for MILC that allows for this: is it just a case of swapping even and odd gauge fields on the odd CPU nodes, or is there something more to do?

I think that any MILC specifics that allow for this functionality should be put into interface_quda.cpp so that any calling package supports this automatically.

[rbabich]

Agreed. The even/odd swapping should ideally happen in QUDA, rather than in the application.

[gshi]

Swapping even/odd gauge field and passing in correct parity do the trick. There is nothing else.

Where or not we need to switch even/odd is determined by the even-odd-ness of the dimension size and the node coordinates in the node grid

---

even_odd_change=0; for(i=0;i < 4;i++){ if(X[i] % 2 == 1 && get_logical_coordinate()[i] % 2 ==1 ){ even_odd_change = 1 - even_odd_change; } }

---

If the even_odd_change is 1, then the fatlink/longlink's even odd fields are switched. The parity passed to the QUDA is determined base on the even_odd_change and the spinor's parity in MILC

---

QudaParity local_parity; if (even_odd_change){ local_parity = (parity == EVEN)? QUDA_ODD_PARITY: QUDA_EVEN_PARITY; }else{ local_parity= (parity == EVEN)? QUDA_EVEN_PARITY: QUDA_ODD_PARITY; }

if(local_parity ==  QUDA_EVEN_PARITY){
  inv_param.matpc_type = QUDA_MATPC_EVEN_EVEN;      
}else{
  inv_param.matpc_type = QUDA_MATPC_ODD_ODD;      
} 

---

There is no switch needed in spinor exchange function because we always work on parity spinor (is it true for Wilson?). QUDA deals with local parity while the MILC assumes global parity, thus the conversion. Right now, MILC+ QUDA works correctly with one local dimension (Y,Z, or T) size being odd.

The MILC (7.6.3) interface code can be found in (search for "even_odd_change") http://kfs3.ncsa.uiuc.edu/projects/ncsa_projects/milc_qcd-7.6.3_multi_dim_gpu/generic_ks/ks_multicg_offset_gpu.c

[maddyscientist]

These modifications look simple, and can surely be brought within the interface I think. If we make this change, then it will require that we update MILC at the same time (if both library and application apply this transformation, then we end up back where we started).