stfc / PSyclone

Domain-specific compiler and code transformation system for Finite Difference/Volume/Element Earth-system models in Fortran
BSD 3-Clause "New" or "Revised" License
103 stars 25 forks source link

No Halo exchange when reading from W3 field #38

Closed christophermaynard closed 7 years ago

christophermaynard commented 7 years ago

PSyclone is unnecessarily generating halo exchanges when reading from a W3 (discontinuous field) as shown in the example below from advection_alg_mod.x90 First halo exchange on mf field is correct. The writes to W3 field r_rho and loops to last edge, not last halo cell. The calls set_dirty() on W3 field, which is correct because the field is dirty then into the second loop which reads and writes to a W3 field, but no halo exchange is required as the loop is only to last edge because we are reading from W3 field.

  IF (mf_proxy%is_dirty(depth=1)) THEN                                      
    CALL mf_proxy%halo_exchange(depth=1)                                    
  END IF                                                                    
  !                                                                         
  DO cell=1,mesh%get_last_edge_cell()                                       
    !                                                                       
    CALL dg_matrix_vector_code(cell, nlayers, r_rho_proxy%data, mf_proxy%data,  &
        div_proxy%ncell_3d, div_proxy%local_stencil, &                              
       &ndf_w3, undf_w3, map_w3(:,cell), ndf_any_space_1_mf, undf_any_space_1_mf, 
        map_any_space_1_mf(:,cell))                                                         
  END DO                                                                    
  !                                                                         
  ! Set halos dirty for fields modified in the above loop                   
  !                                                                         
  CALL r_rho_proxy%set_dirty()                                              
  !                                                                         
  IF (r_rho_proxy%is_dirty(depth=1)) THEN                                   
    CALL r_rho_proxy%halo_exchange(depth=1)                                 
  END IF                                                                    
  !                                                                         
  DO cell=1,mesh%get_last_edge_cell()                                       
    !                                                                       
    CALL dg_matrix_vector_code(cell, nlayers, rrho_prediction_proxy%data, r_rho_proxy%data, &
           mm_w3_inv_proxy%ncell_3d, &                                    
           &mm_w3_inv_proxy%local_stencil, ndf_w3, undf_w3, map_w3(:,cell), &
            ndf_any_space_1_r_rho, undf_any_space_1_r_rho, &                                              
            &map_any_space_1_r_rho(:,cell))                                                 
  END DO                                                                    
  !                                                                         
  ! Set halos dirty for fields modified in the above loop                   
  !                                                                         
  CALL rrho_prediction_proxy%set_dirty()                                    
  !                                          
rupertford commented 7 years ago

Yes I confirm that this is a bug in PSyclone.

The discontinuous function was returning false no matter what input it received.

This has been fixed as part of the redundant computation transformation issue #50. Therefore I am closing this issue.

Note, we might be able to remove all halo exchange calls when iterating over owned dofs i.e .for continuous fields as well as discontinuous (as long as there is no stencil access of course). I am currently checking whether that is the case with Mike Hobson and @christophermaynard.