search

usnistgov / hiperc

High Performance Computing Strategies for Boundary Value Problems
https://pages.nist.gov/hiperc/en/latest/index.html
41 stars 8 forks source link

hierarchical structure of examples #76

Closed tkphd closed 7 years ago

tkphd commented 7 years ago

Which directory tree is more sensible?

  1. Physics at the top level, hardware at the bottom: 
    phasefield-accelerator-benchmarks
+-diffusion
| +-benchmarks
| +-cpu
| | +-serial
| | +-openmp
| | +-tbb
| +-gpu
| | +-cuda
| | +-openacc
| +-phi
+-spinodal
| +-benchmarks
| +-cpu
| | +-serial
| | +-openmp
| | +-tbb
| +-gpu
| | +-cuda
| | +-openacc
| +-phi
+-ripening
| +-benchmarks
| +-cpu
| | +-serial
| | +-openmp
| | +-tbb
| +-gpu
| | +-cuda
| | +-openacc
| +-phi
  2. Hardware at the top level, physics at the bottom: 
    phasefield-accelerator-benchmarks
+-benchmarks
| +-diffusion
| +-spinodal
| +-ripening
+-cpu
| +-serial
| | +-diffusion
| | +-spinodal
| | +-ripening
| +-openmp
| | +-diffusion
| | +-spinodal
| | +-ripening
| +-tbb
| | +-diffusion
| | +-spinodal
| | +-ripening
+-gpu
| +-cuda
| | +-diffusion
| | +-spinodal
| | +-ripening
| +-openacc
| | +-diffusion
| | +-spinodal
| | +-ripening
+-phi
| +-diffusion
| +-spinodal
| +-ripening
  3. Something else?

The goal is to create benchmarks of hardware performance on physical problems, so my inclination is toward choice 1, with graphs in the physical directories. However, plots can be generated anywhere and reported in the documentation, so that's not a strong justification.

@amjokisaari, @guyer, @reid-a, @wd15: I'd be interested in your opinions.

wd15 commented 7 years ago

Use a flat structure with a README or YAML file in each directory which describes its content.

tkphd commented 7 years ago

@wd15 something like this?

phasefield-accelerator-benchmarks
+-benchmarks
+-diffusion_cpu_serial
+-diffusion_cpu_openmp
+-diffusion_gpu_cuda
+-diffusion_phi
+-spinodal_cpu_serial
+-spinodal_cpu_openmp
+-spinodal_gpu_cuda
+-spinodal_phi
amjokisaari commented 7 years ago

leaning toward option 1. Fits your goal the best.

wd15 commented 7 years ago

On Mon, Aug 28, 2017 at 3:20 PM, Trevor Keller notifications@github.com wrote:

@wd15 something like this?

phasefield-accelerator-benchmarks +-benchmarks +-diffusion_cpu_serial +-diffusion_cpu_openmp +-diffusion_gpu_cuda +-diffusion_phi +-spinodal_cpu_serial +-spinodal_cpu_openmp +-spinodal_gpu_cuda +-spinodal_phi

Yes, that seems like a good idea and then you don't need to worry about the structure. Put a README or YAML in each directory with a description of what's going on. Of course, your ideas will change so the structure will need to change and it's easier to migrate a flat structure.

-- Daniel Wheeler

tkphd commented 7 years ago

The thing I don't like about the flat directory structure is that it would force greater duplication of code. The bottom tier of directories implement the subset of functions specific to a threading paradigm, with functions common to the hardware type implemented in the mid-tier directories. For example, the 2D data arrays are implemented in cpu and gpu, while timestepping is implemented in cpu/serial, cpu/openmp, gpu/cuda, etc. If I implement the flat directory structure, it would be harder to share these implementations.

The benefit of the flat structure is, with each directory a self-contained bundle of code, copying and transforming for individual projects would be easier for users. That's worth considering.

reid-a commented 7 years ago

A slightly old-school solution to having one's cake and eating it too, or in this case, having one's easy-to-navigate flat structure and avoiding code duplication, is to use symbolic links to refer to the common elements of the code from the various directories where they need to be seen.

I don't know if git is sym-link-aware enough to not break this sort of thing.

Can the duplicate code be abstracted into libraries? 

                -- A.

On Tue, Aug 29, 2017 at 05:18:24PM +0000, Trevor Keller wrote:

The thing I don't like about the flat directory structure is that it would force greater duplication of code. The bottom tier of directories implement the subset of functions specific to a threading paradigm, with functions common to the hardware type implemented in the mid-tier directories. For example, the 2D data arrays are implemented in cpu and gpu, while timestepping is implemented in cpu/serial, cpu/openmp, gpu/cuda, etc. If I implement the flat directory structure, it would be harder to share these implementations.

The benefit of the flat structure is, with each directory a self-contained bundle of code, copying and transforming for individual projects would be easier for users. That's worth considering.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.*

-- Dr. Andrew C. E. Reid Physical Scientist, Computer Operations Administrator Center for Theoretical and Computational Materials Science National Institute of Standards and Technology, Mail Stop 8555 Gaithersburg MD 20899 USA andrew.reid@nist.gov

amjokisaari commented 7 years ago

The benefit of the flat structure is, with each directory a self-contained bundle of code, copying and transforming for individual projects would be easier for users. That's worth considering.

This is true. I know it was a little frustrating for me to have to dig through the existing file structure to find the pieces of code I wanted to look at.

tkphd commented 7 years ago

Oh, duh. I can use a flat file structure, and just place the shared stuff into clearly labeled folders, e.g.

phasefield-accelerator-benchmarks
+-benchmarks
+-diffusion_cpu_common
+-diffusion_cpu_serial
+-diffusion_gpu_common
+-diffusion_gpu_cuda

Then, to repurpose the code, a user can simply merge the relevant common and hardware-specific folders into a new location. Problem solved.

Thanks @amjokisaari, @reid-a, and @wd15 for your input!

amjokisaari commented 7 years ago

Don't forget to document that ;)

On Aug 29, 2017 2:12 PM, "Trevor Keller" notifications@github.com wrote:

Oh, duh. I can use a flat file structure, and just place the shared stuff into clearly labeled folders, e.g.

phasefield-accelerator-benchmarks +-benchmarks +-diffusion_cpu_common +-diffusion_cpu_serial +-diffusion_gpu_common +-diffusion_gpu_cuda

Then, to repurpose the code, a user can simply merge the relevant common and hardware-specific folders into a new location. Problem solved.

Thanks @amjokisaari https://github.com/amjokisaari, @reid-a https://github.com/reid-a, and @wd15 https://github.com/wd15 for your input!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/usnistgov/phasefield-accelerator-benchmarks/issues/76#issuecomment-325766079, or mute the thread https://github.com/notifications/unsubscribe-auth/AGXflEk3Sf57GXZqdcAm3IBGijD8zSX4ks5sdGKVgaJpZM4PE4hU .