Migrate rocketchip utils to standalone library

[lordspacehog]

Related issue: https://github.com/chipsalliance/rocket-chip/issues/3596

Type of change: other enhancement

Impact: API modification

Development Phase: implementation

Release Notes migrating rocketchip utils to standalone lib

[jerryz123]

Out of my own ignorance, why do you follow this package/package/src/package/*.scala directory structure? Is this what mill prefers?

[lordspacehog]

yeah, it's a different directory layout style adopted to reduce dead paths. in mill if you're not defining "SbtModule" as the module type in the build config it defaults to 'millSourcePath / "src"' as opposed to "millSourcePath / src / main / scala"

[jerryz123]

What happens if the package needs "resources"? Sbt suggests src/main/scala and src/main/resources.

[lordspacehog]

it'll pull from 'millSourcePath/resources' (lots more detail here, https://mill-build.com/mill/Modules.html#_millsourcepath) we can do it the other way too, just need to change the build config to tell it to use the SBT layout

[jerryz123]

I find the mill directory tree much less understandable than the sbt one...

This is how I see rocketutils is structured. The standalone diplomacy is structured the same way.

projectroot/
  build.sc
  project/ <- why does this level of hierarchy need to exist?
    src/   <- does this contain all sources? Or just scala sources? How do you differentiate between main/test sources? 
      project/ <- why does this level of hierarchy need to exist?
        xxx.scala

Compared to SBT, where every level of the tree has some useful meaning.

 projectroot/
   build.sbt
   src/
     main/    <- differentiates core functionality from tests
       scala/  <- differentiates between scala-compiled sources and resources
         xxx.scala
       resources/
         csrc/  <- differentiates filetype
           xxx.cc
         vsrc/
           xxx.v
     test/
       scala/
         xxx.scala
       resources/
         csrc/
           xxx.cc
         vsrc/
           xxx.v

[lordspacehog]

Currently i've organized it this way to prep for pulling the source out into it's own repo.

as far as hierarchy it's specifically aimed at allowing you to organize around build unit vs around a much more complicated package oriented structure.

repo_root/
    build.sc
    contained_build_1/
        src/ <- everything after here is not actually defined by mill and it doesn't care, it just pulls in all scala sources it can find, same as SBT.
            package/ <- this is often the same as the project name and omits the organizational info (eg. org.chipsalliance)
        resources/ <- resource files just like SBT
            some_project_resource.sv
        test/
               package/ <- per package organized unit tests, per scalatest, can be embedded as an object within the larger build
    contained_build_or_alt_lib/
        src/
        resources/
        test/

it's effectively the same but with way fewer dead levels of hierarchy in the folder structure (which can quickly cause problems with long file names and deep nested structures in some filesystems like ext4)

[jerryz123]

Ok I see the motivation. Do you think we can do this as a single PR that just adds rocket-utils as a dependency? Like how it was done for diplomacy? Instead of doing this in a bunch of little steps.

[lordspacehog]

yeah, this PR both breaks out the utils package and adds it as a dependency for rocketchip. This should let us make a new repo for the utils stuff and then just move that one folder out and update the build to pull from the new repo.