Define directory structure conventions

jt-nti commented 6 years ago

A couple of things to consider here:

Modularity, e.g. is it worth separating out scripts in some way?
Versioning, i.e. to support fetching scripts from pipelines with curl while preserving ability to make breaking changes

jorgedr94 commented 6 years ago

We have been discussing what the directory structure should be for all different repos including this one. Adding that discussion here so that we can keep track of of it.

Summary of discussion from @yorhodes:

assumption: blockchain-toolchain repo will have separate branches for fabric and composer rationale: toolchain configuration allows for branch specification in HTTP request assumption: build-lib (and potentially blockchain-pipeline-job) repo will have separate directories for fabric and composer rationale: common directory should exist to avoid duplication, consumer scripts can grab relevant directories assumption: chaincode-bootstrap and api-bootstrap repositories will be separated into separate repositories for fabric and composer (i.e. fabric-chaincode-bootstrap, composer-chaincode-bootstrap, etc) rationale: these repositories will be cloned for developer use and only relevant scaffolding code should be provided assumption: we will use the IBM Staging Cloud for blockchain network instances as well as toolchains, etc. temporarily for initial development efforts (production will use the IBM Public Cloud) rationale: limited ability to create networks on public cloud

jt-nti commented 6 years ago

Thanks for kicking off the discussion here. A few comments on the assumptions...

assumption: blockchain-toolchain repo will have separate branches for fabric and composer rationale: toolchain configuration allows for branch specification in HTTP request

The blockchain-toolchain repo will have separate branches for each toolchain but the toolchains should not reflect a fabric/composer split. I think we were considering the following toolchains, i.e. an all in one vs modular split:

end-to-end toolchain to build/test/deploy everything from chaincode to front end apps (could be Go/node chaincode or Composer business network)
chaincode toolchain to build/test/deploy chaincode only (could be Go/node chaincode or Composer business network)
api toolchain to build/test/deploy an API (REST?) for the chaincode

assumption: build-lib (and potentially blockchain-pipeline-job) repo will have separate directories for fabric and composer rationale: common directory should exist to avoid duplication, consumer scripts can grab relevant directories

It would might be useful to split out functions from pipeline-DEPLOY.sh into Composer/Fabric specific scripts but I'm not sure separate directories are necessary.

My guess is that versioning/script update considerations in #5 are more likely to drive the fundamental directory structure.

assumption: chaincode-bootstrap and api-bootstrap repositories will be separated into separate repositories for fabric and composer (i.e. fabric-chaincode-bootstrap, composer-chaincode-bootstrap, etc) rationale: these repositories will be cloned for developer use and only relevant scaffolding code should be provided

Agreed, I think these two repos are likely to be Fabric specific. I guess we'll need a network-bootstrap repository or something like that for Composer (there's already an empty business network which is probably suitable).

assumption: we will use the IBM Staging Cloud for blockchain network instances as well as toolchains, etc. temporarily for initial development efforts (production will use the IBM Public Cloud) rationale: limited ability to create networks on public cloud

Interesting, I'd not considered trying to support anything other than the public cloud. Public cloud should be the focus but there's probably value in making that configurable. See #7

jt-nti commented 6 years ago

Following decision to use git tags/github releases for versioning purposes in #5, I think the build-lib directory structure can be pretty flat; somewhere for the scripts (src, or scripts or something) and probably a test directory for #2

jt-nti commented 6 years ago

Think this issue can be closed now given common scripts are covered in #10 and test scripts are covered in #2

jt-nti commented 6 years ago

Suggested structure for discussion...

jorgedr94 commented 6 years ago

Based on discussion with James, here is the directory structure we agreed on for build-lib

src 
     common.   # common script library that will be "imported" (sourced) by other scripts
     composer  # scripts that contain the logic to drive  the pipeline stages for composer
     fabric         # scripts that contain the logic to drive the pipeline stages for fabric
     prepare.sh. # common script that will decide to call prepare for composer or fabric 
     build.sh.     # common script that will decide to call build for composer or fabric 
     unitest.sh.  # common script that will decide to call  unitest for composer or fabric 
     deploy.sh   # common script that will decide to call deploy for composer or fabric
test
    common # contains tests for common lib, each file in common should have their own test suite
    fabric     # contains tests for fabric scripts, each file in fabric should have their own test suite 
    composer  # contains tests for fabric scripts, each file in fabric should have their own test suite
    prepare.bats # contains tests for common prepare script 
    build.bats    # contains tests for common build script 
    unitest.bats  # contains tests for common unitest script 
    deploy.bats # contains tests for common deploy script

Sample of released tar files generated by Travis service. These files will be what the pipeline use pipeline-scripts-0.1.tar
pipeline-scripts-0.2.tar

IBM-Blockchain-Starter-Kit / build-lib

Define directory structure conventions #1