Cherry-pick sources

[eurochriskelly]

The current default behaviour for loading data [code||content] into MarkLogic allows for one folder of each type.

However, there are times when the source code and sample is distributed between different software blocks.

Since the source section in the config file already allows for multiple entries, it would be nice to extend the functionality to allow for composition of sources.

e.g instead of

source: [ { name: 'src', ...}, { name: 'data', ... } ]

allow the following

source: [
  {
    name: 'named-src1',
    description: 'some code',
    type: 'src',
    dir: 'src/lib/moduleY/src',
    include: [ '*.js', '*.sjs' ]
  },
  {
    name: 'source-for-module-x,
    description: 'some other code',
    type: 'src',
    dir: 'src/lib/moduleX/src',
    include: [ '*.js', '*.sjs' ]
  }
]

[fgeorges]

Thank you Chris! Before commenting further, I'd like to make sure that your example using src and src/subdir is just an accident, and the fact that one is a subdirectory of the other is not the point you are trying to make.

If it is, then I need further explanation of how it would work.

I guess you rather mean something like:

• src/lib/moduleX/src
• src/lib/moduleY/src

Am I right?

[eurochriskelly]

Yes, you are right. I've updated the config sample to better match what you are saying.

[fgeorges]

Thank you Chris.

The type is used to describe the type of content of the directory, its shape, its morphology. In the simple case, it is a bunch of modules to be uploaded as is. In the case of rest-src, it must have at most 3 subdirs (services, transforms and root), which must be installed differently.

The name of source sets is used to pick one or another, and allow to have several of the same type.

It seems what you want is rather to be able to use 2 different source sets in the same command. More like either:

• have 2 dirs in the same source set
• have one (named) source set being composed of several other source sets

The former case would be something like (and if I am right is what you described in #18):

sources: [{
    name: 'src',
    dir: ['dir/foo', 'dir/bar']
}]

the latter looking something more like:

sources: [{
    name: 'src',
    subsets: ['foo', 'bar']
}, {
    name: 'foo',
    dir: 'dir/foo'
}, {
    name: 'bar',
    dir: 'dir/bar'
}]

Both allow you to use the single named source set src (which in addition is the default one for deployment) for deploying both dirs.

The latter in addition lets you deploy only one of them by explicitly mentioning the named source set foo or `bar.

Is that correct?

[fgeorges]

If we take a step back, the latter option above allows for having one (abstract) coarse-grained source set to deploy everything in the project for instance, and finer-grained source sets for component A, component B, modules for library L, and library M, triggers, REST services, etc.

[eurochriskelly]

Yes, I think the latter is better than my proposal/workaround in #18. I can see several uses for that.

This would be especially useful if the subsets could be deployed in the specified order.