What toolkit files should be included in a toolkit release archive?

[joergboe]

When making a toolkit release we should upload an archive with the binary of the tookit that customers can just download an use. What files should be included in these archive? This list is intended for all standard toolkits (not the topology toolkit).

• toolkit directory : required
• samples directory : required
• demo applications : optional
• tests directory : no
• the generated spldoc (doc folder): required
• the LICENCE.md and README.md : required
• other text files like : DEVELOPMENT.md, GRADUATION_STATUS.md, TESTING.md ... : no
• the global build files : no

What artefacts should the toolkit directory include:

• toolkit.xml, info.xml : required
• java and c++ operator model xml files (folder <namspace>/<operator>) : required
• java and c++ operator icon files (folder <namspace>/<operator>) : required
• C++ operator files (cgt-files, pm-files, icons) : required
• other binaries, required for toolkit usage, common pm files (folder bin, impl/bin) : required
• c/c++ source files of the libraries : no
• c/c++ object files : no
• required c/c++ libraries which are not part of the platform (so-files and header files) : yes, if licence allows re-distribution
• java source files : no
• java class files : no
• toolkit java operator jar file (folder lib, impl/lib, impl/java/lib) : required
• required third party jar files (opt or ext folder) : yes, if licence allows re-distribution
• build scripts to build the libraries : no
• build script for toolkit build : no
• internationalization ressource files c++ (folder impl/nl; .xlf and .dat files) : required
• internationalization ressource files java (messages.properties) : no these are part of the jar-file
• configuration files (etc folder) : required
• git artefacts (.gitignore ... ) : no

What artefacts should the samples directory include:

• All source files of the samples : required
• All required build scriprs for the samples : required
• All StreamsStudio project files : required
• README files for the samples if existing : required
• binary files (sab, class, object, so ...) : no
• configuration files if existing (etc folder) : required
• input sample data files if existing (etc folder) : required
• git artefacts (.gitignore ... ) : no

The toolkit release archive is not intended to be used if the user wants to re-compile the toolkit. Users who want to compile the toolkit should use the source archive that always exists.

Does anyone have any objections or additions?

[markheger]

list seems to be complete This would clean-up the release archive, but in most cases not reduce its size noticeable

[joergboe]

I found that toolkits handle the release archive differently. I would like to establish a common understanding. I would like to have this as a common guideline in the wiki if there are no objections.

[markheger]

+1

[ddebrunner]

I think there's also a place for a toolkit only archive to be part of a release. That would be just the toolkit, maybe even without the docs, no samples.

[ddebrunner]

The list is missing C++ headers required by operators, e.g. toolkits like topology, json that have all their C++ code in headers to keep the toolkit platform indendent.

[ddebrunner]

Also missing Python modules, packages which are under opt including ones provided by the user under opt/python and those installed under opt/.__splpy by spl-python-extract.

[ddebrunner]

the LICENCE.md and README.md : required

Can you expand on how this would work?

I image most install a toolkit release like I do:

cd $HOME/toolkits
tar xzf ~/Downloads/NEW_TOOLKIT_RELEASE.tgz

If the licence and readme files are at the top level then they will overwrite the last toolkit install I did.

Note that (I think) samples works as its sub-directories are just added into the existing ones under $HOME/toolkits/samples, which is why samples in a toolkit should have a unique name.

[ddebrunner]

Actually I think this is overly prescriptive, trying to list every type of file and if it should be included or not. For example, it misses .spl files!

Maybe some higher level rules are enough?

A release contains toolkit(s), documentation and samples. The main purpose is for the toolkit to be used in Streams applications. Documentation is included and samples should be complete and self-explanatory to allow users to understand the toolkit.

Artifacts required to create the toolkit are not included in a release, the complete repo is the mechanism to modify and build a toolkit. Items such as DEVELOPMENT.md, make/ant files, intemediate build files (e.g. .class files), git aritfiacts etc. are not included in a release.

[joergboe]

Yes we should mention the general rule for the file assembly. But anyway I think the list is helpful as an guideline. The list has no claim to completeness. It must be adapted to individual toolkit requirements.

There is more missing:

• then native.function folder with the function.xml
• h/hpp files with operator code and native function code
• spl (splmm) files with composites, type definitions and other stuff

[joergboe]

A further step is to define, how the archive should be organized to enable the user to install more than one toolkit release in parallel.

[joergboe]

And another issue is the place of the generated doc-folder. Currently it is often placed into the toolkit directory. I would propose that the doc folder is placed at the top level (like the samples folder). This way, the toolkit folder alone becomes the minimalistic toolkit installation.

[ddebrunner]

The defined location for documentation artifacts in a toolkit is doc.

https://www.ibm.com/support/knowledgecenter/en/SSCRJU_4.3.0/com.ibm.streams.dev.doc/doc/toolkitartifacts.html

Placing the doc folder at the top level will not work as it can't be used when multiple toolkits are installed to the same location, e.g. $HOME/toolkits.