search

IBMStreams / administration

Umbrella project for the IBMStreams organization. This project will be used for the management of the individual projects within the IBMStreams organization.
Other
19 stars 10 forks source link

Proposal for new repository: streamsx.cybersecurity.accelerator #79

Closed cancilla closed 8 years ago

cancilla commented 8 years ago

I would like to propose a new repository called "streamsx.cybersecurity.accelerator". This repository will be used to maintain the cybersecurity "accelerator" applications. Unlike other toolkit samples that simply demonstrate how to use specific operators within the toolkit, these apps are intended to be used as a starting point for building larger cybersecurity applications.

Furthermore, by maintaining these apps in a separate repository, I can generate releases that contain any necessary dependencies for running the apps, such as the network toolkit.

leongor commented 8 years ago

+1

2015-11-17 22:57 GMT+02:00 James Cancilla notifications@github.com:

I would like to propose a new repository called "streamsx.cybersecurity.accelerator". This repository will be used to maintain the cybersecurity "accelerator" applications. Unlike other toolkit samples that simply demonstrate how to use specific operators within the toolkit, these apps are intended to be used as a starting point for building larger cybersecurity applications.

Furthermore, by maintaining these apps in a separate repository, I can generate releases that contain any necessary dependencies for running the apps, such as the network toolkit.

— Reply to this email directly or view it on GitHub https://github.com/IBMStreams/administration/issues/79.

Best regards, Leonid Gorelik.

ddebrunner commented 8 years ago

Just curious how a 'starting point' is really different to a sample, even it is more involved than a single operator.

Is a 'starting point' point a reuseable asset (composite or application) whose api becomes stable at some point, or just an indication of how the pieces all fit together?

For example I can imagine a sample application including more features over time as more functionality is added to toolkits or the product, would that be true of a 'starting point' or will its api be fixed as customers may have used it directly?

mikespicer commented 8 years ago

I find the name confusing as the cybersecurity toolkit in V4.1 is also called an accelerator. How does this relate to the cybersecurity toolkit in V4.1? and what is the split between open source and proprietary product code?

cancilla commented 8 years ago

These applications use the cybersecurity toolkit from v4.1. The operators in the toolkit use machine learning models to analyze DNS response records. When analyzing raw DNS traffic over the wire, the data needs to be enriched and filtered before being ingested by the analytics. Otherwise, the results of the analytics may not be accurate.

These applications contain the necessary upstream operators to perform the required enrichment and filtering on the raw DNS response records. This ensures that the data being sent to the analytics is complete and correct. Developers who want to build cybersecurity applications using the new toolkit in v4.1 should extend these applications.

The cybersecurity toolkit is only available via the product. These initial or starting applications (or whatever we want to call them) will be available on github since they will continue to be developed and new applications may be added in the future.

engebret commented 8 years ago

I do not think we should use "accelerator" as that has other connotations. Perhaps application would be a better option. Or even just use streamsx.cybersecurity as this would be the open source suite of applications built on the various toolkits. In my view we should include a core set of the necessary cybersecurity components in the github repository so they are fully extensible by people using Streams. The Streams product can pull this repository code, other toolkits, and any components that are closed source as needed.

chanskw commented 8 years ago

In my mind, I use the term "accelerator" loosely.

But I may the only one thinking this way.

The other problem that we are trying to solve with this proposal:

Perhaps the name is confusing. If we agree that a repository is good for these sample applications, perhaps we can call the repository: streamsx.cybersecurityApp

Please let us know what you think. Thanks!

leongor commented 8 years ago

I like the idea, not so like the name. Accelerator usually means for me - take the existing product/toolkit (like cyber toolkit) and build on top of it some half ready solution. You suggest more to extend it with additional filters, enrichers, parsers and etc. Why not just call it streamsx.cyber? Namespaces inside can provide then more specific information.

mikespicer commented 8 years ago

I agree with @leongor, good idea, bad name. Accelerator is an ambiguous and overloaded term that it would be best to avoid in the toolkit name.

chanskw commented 8 years ago

Ok. I think we have consensus that it's a good idea to have a separate repository for starter applications.

Here are some naming suggestions. Please let us know your preference and vote.

streamsx.cycberStarterApp streamsx.cycberApp streamsx.cycberDemoApp streamsx.cybersecurityStarterApp - to more closely tie to the cybersecurity toolkit. I am not sure if just having cyber is confusing

If you have other ideas, please let us know so we can close on this issue.

Thanks...

ddebrunner commented 8 years ago

So it does seem like these are applications, that maybe publish/export their streams to be consumed by downstream customer specific applications, thus they are expected to be stable once a release is made. I.e. I could just run these apps as-is, and depend on them in future releases of the toolkit (once a full release is made). Is that true?

mikespicer commented 8 years ago

I like the use of Starter or demo in the name if they should not be considered a full solution with compatibility across releases.

chanskw commented 8 years ago

Hello, trying to close this issue.

I propose to name the new repository as streamsx.cybersecurity.starterApps. If you disagree with the name, please propose a name that you are comfortable with.

Please vote. Thanks!

ddebrunner commented 8 years ago

Is it intended that the assets in the toolkit are intended to have a stable api that customers can rely on as the toolkit evolves?

chanskw commented 8 years ago

@cancilla Please correct me if I am wrong.

In my understanding, these are not stable APIs that the customer builds applications upon. These are starter applications that customer can use to start building their applications. They are expected to use and modify these applications as they need. They are meant to speed up the adoption of the cybersecurity toolkit.

As the cybersecurity toolkit evolves, the applications will be updated to take advantage of new features.

cancilla commented 8 years ago

@chanskw You are correct. As the toolkit evolves, these applications will continue to evolve. Previous versions of the applications can certainly be maintained in different branches. The intention here is to give new users a starting point for building their own cybersecurity applications.

+1 for streamsx.cybersecurity.starterApps

ddebrunner commented 8 years ago

+1 though reusable assets will be more valuable to users than copy & modify assets. Maybe as the toolkit develops reusable assets will naturally appear.

mikespicer commented 8 years ago

+1

leongor commented 8 years ago

+1

chanskw commented 8 years ago

Closing... repository created