white paper ideas for LAS

[timm]

everyone please read https://ncsu-las.org/wp-content/uploads/2017/06/2018-LAS-Activities.pdf and, in 1 week, propose ONE thing we could do for this work. (edited)

[timm]

from @azhe825

About the LAS 2018 https://ncsu-las.org/wp-content/uploads/2017/06/2018-LAS-Activities.pdf.

ONE THING: Automation of hypothesize-test-evaluate discovery cycle for secondary studies.

Secondary study: summary of existing primary studies instead of innovative works (design of new methods, algorithms)

Why important:

• should be done before any primary study to understand the state-of-the-art and avoid reinventions
• cost large amount of time and effort for completeness

How it maps to our current research: Hypothesize: hypothesize, research questions Test: review (search and selection) Evaluate: analyze relevant results (is there an answer already? requires innovative work?)

FASTREAD solves one part of Test.

Best Regards,

Zhe, Ph.D. scholar @ CS, NcState http://azhe825.github.io

[timm]

@azhe825: re Automation of hypothesize-test-evaluate discovery cycle for secondary studies.

how can we go from your text mining to model generation? might it be possible to learn predictor for supports, reject links between documents? query those links to decide where the next study should occur?

[azhe825]

@timm Do you mean adding non-text features into the model? If so, I assume the answer is yes. However we need data to evaluate that.

[amritbhanu]

About the LAS 2018:

Title: Methods for developing understanding of low-level event data or large corpuses of unstructured text

Why?

• To make sense out of the unstructured text/security logs before using the data for further experiments.
• Saving time and effort in reviewing those logs.

How?

• Using [LDADE] (https://arxiv.org/abs/1608.08176).

[bigfatnoob]

Title: Characterizing Behavior of Heterogeneous Data.

Why its important?

• Data for Intelligence Community comes from different sources. Manually studying all these sources will not be feasible.
• Comparing data from one source to another will also be a challenge as there is no common ground for comparison.

Solution:

• Embedding data from different sources into a standard n-dimensional space.
• Make comparisons on the n-dimensional space to identify
  • Similar documents: Clustering
  • Outliers: Anomaly detection
• Possible Embedding Techniques:
  • Deep Walk
  • LINE

[dichen001]

Title: Parameter tuning with Bayes Optimization and Crowdsourcing.

Why it's important?

• Most of the current ML models, systems require a lot of parameters to configure, while different configurations usually lead to a dramatic difference for both the performance and the results. Fidning an efficient way for parameter tuning will definetely save a lot of time, money and computation resources.
• Traditional parameter tuning methods do not assume any prior knowledge of the models or the systems, which needs a lot of training time. While in the real world, we do have some existing configuration and related results/performance logs. So we badly need a parameter tuning solution for real world situation.

Solution:

• We could better leverage the history data of model/system configurations by using Bayes optimazation to train a model of parameters as inputs and performance/results as output.
• Also, after model is initially trained, we could apply crowdsourcing to improve our model by providing more valid but cheap data compared with asking experts, which is a standard pratice of the leading tech companies. (Human in the loop)

[WeiFoo]

Title: Privacy, Data Sharing in IoT

Why?

• The lack of standardization creates complexity, vulnerabilities, and fragmentation across devices and service sectors in IoT.
  • How to protect sensitive information in IoT is needed to address. There's no policy to guide how to share privatized data in IoT

How?

• is this related to JF's and Fayola's privacy data research ?

[ginfung]

Title: reducing the cost of machine learning algorithms within cloud environment

Why it's important?

• Most of the machine learning is compute-intensive
• Cloud computing services provide us a pay-to-go way to run machine learning algorithms
• However, in many times, engineers do not know how to select best computing resource to run machine learning (GPU, memory, CPU, core#, bandwidth, storage, etc.)

Solution:

• we can start from deep learning algorithms (LSVM, CNN etc)
• create a "simulator-in-the-loop" framework
• find the best way to deploy ML into cloud environment basing on features of dataset (sparse or not, entry number, etc.)
• use SWAY or other related algorithms.

[vivekaxl]

Title: Estimating cost for Analytic Workflows

Why it's important?

While working with a large dataset, it is very difficult for the analyst to determine the cost or the time required to finish a particular task. So, estimating the cost would be useful to quantify the effort required and can be used to deciding which of the several possible workflows to use.

Solution:

Similar to the Fabolas, we can create a model for cost which is a function of data set size, machine learning technique and preprocessing techniques etc. This can also open options for transfer learning etc.

[rahlk]

Bug Triaging: Which bug to address when?

Why?

• Due to the ubiquity of open source software repositories, there are a large number of reported bugs across various open source projects.
• Applying a bug triaging process will greatly assist software maintenance.
• A key step in such a triaging process is assigning a new bug to the most appropriate developer in order to fix it. This, when done manually, is a time-consuming and a tedious task.
• A human triager - the person who considers a bug and assigns it to a developer needs to be cognizant of the developer activities at different parts of the project and also her/his expertise in the area. This gets very difficult in an open-source scenario.
  • A key challenge in automating this process is the need for identifying key features of the bug and the issue report under consideration.

Solution

• We may use topic modeling like LDA to identify key topics in reported bugs by mining the issue reports of the project. Bugs can be featurized based on their probability of belonging to each one of these topics.
• This may be combined with sentiment features extracted before and after a bug was closed (the issue thread can provide this information).
• The above two can be used to identify and rank developers. The goal would be to find a developer who fixed previous issues related to the topics we have identified and has had a positive sentiments in responses to his fixes.
• We can readily mine the data required for this, and compare our findings with other approaches in literature.