Report 1

[efe-onal-2016400267]

• [x] the TA wants every package import to be commented to say why we use it

Abstract

• [x] Provide a short summary of the goal of the project, the implementation approach, the machine learning results, the functionality of the dialog system, and possible improvements (250 words maximum).

Introduction

• [x] A brief introduction to the system design and implementation process, with details on the approach. Discuss the scope of the project, the type of the system and the type of user interaction and the components of the system. Refer to the following sections (using the section numbers) when discussing aspects of the project and system.

Data

• [x] Description of the dataset: domain, number of dialogs, dialog format, number of duplicate utterances, discussion of the language and grammar used in the data and whether this affects system performance or not, and other properties you consider relevant
• [x] The dialog state transition diagram
• [x] Some example dialog snippets to illustrate some of the states, choice points etc. with explicit references (i.e., "this example sentence is related to dialog state 5 in the model because of X")

Machine learning

• [x] Describe how you developed your baseline systems (including any preprocessing and parameter settings)
• [x] Design/layout of the machine learning model
• [x] Data preprocessing, including statistics on the data (label distribution, utterance length, number of out of vocabulary words, etc.)
• [x] Quantitative evaluation for various models and parameter settings
• [x] Error analysis (i.e., for which utterances is the classification wrong, are there specific problematic dialog acts, specific vocabulary, influence of utterance length, could any improvements be made in preprocessing, etc.)
• [x] Discussion of difficult instances with examples

Dialog manager

• [x] A description of the state transition function, including example dialogs and references to the state diagram. Any reference to the state diagram must contain the number of the state that is referenced
• [x] Description of system utterance templates, including examples of generated sentences. The report must contain the complete list of all system utterance templates, showing both the template itself and an example sentence that is generated using this template
• [x] A full example dialog must be provided (possibly in the appendix) that traverses the diagram from the start state to the end state

Reasoning

• [x] A description of the reasoning component, including example dialog snippets
• [x] A discussion on how to handle possible contradictions, with examples

Configurability

• [x] A list of configuration options
• [x] A high-level description of how the options are set by the user and how they are implemented in code

Conclusion Discussion of the approach for the dialog system in general:

• [x] General overview of the project, including a summary of the approach and the main findings
• [x] Limitations
• [x] Possible improvements
• [x] Alternative approaches

Discussion of the approach for dialog act classification:

• [x] General overview of the experimental methodology
• [x] Limitations
• [x] Possible improvements
• [x] Choice of machine learning algorithm and alternative approaches

[karolrogozinski]

Also grading can be helpful:

• [x] Title

• [x] Abstract

• [x] 1.5pt Report (general)

  • [x] 0.5pt: Introduction to the system implementation process
  • [x] 0.5pt: Overview of contributions of individual group members
  • [x] 0.3pt: Conclusion
  • [x] 0.2pt: Possible improvements

• [x] 1.0pt Implementation (general)

  • [x] 1.0pt: Code quality and report guidelines as described in the assignment

• [x] 3.0pt Part 1a

  • [x] 1.7pt Report
  • [x] 0.3pt: Baseline systems
  • [x] 0.3pt: Design of machine learning model
  • [x] 0.3pt: Data description, preprocessing, statistics on data duplication
  • [x] 0.3pt: Quantitative evaluation, including data deduplication
  • [x] 0.4pt: Error analysis
  • [x] 0.1pt: Difficult instances
  • [x] 1.3pt Implementation
  • [x] 0.2pt: Data loading and preprocessing
  • [x] 0.4pt: Baselines
  • [x] 0.3pt: Machine learning
  • [x] 0.1pt: Data deduplication
  • [x] 0.3pt: Code to perform evaluation

• [x] 2.5pt Part 1b

  • [x] 1.1pt Report
  • [x] 0.1pt: Diagram development: choices made, alternatives, description of data exploration related to diagram development
  • [x] 0.4pt: Explanation of final diagram
  • [x] 0.4pt: Example dialog/dialog snippets with references to the state diagram
  • [x] 0.2pt: Description of system utterance templates with example sentences
  • [x] 1.4pt Implementation
  • [x] 0.5pt: Diagram (completeness, level of detail, clarity)
  • [x] 0.3pt: State transition function
  • [x] 0.4pt: Identifying user preferences
  • [x] 0.1pt: Levenshtein distance
  • [x] 0.1pt: CSV lookup function

• [x] 2.0pt Part 1c

  • [x] 0.8pt Report
  • [x] 0.4pt: Description of reasoning component, including example dialog snippets
  • [x] 0.4pt: Description of configurable features
  • [x] 1.2pt Implementation
  • [x] 0.4pt: Implication rules
  • [x] 0.4pt: Four configurable features
  • [x] 0.4pt: Integration with dialog management

Total: 10.0pt