Download and upload intents and examples to CSV format

[vmugra]

AS A Bürokratt Trainer I WANT TO download intents and examples to my computer SO THAT I can store them locally on my computer

Cut this task into smaller chunks

Acceptance Criteria

• [ ] Prepare data for downloading as a CSV file
• [ ] Download intents and current examples with a specific timeframe
• [ ] Add new examples (/intents/examples/add) to a specified intent by the user providing examples by uploading them as CSV

GUI

• [ ] #10

DSL

Views: Intents view (Uploading and downloading nlu files data) Files: rasa-ilmateenused/data/nlu/ each file ending with "nlu.yml" is a separate intent file

Prepare data for downloading as a CSV file

• [ ] Data is fetched from YAML by #65
• [ ] Convert fetched YAML to JSON for further processing by using #45
• [ ] Ruuter DSL extracts only intent of file content for further processing

https://koodivaramu.eesti.ee/buerokratt/plug-and-play/rasa-ilmateenused/-/blob/main/data/nlu/common_teenus_ilm_nlu.yml

Sample input

version: "3.0"
nlu:
- regex: asukoht
  examples: |
    - [Aa]bja[- ][Pp]aluoja
    - [Aa]ntsla
- intent: common_teenus_ilm
  examples: |
    - mis ilm [tallinnas](asukoht) on
    - mis ilm [tallinnas](asukoht) täna on

Sample output

- intent: common_teenus_ilm
  examples: |
    - mis ilm [tallinnas](asukoht) on
    - mis ilm [tallinnas](asukoht) täna on

• [ ] Generate CSV files based on JSON, see https://github.com/buerokratt/Analytics-Module/issues/22 for reference

Sample input based on original YAML file

See https://koodivaramu.eesti.ee/buerokratt/plug-and-play/rasa-ilmateenused/-/tree/main/data/nlu for intents references The below sample covers YAML to CSV, not the Ruuter DSL text format

version: "3.0"
nlu:
- intent: common_eitamine
  examples: |
    - ei
    - mkm
    - iialgi
    - ei meeldi
    - mitte väga
    - Ei ,pole vaja!

Sample output based on sample input (see below)

"ei"
"mkm"
"iialgi"
"ei meeldi"
"mitte väga"
"Ei , pole vaja!"

Download intents and current examples with specific timeframe

• [ ] Download the prepared CSV file via a REST call, see https://github.com/buerokratt/Analytics-Module/issues/14 for reference

Add new examples to a specified intent by uploading examples as CSV

• [ ] #Step 1: REST call to convert CSV to JSON via #45
• [ ] #Step 2: REST call to read the content of a specific intent's YAML defined as part of POST body by using #65

  Step 3 and step 4 might / should be done by /intents/examples/add

• [ ] #Step 3: Convert #Step 2 YAML output to JSON via #45
• [ ] #Step 4: Merge #Step 1 output with intent of #Step 3 via #67

  Step 3 and step 4 might / should be done by /intents/examples/add

• [ ] Save #Step 4 output as a YAML file defined as part of POST body by using #44

[turnerrainer]

@vmugra just the local intents or are we missing a "Publish intents" as a separate issue?

[vmugra]

@turnerrainer applies to all intents (local and public)

[ceirowe]

@vmugra Should intents be downloadable only separately or should there be and option for downloading ALL intents at once? PS! This functionality is not currently designed and should be added to figma.

[ceirowe]

@vmugra GUI buttons for downloading / uploading added. Uploading file type limited to CSV only.

[turnerrainer]

This issue should be cut into smaller chunks.

[turnerrainer]

@RaulAltmae an example of downloading a result as CSV can be seen in http://test.buerokratt.ee:3010/analytics/chats when you click on "CSV"

This will trigger https://github.com/buerokratt/Analytics-Module/blob/main/DSL/Ruuter/POST/csv.yml, which in turn makes a request to https://github.com/buerokratt/Analytics-Module/blob/main/DSL/DMapper/hbs/get-csv.handlebars

[RaulAltmae]

csv file is converted in front-end to base64

"ei"
"mkm"
"iialgi"
"ei meeldi"
"mitte väga"
"Ei , pole vaja!"

Sample input for /rasa/intents/upload

{
  "intent": "common_tervitus",
  "file": "ImVpIgoibWttIgoiaWlhbGdpIgoiZWkgbWVlbGRpIgoibWl0dGUgdsOkZ2EiCiJFaSAsIHBvbGUgdmFqYSEiCg=="
}

[RaulAltmae]

Sample input for /rasa/intents/download

{
    "intent": "common_tervitus"
}

[RaulAltmae]

Pull request https://github.com/buerokratt/Training-Module/pull/185

[KasparMinn]

PR https://github.com/buerokratt/Training-Module/pull/290

This pull request finalizes both downloading and uploading CSV files in adding or receiving examples of any intent on the intents page. Both ways, CSV files are processed with a simple Handlebars script through a Datamapper request. The incoming examples are added to the already existing ones and outgoing ones are compiled from all examples under the intent. It is possible to download one intent's examples and upload them to another. The added examples are reflected in OpenSearch and the timestamp of the change is written into the intents database.

• Added and verified both CSV upload & download capabilities through the GUI
• Fixed the uncertain path issue with deleting intents to work the same as file checking

[ffrose]

Download as well as uploading works as intended.