Feature Request: Formation Output

[mysticole928]

I don't know how to make a feature request.

What I'd like is the ability to get the array data after each call. I want to do some analysis (big data style) and could use the data points. (Even if this is a feature that is hidden from the naked eye.)

[danlyke]

Tell me more: When you say "array data", you mean the formation information? And are you wanting this in a saved file, or are you talking about hacking on code?

My eventual goal is to save sequences, and save the formation transformation in those sequences so that we can automatically find submodules, and suggest alternative modules.

[mysticole928]

Hi Dan!

I've looked at the SD source code.

It seems as if, after each command, to display the checkers, It uses an 8 item array. (I think that's what I saw.)

I'm pretty sure, but not positive, that each element is the location of one of the dancers. I would like that data.

My goals are to use it for analytics, visualization, and other machine learning tools.

Stephen

p.s. I started working on a project similar to yours, minus the DS integration. I want to use the Electron (JavaScript) framework and integrate it with iTunes playlists. It's still in the planning stages.

Currently, I'm working on a Python program to add mp3 metadata to my music.

My filenames have song name, label, and number. I also have square brackets in the filename that let me know if there a background vocals, leads, etc.

I want to pull that data out and have the song name be one of the ID3 tags, then have information about label, number, and genre, etc.

I want to be able to build smart playlists in iTunes that I can use in my music player.

---

From: Dan Lyke notifications@github.com Sent: Wednesday, April 18, 2018 2:27 PM To: mpogue2/SquareDesk Cc: Stephen Cole; Author Subject: Re: [mpogue2/SquareDesk] Feature Request: Formation Output (#339)

Tell me more: When you say "array data", you mean the formation information? And are you wanting this in a saved file, or are you talking about hacking on code?

My eventual goal is to save sequences, and save the formation transformation in those sequences so that we can automatically find submodules, and suggest alternative modules.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/mpogue2/SquareDesk/issues/339#issuecomment-382535051, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ASod6s3QpBMi5-xg5S6a7zcJA3e5ZAWlks5tp6-_gaJpZM4Tawu5.

[mpogue2]

I'm not sure how having access to the internal SD data would help the caller using SD...could give some examples of how an SD user might use this data? I'm generally in support of analytics and machine learning, but since the amount of data required for those things is generally quite large, I'm not sure how anything that one caller does on one machine in SD (inside SquareDesk) would help.

An alternate approach (if I understand what you're trying to do) might be to process sequences of calls off the internet, using a customized version of SD that dumps internal state into, say, CSV format. Then you could process that with Python or R or Tensorflow or whatever is in your preferred toolbox (I'm a data analysis/analytics guy in my day job -- can you tell? :-)

[mysticole928]

Regarding data collection, I'm going to need help from the square dance tribe, if you will, to gather data points.

This means I either need a tool that i can put in people's hands or have a way to import data sets. I'm not a CPP programmer but I'll probably be able to figure out SD. It is well written with lots of solid comments.

I'm going to start with NLP and singing call figures. However, I want more data. (Because I don't know what I don't know.)

I have three goals.

Visualization of the dance experience. Computer generated sequences in the style of certain callers. Analysis of calling styles, in general.

I'm not a data scientist, but I play one (from time to time) at my day job at Amazon Web Services.

Stephen

---

From: Mike Pogue notifications@github.com Sent: Wednesday, April 18, 2018 7:54 PM To: mpogue2/SquareDesk Cc: Stephen Cole; Author Subject: Re: [mpogue2/SquareDesk] Feature Request: Formation Output (#339)

I'm not sure how having access to the internal SD data would help the caller using SD...could give some examples of how an SD user might use this data? I'm generally in support of analytics and machine learning, but since the amount of data required for those things is generally quite large, I'm not sure how anything that one caller does on one machine in SD (inside SquareDesk) would help.

An alternate approach (if I understand what you're trying to do) might be to process sequences of calls off the internet, using a customized version of SD that dumps internal state into, say, CSV format. Then you could process that with Python or R or Tensorflow or whatever is in your preferred toolbox (I'm a data analysis/analytics guy in my day job -- can you tell? :-)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/mpogue2/SquareDesk/issues/339#issuecomment-382593712, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ASod6mt47UlrgASReRgFwsRdXOXeIVWqks5tp_x7gaJpZM4Tawu5.

[danlyke]

We grab the ASCII picture that sd outputs and convert that into coordinates and rotations for the pictures. So we definitely have that data internally, and as I figure out how I want to save sequences, I will be saving the formations (another hope is that by matching the before and after formations we can use Taminations animation data for intermediate positions).

Mike has already experimented with Markof chains for singing call data, and I definitely think there's interesting stuff to be mined there, but there'll be a bunch of reformatting and typing to get data into it.

When we do, well likely store everything in SQLite, so access will be language independent.

If you have ideas for a schema for that, I'd love to bat those around!

[mpogue2]

Hey, Stephen, where are you located? I work for Amazon on Echo products, and I'm in SJC18 (East Palo Alto)...

[mysticole928]

Markof chains are a bit above my current knowledge level but I'm going to get there. I'm getting up to speed on linear algebra. I'm taking the long way in my refresh because I want to fill in the gaps of my understanding on the way.

I look at modules and choreography other's have written to see how they connect calls. For a while, I was playing with some natural language parsers because I wanted to find out, for a given call, what preceded and followed it.

To illustrate, in Advanced, I know far too many callers that just throw calls out because they fit. Steve Kopman's choreo, I've found, moves people from place to place and, often, they still feel "normal." (They get to a normalized state quickly.)

To use AI and ML, I need to get data as numbers. Using the array output of SD, I think, would help. I just don't feel like sorting through C++ code. (My language is Python.... I'm going to start learning JavaScript so I can do some dashboards .)

[danlyke]

So I'm starting to get further on the notion of building a database schema and storing the output of SD. If you've got further input on this, I'm trying to figure out things like how to think about technical zeroes, especially of the "star thru, pass thru, bend the line, star thru" form...