a surveyor for Widget 2-grams

[pq]

A jumping off point for iteration is defined in example/widget_surveyor.dart

https://github.com/pq/surveyor/blob/96cd7507028b95c4d529401797db224959eabcc8/example/widget_surveyor.dart#L13-L18

More details/instructions in the README.

Feedback welcome!

/cc @jayoung-lee @InMatrix

[pq]

@jayoung-lee @InMatrix, the current output looks like:

Column->Container : 1
Column->null : 1
Container->Row : 1
Expanded->Column : 1
FlareActor->Container : 1
FlareActor->GestureDetector : 1
FlareActor->Scaffold : 1
GestureDetector->Expanded : 1
HomePage->null : 1
MaterialApp->null : 1
PageView->Scaffold : 1
Row->GestureDetector : 1
Scaffold->Column : 1
Text->Column : 5

where -> signifies "child of" which is pretty arbitrary. Any other ideas most welcome.

[jayoung-lee]

Great!

I have one suggestion to make analysis easier: Could you replace ":" with "," so that I can easily export this output to a csv file? Actually, it will be awesome if somehow your code can generate such a csv file as an output!

[pq]

I have one suggestion to make analysis easier: Could you replace ":" with "," so that I can easily export this output to a csv file?

Sure! As of https://github.com/pq/surveyor/commit/e54bad0b00fac543415cf3bd9545523ec5a15efb, it now emits a list of entries like this:

Column->Container,1

Is that good?

Actually, it will be awesome if somehow your code can generate such a csv file as an output!

That's easy too.

Let's talk about what we want ultimately... One file per project? Or one file for all of them? If it's one file for all of them do we just want total counts?

[jayoung-lee]

From my perspective, the easiest approach is a single csv file for each analysis (e.g, widget usage, widget 2-gram usage, package usage, etc.) with distinct and consistent patterns in the filename.

I’ll just describe what I anticipate below (based on our conversation so far)!

Let’s say there is a submission titled “flutter_create” and we are interested in widget usage, 2-gram widgets, and package usage. Then I’d expect three files,

“flutter_create_2gram.csv”
“flutter_create_widget.csv”
“flutter_create_package.csv”

Then the csv file that contains 2-gram analysis results may be titled flutter_create_2gram.csv with these:

Column->Container, 1
Column->null, 1
Container->Row, 1
Expanded->Column, 1
FlareActor->Container, 1
FlareActor->GestureDetector, 1
FlareActor->Scaffold, 1
GestureDetector->Expanded, 1
HomePage->null, 1
MaterialApp->null, 1
PageView->Scaffold, 1
Row->GestureDetector, 1
Scaffold->Column, 1
Text->Column, 5

Similarly, for the general widget usage analysis (i.e., counting the number of widget appearance in the code), I’d expect something like flutter_create_widget.csv that has:

Column, 7
GestureDetector, 1
Expanded, 10
Container, 3
FlareActor, 4
Text, 10
…

And for packages, flutter_create_package.csv and something like above.

In addition to these files, If we have some interesting submission-level stats, we can also have a single summary file for all of the submissions (summary.csv), each row being a submissions and each column being a summary stat, such as:

submission_name, unique_widgets_count, unique_2grams_count // this is a header and may be expanded with any summary measures we find interesting
flutter_create, 22, 34
project_a, 30, 35
project_b, 25, 28
…

Taken all together, the widget_surveyor may generate a set of files:

“summary.csv”
“flutter_create_2gram.csv”
“flutter_create_widget.csv”
“flutter_create_package.csv”
“project_a_2gram.csv”
“project_a_widget.csv”
“project_a_package.csv”
“project_b_2gram.csv”
“project_b_widget.csv”
“project_b_package.csv”
…

Does this make sense to you? Am I missing anything? I’m open to any suggestions! :)

[pq]

Sounds good!

So far I've got support for 2 grams.

Running dart example/widget_surveyor.dart test/data yields two files:

[~/src/repos/surveyor] (master) $ dart example/widget_surveyor.dart test/data
Recursing into "test/data"...
Checking dependencies...
Analyzing...
Analyzing "basic_app"...
Writing 2-Grams to "basic_app_2gram.csv"...
Analyzing "animated_container"...
Writing 2-Grams to "animated_container_2gram.csv"...
Finished.
No issues found!

[~/src/repos/surveyor] (master) $ cat basic_app_2gram.csv 
AppBar->Scaffold,1
Center->AppBar,1
Column->Center,1
FloatingActionButton->Column,1
Icon->FloatingActionButton,1
MaterialApp->null,1
MyApp->null,1
MyHomePage->null,1
Scaffold->null,1
Text->AppBar,1
Text->Column,3

[~/src/repos/surveyor] (master) $ cat animated_container_2gram.csv 
AnimatedContainer->Center,1
AnimatedContainerApp->FloatingActionButton,1
AppBar->Scaffold,2
Center->AppBar,2
Column->Center,1
FloatingActionButton->Center,1
FloatingActionButton->Column,1
Icon->FloatingActionButton,2
MaterialApp->FloatingActionButton,1
MaterialApp->null,1
MyApp->null,1
MyHomePage->null,1
Scaffold->MaterialApp,1
Scaffold->null,1
Text->AppBar,2
Text->Column,3

I'll take a look at widget counts and summaries in a bit.

In the meantime, @jayoung-lee: does this look about right?

[jayoung-lee]

Looks perfect!

[pq]

Widget counts are being collected now:

[~/src/repos/surveyor] (master) $ cat animated_container_widget.csv 
MyApp, 1
MaterialApp, 2
MyHomePage, 1
Scaffold, 2
AppBar, 2
Text, 5
Center, 2
Column, 1
FloatingActionButton, 2
Icon, 2
AnimatedContainerApp, 1
AnimatedContainer, 1

[~/src/repos/surveyor] (master) $ cat basic_app_widget.csv 
MyApp, 1
MaterialApp, 1
MyHomePage, 1
Scaffold, 1
AppBar, 1
Text, 4
Center, 1
Column, 1
FloatingActionButton, 1
Icon, 1
[~/src/repos

@jayoung-lee: as a next step I think we should do some validation. Perhaps we could select a project or two, do a manual count and verify that the results are as expected?

[jayoung-lee]

Sounds like a good idea! If you could pick a couple of examples (ones that you've already run the surveyor on?), I will take a look at them.

[pq]

As a quick sanity check, you could look at the two projects checked into test/data.

https://github.com/pq/surveyor/blob/a8f68cbebf1a706ef82627e9fac13280eef1ca1f/test/data/basic_app/lib/main.dart#L37-L65

and

https://github.com/pq/surveyor/blob/a8f68cbebf1a706ef82627e9fac13280eef1ca1f/test/data/animated_container/lib/main.dart#L21-L71

Their 2-grams and counts are inlined in the comments above (https://github.com/pq/surveyor/issues/1#issuecomment-491085264 and https://github.com/pq/surveyor/issues/1#issuecomment-491442839 respectively.)

[InMatrix]

Wow, nice progress!

Some minor feedback on the notation:

where -> signifies "child of" which is pretty arbitrary. Any other ideas most welcome.

Would it be more intuitive to have -> signify "parent of", since the parent is on the left (and upper) side of the child in the code? I was initially puzzled by Text->AppBar, assuming Text was the parent.

[pq]

Would it be more intuitive to have -> signify "parent of"

Totally up to you! I think my point of reference was inheritance diagrams where -> means "subtype of" but that's not really the same thing either. 😬

Another thought is to be less suggestive and just use a delimiter like : for the pairing.

Whatever makes the most sense to you all works for me and it's an easy change!

[jayoung-lee]

I took a look at the basic_app first. This is what the Flutter Outline gives me:

So I guess the FloatingActionButton Is under Scaffold instead of Column? The surveyor output says FloatingActionButton->Column,1. Similarly, Center is under Scaffold instead of AppBar?

And for the widget counts, why was MaterialApp counted twice? Is it because MyApp extends MaterialApp and returns a new MaterialApp again? It makes other widgets, for example FloatingActionButton, counted twice as well. I thought what we want to count is the number of the widget's appearance in the code itself, but I might be wrong.. WDYT @pq and @InMatrix ?

And I'd like +1 to @InMatrix 's suggestion, just because it is easier for humans to read (or me!). I find it much easier to read things top-down and left-to-right :) For example,

null -> Scaffold, 1
Scaffold -> AppBar, 1
AppBar -> Text, 1
Scaffold -> Center, 1
Center -> Column, 1
Column -> Text, 3
Scaffold -> FloatingActionButton, 1
FloatingActionButton -> Icon, 1

[pq]

Ok, so that was obviously not right! Thanks for taking a look (though I should probably have noticed 😬).

Here's where we are after some fixes:

[~/src/repos/surveyor] (master) $ cat basic_app_2gram.csv 
AppBar -> Text, 1
Center -> Column, 1
Column -> Text, 3
FloatingActionButton -> Icon, 1
MaterialApp -> MyHomePage, 1
Scaffold -> AppBar, 1
Scaffold -> Center, 1
Scaffold -> FloatingActionButton, 1
null -> MaterialApp, 1
null -> MyApp, 1
null -> Scaffold, 1

At a quick inspection that looks closer to the outline:

But please do double-check!

[jayoung-lee]

No worries! The new output looks correct to me too :D