Add CLI Support - Githubissues

I've recently joined a group in North Carolina that wants to build a platform for citizens to submit redistricting proposals. I've just setup a repository for this, https://github.com/CitizenRedistrictingNC/nc-redistricting where we plan to build the system.

I'm not familiar with the algorithm that the team plans to use (yet), but I've been told that the autoredistrict project could be used if it had support for automation.

I plan to start working on a PR for your project to add CLI support. If you have any requests or thoughts about I'd love to hear them.

awesome.

let me say right away i've down a lot of work in that regard. check out the "InstructionProcessor" class ( https://github.com/happyjack27/autoredistrict/blob/master/src/ui/InstructionProcessor.java ) and... whoa do i not have the script window class checked in? ... i'll check tonight.

but yeah, right now you can write a script in a file, and then run autoredistrict with a command--line argument to run it.

you can also have it turn off the ui. though it'll still complain if there's no window manager, so on linux you have to use a framebuffer. (xvfb)

so, it's largely there. it already does have support for automation, just not via a CLI, via passing it a script file as a command line argument.

(I haven't documented any of the commands yet, though.)

I'll be publishing a new release soon.

On Tue, Jun 21, 2016 at 4:10 PM, Dane Summers notifications@github.com wrote:

I've recently joined a group in North Carolina that wants to build a platform for citizens to submit redistricting proposals. I've just setup a repository for this, https://github.com/CitizenRedistrictingNC/nc-redistricting where we plan to build the system.

I'm not familiar with the algorithm that the team plans to use (yet), but I've been told that the autoredistrict project could be used if it had support for automation.

I plan to start working on a PR for your project to add CLI support. If you have any requests or thoughts about I'd love to hear them.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/happyjack27/autoredistrict/issues/154, or mute the thread https://github.com/notifications/unsubscribe/AB7NQfygxgI3YFreyKk-LkzJ1YxxSHLTks5qOFNMgaJpZM4I7LQf .

Thats great! I have so far only gone skin deep in my exploration of the Applet.java and saw the no_gui option; the xvfb reference is definitely helpful in that we plan to make an automated processor on some flavor of linux that would launch autoredistrict.

I'll start testing it out. If all works well at the least I can contribute some documentation for you.

i've actually been using my web server to run optimizations. Here's a sample of how to run a script headless on linux:

xvfb-run -a -e xvfb.log java -jar -Xmx4096M -Xms512M autoredistrict.jar run subscript56

56 is the state FIPS code ( https://www.epa.gov/enviro/state-fips-code-listing ) for Wyoming.

that line is from a shell script that does all 50 states, which i used to generate optimized multi-member and single-member districts for this: http://autoredistrict.org/all50/version3/CD_PRES/interactive_map.php

i just looked over the bill - http://www.ncga.state.nc.us/Sessions/2007/Bills/Senate/PDF/S1093v1.pdf

i see it doesn't include any anti-gerrymandering rules.

In that case, you can disregard all the fairness criteria. The script would be:

LOAD 37 2020 2012 COPY FEATURE CD_2010 CD_2020

SET DISTRICTS COLUMN CD_2020 SET DISTRICTS NUM_DISTRICTS 120 SET DISTRICTS SEATS_PER_DISTRICT 1

SET WEIGHT GEOMETRY_FAIRNESS 0.0 SET WEIGHT SPLITS 0.5 SET WEIGHT CONTIGUITY 1.0 SET WEIGHT POPULATION 1.0

SET EVOLUTION MUTATE_RATE 0.8 SET EVOLUTION POPULATION 200

GO WHEN MUTATE_RATE 0.1 STOP SAVE EXPORT STATS EXPORT PIE EXPORT BLOCK EXIT

(note, i'm a little concerned about the transition to 2020 - whether census.gov remains consistent with their file placement, etc. may need to adjust the programming a little after the 2020 data is published.)

er.... that would be SLDL_2010 & SLDL_2020.

Wow, thank you! You are correct, there is no gerrymandering built into that bill - and currently it isn't law. Our group hopes to build a platform that conforms to S1093 and hold a redistricting competition in the Spring to show that such a system is possible, and garner support around it (I'm new to the project and a programmer, not a policy/GIS person, so my knowledge around these particular aspects are fuzzy at best!).

I'd just worked out how to manually download the NC shapefiles and start the algorithm; this looks much more complete than what I'd come up with so far:

SET EVOLUTION ELITE_FRAC 0.33
SET EVOLUTION ELITE_MUTATE_FRAC 1.0
SET WEIGHT DESCRIPTIVE 0.0
OPEN "../Documents/classes/autoredistrict/nc/tl_2012_37_vtd10.shp"
SET ETHNICITY COLUMNS
SET ELECTION COLUMNS
SET POPULATION COLUMN POPULATION
SET COUNTY COLUMN COUNTY_NAM
SET WEIGHT COUNT_SPLITS TRUE
SET ETHNICITY COLUMNS
SET ELECTION COLUMNS
SET ETHNICITY COLUMNS VAP_WHITE VAP_BLACK VAP_HISPAN VAP_ASIAN VAP_HAWAII VAP_INDIAN VAP_MULTI VAP_OTHER
SET ELECTION COLUMNS PRES12_DEM PRES12_REP
SET DISTRICTS NUM_DISTRICTS 13
SET CONSTRAIN CONTIGUITY  TRUE
SET CONSTRAIN POPULATION  TRUE
GO

I'm running your version now...clearly automation is working!

Ultimately we want to present the result of a run on a map on a website; do the export * commands provide data that I could derive a geojson format from?

the LOAD [FIPS code or State] command will start by downloading precinct shapes and integrating in census data and what not from census.gov. you'll see that in [user home]/autoredistrict_data. then it marks it as done so next time it can see it already did all that and just open the files that are there.

I initially had geojson support but i removed it 'cause the files were just too big, and i realized i was kind of ignoring it anyways in lieu of ESRI shapefiles.

for geographic data format, you can

save it back to the shapefile's .dbf file via "SAVE". this is essentially a precinct-assignment file.
or you can do "EXPORT BLOCKS" which will create a block-assignment file (a comma-separated values file of {geoid of block,district})

nothing creates polygons that outline it (well, except for the export stats or export national, which produces rasters (.png)) - it's either precinct assignment or block assignments. you'd have to combine that with the precinct shapes or block shapes to get polygons.

your current choices are block assignments or precinct assignments.

the other export commands give you stats and pictures:

EXPORT STATS exports html stats files and map images
EXPORT PIE exports pie charts
EXPORT NATIONAL gives you transparent map images, scaled to fit on a national map, so you can superimpose multiple states.

these will export to: [user home]/autoredistrict_data/[state]/[census_year]/[district column]

On Wed, Jun 22, 2016 at 2:44 PM, Dane Summers notifications@github.com wrote:

Wow, thank you! You are correct, there is no gerrymandering built into that bill - and currently it isn't law. Our group hopes to build a platform that conforms to S1093 and hold a redistricting competition in the Spring to show that such a system is possible, and garner support around it (I'm new to the project and a programmer, not a policy/GIS person, so my knowledge around these particular aspects are fuzzy at best!).

I'd just worked out how to manually download the NC shapefiles and start the algorithm; this looks much more complete than what I'd come up with so far:

SET EVOLUTION ELITE_FRAC 0.33 SET EVOLUTION ELITE_MUTATE_FRAC 1.0 SET WEIGHT DESCRIPTIVE 0.0 OPEN "../Documents/classes/autoredistrict/nc/tl_2012_37_vtd10.shp" SET ETHNICITY COLUMNS SET ELECTION COLUMNS SET POPULATION COLUMN POPULATION SET COUNTY COLUMN COUNTY_NAM SET WEIGHT COUNT_SPLITS TRUE SET ETHNICITY COLUMNS SET ELECTION COLUMNS SET ETHNICITY COLUMNS VAP_WHITE VAP_BLACK VAP_HISPAN VAP_ASIAN VAP_HAWAII VAP_INDIAN VAP_MULTI VAP_OTHER SET ELECTION COLUMNS PRES12_DEM PRES12_REP SET DISTRICTS NUM_DISTRICTS 13 SET CONSTRAIN CONTIGUITY TRUE SET CONSTRAIN POPULATION TRUE GO

I'm running your version now...clearly automation is working!

Ultimately we want to present the result of a run on a map on a website; do the export * commands provide data that I could derive a geojson format from?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/happyjack27/autoredistrict/issues/154#issuecomment-227855128, or mute the thread https://github.com/notifications/unsubscribe/AB7NQdlpVEEnkNlig73QAaNDGTGmRx-Tks5qOZCFgaJpZM4I7LQf .

here's a sample result for north carolina congressional districts, created with auto-redistrict, compared to the current districts:

http://autoredistrict.org/all50/version3/CD_PRES/single_state_embeddable_compare2.php?source=CD_SM&source2=CD_2010&mode=map_districts&unit=district&count=%2E&state=North%20Carolina

it was designed to minimize gerrymandering using the 2012 presidential vote counts for the PVI of each precinct.

Note: with assembly districts, it'll be tougher to get equal population, because each district will be composed of fewer precincts, and the precincts vary drastically in population. nonetheless, any population variance will be minor enough that it can be fixed by post-processing with some block-level program and splitting a precinct or two. but imo the precincts shouldn't vary so much in population originally.

On Wed, Jun 22, 2016 at 3:13 PM, Kevin Baas happyjack27@gmail.com wrote:

the LOAD [FIPS code or State] command will start by downloading precinct shapes and integrating in census data and what not from census.gov. you'll see that in [user home]/autoredistrict_data. then it marks it as done so next time it can see it already did all that and just open the files that are there.

I initially had geojson support but i removed it 'cause the files were just too big, and i realized i was kind of ignoring it anyways in lieu of ESRI shapefiles.

for geographic data format, you can

save it back to the shapefile's .dbf file via "SAVE". this is essentially a precinct-assignment file.

or you can do "EXPORT BLOCKS" which will create a block-assignment file (a comma-separated values file of {geoid of block,district})

nothing creates polygons that outline it (well, except for the export stats or export national, which produces rasters (.png)) - it's either precinct assignment or block assignments. you'd have to combine that with the precinct shapes or block shapes to get polygons.

your current choices are block assignments or precinct assignments.

the other export commands give you stats and pictures:

EXPORT STATS exports html stats files and map images

EXPORT PIE exports pie charts

EXPORT NATIONAL gives you transparent map images, scaled to fit on a national map, so you can superimpose multiple states.

these will export to: [user home]/autoredistrict_data/[state]/[census_year]/[district column]

On Wed, Jun 22, 2016 at 2:44 PM, Dane Summers notifications@github.com wrote:

Wow, thank you! You are correct, there is no gerrymandering built into that bill - and currently it isn't law. Our group hopes to build a platform that conforms to S1093 and hold a redistricting competition in the Spring to show that such a system is possible, and garner support around it (I'm new to the project and a programmer, not a policy/GIS person, so my knowledge around these particular aspects are fuzzy at best!).

I'd just worked out how to manually download the NC shapefiles and start the algorithm; this looks much more complete than what I'd come up with so far:

SET EVOLUTION ELITE_FRAC 0.33 SET EVOLUTION ELITE_MUTATE_FRAC 1.0 SET WEIGHT DESCRIPTIVE 0.0 OPEN "../Documents/classes/autoredistrict/nc/tl_2012_37_vtd10.shp" SET ETHNICITY COLUMNS SET ELECTION COLUMNS SET POPULATION COLUMN POPULATION SET COUNTY COLUMN COUNTY_NAM SET WEIGHT COUNT_SPLITS TRUE SET ETHNICITY COLUMNS SET ELECTION COLUMNS SET ETHNICITY COLUMNS VAP_WHITE VAP_BLACK VAP_HISPAN VAP_ASIAN VAP_HAWAII VAP_INDIAN VAP_MULTI VAP_OTHER SET ELECTION COLUMNS PRES12_DEM PRES12_REP SET DISTRICTS NUM_DISTRICTS 13 SET CONSTRAIN CONTIGUITY TRUE SET CONSTRAIN POPULATION TRUE GO

I'm running your version now...clearly automation is working!

Ultimately we want to present the result of a run on a map on a website; do the export * commands provide data that I could derive a geojson format from?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/happyjack27/autoredistrict/issues/154#issuecomment-227855128, or mute the thread https://github.com/notifications/unsubscribe/AB7NQdlpVEEnkNlig73QAaNDGTGmRx-Tks5qOZCFgaJpZM4I7LQf .

oh and forgot one more thing: checking the "constrain" boxes will make each iteration take longer. from my experience these are more for final tweaking,

On Wed, Jun 22, 2016 at 3:22 PM, Kevin Baas happyjack27@gmail.com wrote:

here's a sample result for north carolina congressional districts, created with auto-redistrict, compared to the current districts:

http://autoredistrict.org/all50/version3/CD_PRES/single_state_embeddable_compare2.php?source=CD_SM&source2=CD_2010&mode=map_districts&unit=district&count=%2E&state=North%20Carolina

it was designed to minimize gerrymandering using the 2012 presidential vote counts for the PVI of each precinct.

Note: with assembly districts, it'll be tougher to get equal population, because each district will be composed of fewer precincts, and the precincts vary drastically in population. nonetheless, any population variance will be minor enough that it can be fixed by post-processing with some block-level program and splitting a precinct or two. but imo the precincts shouldn't vary so much in population originally.

On Wed, Jun 22, 2016 at 3:13 PM, Kevin Baas happyjack27@gmail.com wrote:

the LOAD [FIPS code or State] command will start by downloading precinct shapes and integrating in census data and what not from census.gov. you'll see that in [user home]/autoredistrict_data. then it marks it as done so next time it can see it already did all that and just open the files that are there.

I initially had geojson support but i removed it 'cause the files were just too big, and i realized i was kind of ignoring it anyways in lieu of ESRI shapefiles.

for geographic data format, you can

save it back to the shapefile's .dbf file via "SAVE". this is essentially a precinct-assignment file.

or you can do "EXPORT BLOCKS" which will create a block-assignment file (a comma-separated values file of {geoid of block,district})

nothing creates polygons that outline it (well, except for the export stats or export national, which produces rasters (.png)) - it's either precinct assignment or block assignments. you'd have to combine that with the precinct shapes or block shapes to get polygons.

your current choices are block assignments or precinct assignments.

the other export commands give you stats and pictures:

EXPORT STATS exports html stats files and map images

EXPORT PIE exports pie charts

EXPORT NATIONAL gives you transparent map images, scaled to fit on a national map, so you can superimpose multiple states.

these will export to: [user home]/autoredistrict_data/[state]/[census_year]/[district column]

On Wed, Jun 22, 2016 at 2:44 PM, Dane Summers notifications@github.com wrote:

Wow, thank you! You are correct, there is no gerrymandering built into that bill - and currently it isn't law. Our group hopes to build a platform that conforms to S1093 and hold a redistricting competition in the Spring to show that such a system is possible, and garner support around it (I'm new to the project and a programmer, not a policy/GIS person, so my knowledge around these particular aspects are fuzzy at best!).

I'd just worked out how to manually download the NC shapefiles and start the algorithm; this looks much more complete than what I'd come up with so far:

SET EVOLUTION ELITE_FRAC 0.33 SET EVOLUTION ELITE_MUTATE_FRAC 1.0 SET WEIGHT DESCRIPTIVE 0.0 OPEN "../Documents/classes/autoredistrict/nc/tl_2012_37_vtd10.shp" SET ETHNICITY COLUMNS SET ELECTION COLUMNS SET POPULATION COLUMN POPULATION SET COUNTY COLUMN COUNTY_NAM SET WEIGHT COUNT_SPLITS TRUE SET ETHNICITY COLUMNS SET ELECTION COLUMNS SET ETHNICITY COLUMNS VAP_WHITE VAP_BLACK VAP_HISPAN VAP_ASIAN VAP_HAWAII VAP_INDIAN VAP_MULTI VAP_OTHER SET ELECTION COLUMNS PRES12_DEM PRES12_REP SET DISTRICTS NUM_DISTRICTS 13 SET CONSTRAIN CONTIGUITY TRUE SET CONSTRAIN POPULATION TRUE GO

I'm running your version now...clearly automation is working!

Ultimately we want to present the result of a run on a map on a website; do the export * commands provide data that I could derive a geojson format from?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/happyjack27/autoredistrict/issues/154#issuecomment-227855128, or mute the thread https://github.com/notifications/unsubscribe/AB7NQdlpVEEnkNlig73QAaNDGTGmRx-Tks5qOZCFgaJpZM4I7LQf .

"there is no gerrymandering built into that bill" actually, there is, it's right here:

(12) The total quality score for each proposed redistricting plan shall be computed as the sum of three component scores, each of which is chosen to achieve a particular goal. The three goals are: compactness, one person, one vote, and minimizing the number of split counties, municipalities, and precincts:

Because democratic voters cluster in cities, these rules produce a pro-GOP partisan gerrymander.

What I said was there are no ANTI-gerrymandering rules in the law.

Were that law passed, it would still be entirely legal to gerrymander, so long as the gerrymander was compact, which is really easy to do with a simple computer algorithm.

happyjack27 / autoredistrict

Add CLI Support #154