Closed fungjj92 closed 5 years ago
This also needs a management command, so it can run like this:
$ ./scripts/manage.sh beekeepers_export
+ ARGS=beekeepers_export
+ [[ 1 -eq 1 ]]
+ [[ beekeepers_export == runserver ]]
+ vagrant ssh app -c 'cd /opt/app && envdir /etc/icp.d/env ./manage.py beekeepers_export'
Export Complete
Connection to 127.0.0.1 closed.
I added a commit for that (since it was easier than writing it out in a comment).
Currently any new export overwrites the old files. Can we make it so that the exports are zipped into an archive with a timestamp, so that later exports do not overwrite the old ones?
Can we have the CSV export quote string and date fields? The current export looks like this:
id,inspection_date,colony_name,colony_alive,colony_loss_reason,num_bodies_supers_deep,num_bodies_supers_medium,num_bodies_supers_shallow,activity_since_last,queenright,same_queen,queen_stock,queen_source,varroa_count_performed,varroa_count_technique,varroa_count_result,varroa_treatment,survey_id
1,2018-10-09,A,t, ,0,0,3, ,f,YES, ,NON_LOCAL_COMMERCIAL,f, ,10, ,5
2,2018-11-12,A,t, ,0,0,3, ,f,YES, ,NON_LOCAL_COMMERCIAL,f, ,10, ,8
3,2018-12-27,A,t, ,0,0,3, ,f,YES, ,NON_LOCAL_COMMERCIAL,f, ,10,CHEMICAL_FORMIC_ACID_FORMIC_PRO;CHEMICAL_OXALIC_ACID_DRIBBLE;CHEMICAL_THYMOL_MENTHOL_APILIFE;CHEMICAL_SYNTHETIC_CHECKMITE_PLUS;MECHANICAL_QUEEN_MANIPULATION;OTHER-heat vision;CHEMICAL_ORGANIC_OTHER-freeze breath;MECHANICAL_OTHER-punching,7
4,2019-01-14,A,f,INADEQUETE_FOOD_STORES;COLONY_TOO_SMALL_IN_NOVEMBER,0,2,0,REMOVED_BROOD,f,YES,"",NON_LOCAL_COMMERCIAL,f,"",0,CHEMICAL_OXALIC_ACID_DRIBBLE;CHEMICAL_SYNTHETIC_CHECKMITE_PLUS;CHEMICAL_ORGANIC_OTHER-asdsadadas,21
5,2019-01-15,A,t,"",23,0,0,FED_POLLEN_PROTEIN,f,YES,"",NON_LOCAL_COMMERCIAL,f,"",0,CHEMICAL_OXALIC_ACID_VAPORIZATION;CHEMICAL_SYNTHETIC_APIVAR;CHEMICAL_ORGANIC_OTHER-asdsadadas,22
6,2019-01-16,A,t,"",0,0,0,"",f,YES,"",REQUEENED,f,"",0,CHEMICAL_THYMOL_MENTHOL_APILIFE;CHEMICAL_SYNTHETIC_APISTAN;MECHANICAL_QUEEN_MANIPULATION;CHEMICAL_ORGANIC_OTHER-asdsadadas,25
whereas it should look like this:
"id","inspection_date","colony_name","colony_alive","colony_loss_reason","num_bodies_supers_deep","num_bodies_supers_medium","num_bodies_supers_shallow","activity_since_last","queenright","same_queen","queen_stock","queen_source","varroa_count_performed","varroa_count_technique","varroa_count_result","varroa_treatment","survey_id"
1,"2018-10-09","A",TRUE," ",0,0,3," ",FALSE,"YES"," ","NON_LOCAL_COMMERCIAL",FALSE," ",10," ",5
2,"2018-11-12","A",TRUE," ",0,0,3," ",FALSE,"YES"," ","NON_LOCAL_COMMERCIAL",FALSE," ",10," ",8
3,"2018-12-27","A",TRUE," ",0,0,3," ",FALSE,"YES"," ","NON_LOCAL_COMMERCIAL",FALSE," ",10,"CHEMICAL_FORMIC_ACID_FORMIC_PRO;CHEMICAL_OXALIC_ACID_DRIBBLE;CHEMICAL_THYMOL_MENTHOL_APILIFE;CHEMICAL_SYNTHETIC_CHECKMITE_PLUS;MECHANICAL_QUEEN_MANIPULATION;OTHER-heat vision;CHEMICAL_ORGANIC_OTHER-freeze breath;MECHANICAL_OTHER-punching",7
4,"2019-01-14","A",FALSE,"INADEQUETE_FOOD_STORES;COLONY_TOO_SMALL_IN_NOVEMBER",0,2,0,"REMOVED_BROOD",FALSE,"YES","","NON_LOCAL_COMMERCIAL",FALSE,"",0,"CHEMICAL_OXALIC_ACID_DRIBBLE;CHEMICAL_SYNTHETIC_CHECKMITE_PLUS;CHEMICAL_ORGANIC_OTHER-asdsadadas",21
5,"2019-01-15","A",TRUE,"",23,0,0,"FED_POLLEN_PROTEIN",FALSE,"YES","","NON_LOCAL_COMMERCIAL",FALSE,"",0,"CHEMICAL_OXALIC_ACID_VAPORIZATION;CHEMICAL_SYNTHETIC_APIVAR;CHEMICAL_ORGANIC_OTHER-asdsadadas",22
6,"2019-01-16","A",TRUE,"",0,0,0,"",FALSE,"YES","","REQUEENED",FALSE,"",0,"CHEMICAL_THYMOL_MENTHOL_APILIFE;CHEMICAL_SYNTHETIC_APISTAN;MECHANICAL_QUEEN_MANIPULATION;CHEMICAL_ORGANIC_OTHER-asdsadadas",25
That'll protect from any commas in the content (which may be in the free-entry fields).
The good news is that postgres or psycopg
must add quotes when appropriate. Here is the output of april surveys when a string entry contains commas (last row).
survey_id,colony_loss_reason
55,varroa mites
60,varroa mites
63,INADEQUETE_FOOD_STORES;OTHER-super sad bee killer
64,VARROA_MITES;INADEQUETE_FOOD_STORES;OTHER-super sad bee killer
65,VARROA_MITES;OTHER-super sad bee killer2
66,COLONY_TOO_SMALL;PESTICIDE_EXPOSURE;OTHER-
67,VARROA_MITES;POOR_WEATHER_CONDITIONS;OTHER-
69,"VARROA_MITES;OTHER-super, sad, bee, killer"
Heads up that there's a merge conflict now.
Ready for a look
Taking a look now.
Thanks! Helpful clean up.
Non-staff user:
The button on web and on a screen reader:
Taking another look.
😅 Thanks for the reviews.
Overview
Site admins want the data as CSV to do analyses. This PR adds a view at http://localhost:8000/beekeepers/export for admins to download tables as a zip of CSVs. SQL queries are hardcoded.
Connects #384 Connects #385
Demo
The idea: (i fussed with the filename timestamp a little since this screenshot)
The date formatter doesn't allow colons in the time (only
-
). It's livable. It's main purpose is to give the files a unique ID.Non-admins can't access:
Testing Instructions
Visit the URL and sign in as an admin. Check that the zip downloads and the CSVs look good.