create WorldUnit module for creating and handling fiscalunits

[jschalk]

The process is that a folder has all fiscal information in it including bricks and fiscal info. Then run a function like "initialize_fiscal_system_full_rebuild". that

• [x] Create all the bricks necessary to create a minimal fiscalunit
• [ ] create all the bricks needed to create a fiscalunit with 2 person output
• [ ] create all the bricks needed for fiscalunit with 3 person output Every WorldUnit needs to be able to take data from a "jungle", put it in a "zoo", the zoo confirms that all the data is in a certain format and that there are no logically conflicting row, ie any set of rows with equivalent keys does not have conflicting values. Then the zoo data is put in the otx directory where it is divided by faces and eons. Eons are only used for the translation work. The face data is then moved to the inx folder. The inx data is also checked for logically conflicting rows. After that the fiscalunits are built.
• [x] create test that all brick formats have fiscal_id, face_id, eon_id
• [x] create brick format for explicit_label (otx_wall, inx_wall, unknown_word, otx_label, inx_label)
• [x] create brick format for otx_to_inx (otx_wall, inx_wall, unknown_word, otx_word, inx_word)
• [x] jungle_to_zoo validation : every sheet has brick required columns (must include eon_id, face_id
• [x] jungle_to_zoo_validator db - create db test
• [x] jungle_to_zoo_validator db - create table for each brick_format file
• [x] jungle_to_zoo_validator db - create column for face_id, eon_id
• [x] jungle_to_zoo_validator db - create column for each brick_format column
• [ ] inx_landing_inx_table_validator db - create db test
• [ ] inx_landing_inx_table_validator db - create table for each brick_format file
• [ ] inx_landing_inx_table_validator db - create column for each brick_format column
• [x] #358
• [x] create tool that takes all excel files in a directory and saves each tab as a csv (only tabs with brick format number in name of file name)
• [ ] pick names for the runs "run-all" "run-world-world"
• [ ] identify the output of a "run-world". Should be timestamp, otx_accts_net, otx_facts?, otx_pledges, otx_voices?
• [x] figure out how to map time between fiscal systems.
• [x] all worldunit operations must involve a timestamp "time_id" that describes when the context of each concept.
• [ ] buildup process: data starts in excel in "jungle"-> becomes csvs in "zoo", -> gets translated in "faces" -> gets moved into "fiscal"
• [ ] fiscalunit is rebuilt with budgets and voices.
• [x] #376
• [x] #377
• [x] figure out how otx_to_inx csv creation works.
• [ ] #381
• [x] #388

[jschalk]

• [x] create WorldUnit.get_dict functons to help clear up the metaspace. What is a worldunit?

[jschalk]

• [x] make face_id point to BridgeUnit
• [x] create save_face_id_bridgeunit_files
• [x] create init_face_id_bridgeunit loads BridgeUnit from face_id

[jschalk]

• [x] change "bridgeunit" to "filterunit", "bridgekind" to "bridgeunit"
• [x] put filter.py in bridge.py, change bridge.py to filter.py

[jschalk]

• [x] move pandas_tool to brick
• [x] move _save_otx_to_inx_csv to src/brick
• [x] move _save_explicit_label_csv to src/Brick
• [x] move _load_otx_to_inx_from_csv to src/brick
• [x] move _load_explicit_label_from_csv to src/brick
• [x] table_sorting_priority[47] == "otx_wall" change to str functions
• [x] try to move f09_filter to f08_fitler and f08_brick to f09_filter

[jschalk]

• [x] create test that grabbed excel file is valid brick (has all columns needed)

[jschalk]

• [ ] create test that db exists. copy test_FiscalUnit_get_journal_db_path_ReturnsCorrectObj
• [ ] refactor get_all_brick_dataframes so that instead of tuple you get BrickSrcUnit class
• [ ] figfure out if inserting can be automatic for each brick
• [ ] try to change src dir and filename to single "brick_path" variable
• [ ] figure out how to insert quickly.

[jschalk]

I've been beating my head against the wall for two days on getting dataframe information into sql database. I should have just gone with the custom string inserts that I control. Pandas tools were not the help I thought they could be.

• [x] get something working: get dataframes from source into zoo.

[jschalk]

• [x] make holder class for brick_dataframes file/sheet references