Error in running the code run_ogindia_simple_fast.py

[Abhinav1109]

@rickecon @jdebacker We have been trying to run the OG Model using the file run_ogindia_simple_fast.py which we created during the sessions om a different computer. However, we are encountering the error as per the attached screenshot. Could you please see what we are missing in this.
Thanks, Abhinav

run_ogindia_simple_fast.txt

[jdebacker]

@Abhinav1109 The issue you are having can be seen in the line that says KeyError: 'Age'. This is pointing towards some issue when the demographic data is read in from the csv files. On some machines, the encoding of these text data is not handled appropriately.

I had hoped to have addressed this in PR #12. Can you confirm that you are running the ogindia package with:

1. The most recent changes from the master branch of TPRU-India/OG-India and then
2. Building the ogindia package (via pip install -e . OR python setup.py develop run from the OG-India directory and in the ogindia-dev environment)?

If you are doing 1 and 2, let me know because I thought I had addressed this issue. If not, try to update your local files and then build the ogindia package again.

HOWEVER, if you want this to run, be sure to just do your git sync stuff to get the master changes and then build the new ogindia package. You do not want to delete and rebuild the ogindia-dev virtual environment because the new environment.yml file in the TPRU-India/OG-Inda master branch will install the India taxcalc package instead of the USA taxcalc package (and the India taxcalc will not work with the parameter names passed in the

reform = {
        'II_rt7': {2020: 0.7},
        'PT_rt7': {2020: 0.7}
    }

dictionary. This is all a little bit confusing as we move between two microsimulation models with the same name!

[Abhinav1109]

@jdebacker @rickecon @kumar-ab @sktpru @dc4tpru

Let me enumerate the steps we have taken in trying to run the package:

1) Cloned the repository to a directory on the computer

2) Navigated to the directory to which we had cloned this repository and run conda env create -f environment.yml

3) Activated the ogindia-dev environment using the command activate ogindia-dev on Windows. Further, installed the packages by running the command by pip install -e .

4) We have also modified the demographics.py file to reflect the changes shown in PR #12.

5) Navigated to ./run_examples

6) Run the model using the file run_ogindia_simple_fast.py

7) Since the line says KeyError: 'Age', we physically modified the file indian_pop_data.csv to remove that error. Its now done and we did not encounter errors on this count.

8) Then, we created another environment named ogindia-dev-tpru in which we have modified the taxcalc to point to the US Tax Calculator by modifying Line 24 of the file environment.yml to "git+https://github.com/PSLmodels/Tax-Calculator.git@master". This is based on your observation that "environment.yml file in the TPRU-India/OG-Inda master branch will install the India taxcalc package instead of the USA taxcalc package".

9) After this, we tried to run the file "run_ogindia_simple_fast.py" from which we got an error "could not read puf.csv data from egg". The screenshot is attached as below. This error is due to the non-availability of the file "puf.csv".

10) We are currently stuck at this step. It would be helpful if you can point us to the steps we may be skipping in getting this running.

12) Further, while trying to run the file "demographics.py", as per the attached/modified file, no output is generated even though the code executes. Which would be the correct file to execute this data to get the demographics.

[jdebacker]

@Abhinav1109 It looks like you are doing all the right steps.

I am not sure how your run_ogindia_simple_fast.py looks, but you should be able to avoid the need for the puf.csv file by passing the following in the key word arguments to runner(): 'data': 'cps', which will specify the use of the CPS survey data that is included in the USA taxcalc package.

Re your demographics question, I'm not sure of the answer. @rickecon was working on this and I may have merged an incomplete set of changes into the repo. We will work to try to get this straightened out.

[rickecon]

@Abhinav1109 and @jdebacker. Some background for Abhinav is the following. We came to TPRU on August 19 with a working copy of OG-India. However, it called on the U.S. version of the taxcalc package. We didn't realize that the Indian version had the same package name of taxcalc, which caused some problems after we integrated OG-India with the Indian taxcalc. I think the current working version of OG-India should use the Indian taxcalc so that we can have simultaneous development on both packages. So here is my proposal for moving forward.

1. Only use run_ogindia_*.py files that call on the Indian tax calculator with the 10,000-observation tax data. This will take some work to get it up and running. For example, PR 156 in the Indian taxcalc proposes to change the source data from the 200-observation to the 10,000-observation data in records.py.

2. Update the OG-India/run_examples/run_ogindia_example.py script so that it executes a simple policy reform that calls on the Indian taxcalc package.

3. After (1) and (2) are done, then I will update the demographics.py file in OG-India.

[jdebacker]

@rickecon FYI - you can choose what data file to use for the taxcalc package with the data and weights arguments to the Records() class. E.g., in OG-India, we specify taking the larger dataset here, although one could change this to pass through the datafile name from the key word arguments from execute.runner().

[Abhinav1109]

@jdebacker @rickecon @kumar-ab @sktpru @dc4tpru

As per the recommendation made by @rickecon , we are trying to run the runogindia*.py files that call on the Indian tax calculator. Further, we also ran a policy which is consistent with the Indian taxcalc package as defined in https://github.com/TPRU-India/taxcalc/blob/master/Budget2019_reform.json.

While running this code, we encountered the error as below:

From this, it is apparent that "GSTRecords.FIELD_VARS" is not being populated properly while executing the file "gstrecords.py". We tried to debug it unsuccessfully. Our microsimulation model seems to be running fine with the same set of files/data. Could you help regarding the same.

[rickecon]

I am getting a different error trying to run the run_ogindia_experiment.py file. The larger tax and weights datasets are referred to in lines 46 and 63 and lines 48-49 of get_micro_data.py. However, when I run the run_ogindia_experiment.py script, I get the following error because the pitSmallData.csv and pit_weightsSD.csv are not in the taxcalc package path.

(ogindia-dev) Richards-MBP-3:run_examples rwe$ python run_ogindia_experiment.py
Number of workers =  4
making dir:  ./OUTPUT_BASELINE/SS
making dir:  ./OUTPUT_BASELINE/TPI
In runner, baseline is  True
path for tax functions:  ./OUTPUT_BASELINE
BW =  6 begin year =  2018 end year =  2023
Traceback (most recent call last):
  File "run_ogindia_experiment.py", line 80, in <module>
    run_micro_macro(user_params={})
  File "run_ogindia_experiment.py", line 50, in run_micro_macro
    runner(**kwargs)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/execute.py", line 68, in runner
    spec.get_tax_function_parameters(client, run_micro)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/parameters.py", line 296, in get_tax_function_parameters
    tx_func_est_path, self.data, client, self.num_workers)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/txfunc.py", line 1628, in get_tax_func_estimate
    graph_data=graph_data, graph_est=graph_est)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/txfunc.py", line 1395, in tax_func_estimate
    client=client, num_workers=num_workers)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/get_micro_data.py", line 112, in get_data
    reform=reform, data=data)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/get_micro_data.py", line 49, in get_calculator
    weights='pit_weightsSD.csv')
  File "/anaconda/envs/ogindia-dev/lib/python3.7/site-packages/taxcalc/records.py", line 93, in __init__
    self._read_data(data)
  File "/anaconda/envs/ogindia-dev/lib/python3.7/site-packages/taxcalc/records.py", line 272, in _read_data
    raise ValueError(msg)
ValueError: file /anaconda/envs/ogindia-dev/lib/python3.7/site-packages/taxcalc/pitSmallData.csv cannot be found

For this reason, I want to make the new files the default in the Indian taxcalc. However, in writing this down. It is not clear to me that changing the default data will solve my problem.

[rickecon]

I just removed all my conda environments and packages and uninstalled Anaconda and Conda. I reinstalled Anaconda and Conda. Then I just did the following steps from my local OG-India directory.

conda env create -f environment.yml
conda activate ogindia-dev
pip install -e .

Then I went to the /run_examples/ directory and ran the following script.

python run_ogindia_experiments.py

I am still getting the same error from the pitSmallData.csv and pit_weightsSD.csv files not being installed with my Indian taxcalc package.

(ogindia-dev) Richards-MBP-3:run_examples rwe$ python run_ogindia_experiment.py
Number of workers =  4
making dir:  ./OUTPUT_BASELINE/SS
making dir:  ./OUTPUT_BASELINE/TPI
In runner, baseline is  True
age_specific  has been updated to:  False
tax_func_type  has been updated to:  linear
path for tax functions:  ./OUTPUT_BASELINE
BW =  5 begin year =  2019 end year =  2023
Traceback (most recent call last):
  File "run_ogindia_experiment.py", line 52, in <module>
    runner(**kwargs)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/execute.py", line 68, in runner
    spec.get_tax_function_parameters(client, run_micro)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/parameters.py", line 296, in get_tax_function_parameters
    tx_func_est_path, self.data, client, self.num_workers)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/txfunc.py", line 1628, in get_tax_func_estimate
    graph_data=graph_data, graph_est=graph_est)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/txfunc.py", line 1395, in tax_func_estimate
    client=client, num_workers=num_workers)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/get_micro_data.py", line 111, in get_data
    reform=reform, data=data)
  File "/Users/rwe/Documents/Economics/OSE/OG-India/ogindia/get_micro_data.py", line 48, in get_calculator
    records1 = Records(data=data, weights=weights)
  File "/Users/rwe/anaconda3/envs/ogindia-dev/lib/python3.7/site-packages/taxcalc/records.py", line 93, in __init__
    self._read_data(data)
  File "/Users/rwe/anaconda3/envs/ogindia-dev/lib/python3.7/site-packages/taxcalc/records.py", line 272, in _read_data
    raise ValueError(msg)
ValueError: file /Users/rwe/anaconda3/envs/ogindia-dev/lib/python3.7/site-packages/taxcalc/pitSmallData.csv cannot be found

Any ideas why these two new data files are not uploading with the taxcalc package.

[rickecon]

In looking through the OG-India files, I noticed that environment.yml has no reference to taxcalc as a dependency, requirements.txt has no reference to taxcalc as a dependency, and setup.py has no reference to taxcalc. The only reference I found to taxcalc was in `/conda.recipe/meta.yaml in lines 11 and 20.

requirements:
  build:
    - python
    - "taxcalc>=2.4.2"
    - setuptools
    - scipy
    - matplotlib
    - dask
    - mkl

  run:
    - python
    - "taxcalc>=2.4.2"
    - setuptools
    - scipy
    - matplotlib
    - dask
    - mkl

Note that the build and run requirements in meta.yaml require taxcalc>=2.4.2. I think the Indian taxcalc is version 0.0.0. Any ideas? How is this ogindia-dev environment downloading the Indian taxcalc package?

[rickecon]

OK. I see that the last two lines of environment.yml have the following instruction to download Indian taxcalc.

- pip:
    - git+https://github.com/TPRU-India/taxcalc.git@master

Why is this not downloading all the files in the Indian taxcalc repo?

[jdebacker]

@rickecon environment.yml should download and install taxcalc via:

- pip:
    - git+https://github.com/TPRU-India/taxcalc.git@master

I'm just not sure if this is somehow missing those data files (it sounds like it is from what you describe - the package is build, it just can't find those files). But it looks like they are in the folder the package is created from. I will say that I've only used this method to build a package once before, so I may be missing something. For the India taxcalc package, I've just installed it manually from the source code in my cloned repo.

The meta.yaml file is only for creating a conda package like those distributed on the PSL Anaconda channel, I think. The setup.py is needed for creating a local package, but it's not necessary to list dependencies in that file.

[rickecon]

@jdebacker. OK, so I think those last two lines in the environment.yml are right. I am going to look in the Indian taxcalc to see if there is something there that is making those two new data files not upload with the package.

[rickecon]

I think the problem is the version of Indian taxcalc which is currently 2.0.0. I think the package information refers to version 0.0.0. The new data files were uploaded after version 2.0.0. Let me see if I can make some changes to fix this.

By the way, Indian taxcalc should not be at version 2.0.0. We'll have to roll that back somehow.

[rickecon]

I just submitted a PR to the Indian taxcalc repository that updates the version of taxcalc that gets automatically installed in the ogindia-dev environment. I think this will fix the issue.

[sebastiansajie]

@rickecon @jdebacker I have been having the same issue "pitSmallData.csv not found". Looking forward to finding a fix.

[jdebacker]

One fix to this problem is to just build the India taxcalc package manually. To do this:

1) (If you haven't already) clone the taxcalc repo 2) Put yourself in the virtual environment where you have ogindia installed (e.g., ogindia-dev). 3) from the \taxcalc directory, run pip install -e . OR python setup.py develop. 4) Run ogindia as you would before.

I'll look more into why the pip install from the GiHub repo is not including that datafile. But in the meantime, you can build your taxcalc package from the source code locally and that should point you to the datafile (which will have been in the cloned repo). This has been working for me.

[jdebacker]

FYI, this issue is waiting for PR #159 to be merged into the TPRU-India/taxcalc repo.

[jdebacker]

After @sebastiansajie has merged PR #159 in the taxcalc repo, I've tested running the example script in OG-India/run_examples. Everything works as expected. Here's what I did:

(base) A02655JDEBACKER:run_examples jason.debacker$ cd repos
(base) A02655JDEBACKER:run_examples jason.debacker$ mkdir test_OGIndia
(base) A02655JDEBACKER:run_examples jason.debacker$ cd test_OGIndia
(base) A02655JDEBACKER:run_examples jason.debacker$ git clone https://github.com/TPRU-India/OG-India.git
(base) A02655JDEBACKER:run_examples jason.debacker$ cd OG-India
(base) A02655JDEBACKER:run_examples jason.debacker$ conda env create
(base) A02655JDEBACKER:run_examples jason.debacker$ conda activate ogindia-dev
remote: Counting objects: 100% (1/1), done.
remote: Total 441 (delta 0), reused 0 (delta 0), pack-reused 440
Receiving objects: 100% (441/441), 125.56 MiB | 4.86 MiB/s, done.
Resolving deltas: 100% (169/169), done.
Checking out files: 100% (188/188), done.
(base) A02655JDEBACKER:test_OGIndia jason.debacker$ cd OG-India
(base) A02655JDEBACKER:OG-India jason.debacker$ conda env create
Collecting package metadata (repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
  current version: 4.7.11
  latest version: 4.7.12

Please update conda by running

    $ conda update -n base conda

Downloading and Extracting Packages
pillow-6.2.0         | 592 KB    | ##################################### | 100%
attrs-19.2.0         | 35 KB     | ##################################### | 100%
sqlite-3.30.0        | 2.5 MB    | ##################################### | 100%
msgpack-python-0.6.2 | 83 KB     | ##################################### | 100%
cloudpickle-1.2.2    | 23 KB     | ##################################### | 100%
dask-2.5.2           | 4 KB      | ##################################### | 100%
libblas-3.8.0        | 10 KB     | ##################################### | 100%
setuptools-41.4.0    | 621 KB    | ##################################### | 100%
scipy-1.3.1          | 18.3 MB   | ##################################### | 100%
pexpect-4.7.0        | 78 KB     | ##################################### | 100%
pylint-2.4.2         | 423 KB    | ##################################### | 100%
jedi-0.15.1          | 706 KB    | ##################################### | 100%
traitlets-4.3.3      | 133 KB    | ##################################### | 100%
heapdict-1.0.1       | 7 KB      | ##################################### | 100%
llvm-openmp-9.0.0    | 266 KB    | ##################################### | 100%
matplotlib-3.1.1     | 6 KB      | ##################################### | 100%
pytz-2019.3          | 237 KB    | ##################################### | 100%
pytest-5.2.1         | 354 KB    | ##################################### | 100%
psutil-5.6.3         | 329 KB    | ##################################### | 100%
appnope-0.1.0        | 8 KB      | ##################################### | 100%
distributed-2.5.2    | 380 KB    | ##################################### | 100%
fsspec-0.5.2         | 43 KB     | ##################################### | 100%
jinja2-2.10.3        | 91 KB     | ##################################### | 100%
prompt_toolkit-2.0.1 | 221 KB    | ##################################### | 100%
astroid-2.3.1        | 280 KB    | ##################################### | 100%
dask-core-2.5.2      | 574 KB    | ##################################### | 100%
ipython-7.8.0        | 1.1 MB    | ##################################### | 100%
pickleshare-0.7.5    | 12 KB     | ##################################### | 100%
asn1crypto-1.0.1     | 157 KB    | ##################################### | 100%
liblapack-3.8.0      | 10 KB     | ##################################### | 100%
coverage-4.5.4       | 217 KB    | ##################################### | 100%
cytoolz-0.10.0       | 364 KB    | ##################################### | 100%
libcblas-3.8.0       | 10 KB     | ##################################### | 100%
pytest-xdist-1.30.0  | 28 KB     | ##################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Ran pip subprocess with arguments:
['/anaconda3/envs/ogindia-dev/bin/python', '-m', 'pip', 'install', '-U', '-r', '/Users/jason.debacker/repos/test_OGIndia/OG-India/condaenv.thyevwyf.requirements.txt']
Pip subprocess output:
Collecting git+https://github.com/TPRU-India/taxcalc.git@master (from -r /Users/jason.debacker/repos/test_OGIndia/OG-India/condaenv.thyevwyf.requirements.txt (line 1))
  Cloning https://github.com/TPRU-India/taxcalc.git (to revision master) to /private/var/folders/b0/wwxd0byd1hx0y_rqrnqhmc0m0000gn/T/pip-req-build-9yhxpnz1
Requirement already satisfied, skipping upgrade: numpy in /anaconda3/envs/ogindia-dev/lib/python3.6/site-packages (from taxcalc==0.0.0->-r /Users/jason.debacker/repos/test_OGIndia/OG-India/condaenv.thyevwyf.requirements.txt (line 1)) (1.17.2)
Requirement already satisfied, skipping upgrade: pandas in /anaconda3/envs/ogindia-dev/lib/python3.6/site-packages (from taxcalc==0.0.0->-r /Users/jason.debacker/repos/test_OGIndia/OG-India/condaenv.thyevwyf.requirements.txt (line 1)) (0.25.1)
Requirement already satisfied, skipping upgrade: numba in /anaconda3/envs/ogindia-dev/lib/python3.6/site-packages (from taxcalc==0.0.0->-r /Users/jason.debacker/repos/test_OGIndia/OG-India/condaenv.thyevwyf.requirements.txt (line 1)) (0.45.1)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.6.1 in /anaconda3/envs/ogindia-dev/lib/python3.6/site-packages (from pandas->taxcalc==0.0.0->-r /Users/jason.debacker/repos/test_OGIndia/OG-India/condaenv.thyevwyf.requirements.txt (line 1)) (2.8.0)
Requirement already satisfied, skipping upgrade: pytz>=2017.2 in /anaconda3/envs/ogindia-dev/lib/python3.6/site-packages (from pandas->taxcalc==0.0.0->-r /Users/jason.debacker/repos/test_OGIndia/OG-India/condaenv.thyevwyf.requirements.txt (line 1)) (2019.3)
Requirement already satisfied, skipping upgrade: llvmlite>=0.29.0dev0 in /anaconda3/envs/ogindia-dev/lib/python3.6/site-packages (from numba->taxcalc==0.0.0->-r /Users/jason.debacker/repos/test_OGIndia/OG-India/condaenv.thyevwyf.requirements.txt (line 1)) (0.29.0)
Requirement already satisfied, skipping upgrade: six>=1.5 in /anaconda3/envs/ogindia-dev/lib/python3.6/site-packages (from python-dateutil>=2.6.1->pandas->taxcalc==0.0.0->-r /Users/jason.debacker/repos/test_OGIndia/OG-India/condaenv.thyevwyf.requirements.txt (line 1)) (1.12.0)
Building wheels for collected packages: taxcalc
  Building wheel for taxcalc (setup.py): started
  Building wheel for taxcalc (setup.py): finished with status 'done'
  Created wheel for taxcalc: filename=taxcalc-0.0.0-cp36-none-any.whl size=17809922 sha256=e12ba0ea964052d2ba2b823f1c7291889d71d0a45416e49ab7f8e6bc374f3a7c
  Stored in directory: /private/var/folders/b0/wwxd0byd1hx0y_rqrnqhmc0m0000gn/T/pip-ephem-wheel-cache-w1mgb2ha/wheels/23/77/e6/63979a3b8f8d0e7f48ef5021c6a584b0d97ccc2dc26b5e77d9
Successfully built taxcalc
Installing collected packages: taxcalc
Successfully installed taxcalc-0.0.0

#
# To activate this environment, use
#
#     $ conda activate ogindia-dev
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) A02655JDEBACKER:OG-India jason.debacker$ conda activate ogindia-dev
(ogindia-dev) A02655JDEBACKER:OG-India jason.debacker$ pip install -e .
Obtaining file:///Users/jason.debacker/repos/test_OGIndia/OG-India
Installing collected packages: ogindia
  Running setup.py develop for ogindia
Successfully installed ogindia
(ogindia-dev) A02655JDEBACKER:OG-India jason.debacker$ ls
Icon?           docs            ogindia.egg-info
LICENSE.md      environment.yml     regression
MANIFEST.in     gitpr           requirements.txt
Makefile        gitpr.bat       run_examples
README.md       gitsync         setup.py
calibration     gitsync.bat
conda.recipe        ogindia
(ogindia-dev) A02655JDEBACKER:OG-India jason.debacker$ cd run_examples
(ogindia-dev) A02655JDEBACKER:run_examples jason.debacker$ ls
run_ogindia_experiment.py
(ogindia-dev) A02655JDEBACKER:run_examples jason.debacker$ python run_ogindia_experiment.py
Number of workers =  4
making dir:  ./OUTPUT_BASELINE/SS
making dir:  ./OUTPUT_BASELINE/TPI
In runner, baseline is  True
age_specific  has been updated to:  False
tax_func_type  has been updated to:  linear
path for tax functions:  ./OUTPUT_BASELINE
BW =  5 begin year =  2019 end year =  2023
Running current law policy baseline
year:  2023
year:  2022
year:  2021
year:  2020
year= 2022 Age= all ages
year= 2019 Age= all ages
year= 2021 Age= all ages
year= 2020 Age= all ages
year= 2023 Age= all ages
Finished tax function loop through 5 years and 1 ages per year.
Tax function estimation time: 11.116 sec
Using baseline tax parameters from  ./OUTPUT_BASELINE/TxFuncEst_baseline_TPRU_19232019.pkl
pickle path exists
...

...cutting off a bunch of output with warnings due to poor starting values...

...
GE loop errors =  -0.02438503665399344 [0.01293691 0.01982983 0.02457223 0.01601295 0.01977459 0.02382384
 0.00496817] 0.017564950881351307 -4.214613596695397
GE loop errors =  -0.02438417211603463 [0.01293447 0.01983211 0.02457479 0.01600998 0.01977079 0.02381923
 0.00496891] 0.017564555499285525 -4.214493672192597
GE loop errors =  -0.02438343000365523 [0.01293312 0.01983308 0.02457587 0.01600835 0.01976871 0.02381672
 0.00496923] 0.017564167372556838 -4.214379285208071
GE loop errors =  -0.024381324619353537 [0.01293022 0.01983471 0.02457764 0.01600482 0.01976422 0.02381129
 0.0049698 ] 0.017563027571543426 -4.214045795629855
GE loop errors =  -0.024375940363582843 [0.01292399 0.01983738 0.02458043 0.01599724 0.0197546  0.02379965
 0.0049708 ] 0.017560050222608287 -4.213178328309521
GE loop errors =  -0.02436294788340754 [0.01291071 0.01984156 0.0245846  0.01598109 0.01973414 0.02377489
 0.00497251] 0.017552760632794588 -4.211060074415755
GE loop errors =  -0.024332698378331765 [0.0128826  0.01984772 0.02459023 0.01594689 0.01969086 0.02372256
 0.00497538] 0.01753563580391613 -4.206089916710641
GE loop errors =  -0.024263845311860346 [0.01282348 0.01985593 0.02459639 0.01587495 0.01959993 0.02361258
 0.00498011] 0.017496550994471956 -4.194738710077097
GE loop errors =  -0.024109035031302517 [0.01269997 0.0198649  0.0245995  0.01572461 0.01941007 0.02338299
 0.00498772] 0.01740926904545627 -4.1692886758358645
GE loop errors =  -0.02376118195033144 [0.01244405 0.01987038 0.02459036 0.01541311 0.019017   0.02290765
 0.00499986] 0.017217843690481897 -4.112878331081717
GE loop errors =  -0.022966527159414205 [0.01192062 0.01986549 0.02455261 0.01477612 0.01821355 0.02193589
 0.00502021] 0.016805169311234228 -3.9883002031405446
GE loop errors =  -0.021069313434798367 [0.01087363 0.01985807 0.02448059 0.01350283 0.01660747 0.01999282
 0.00506155] 0.015932916563007278 -3.711195190014176
GE loop errors =  -0.016146339745058514 [0.0088607  0.01996685 0.02449105 0.01105832 0.01352136 0.01625789
 0.00517552] 0.014138081914115738 -3.082295745943534
GE loop errors =  -0.00245955336306777 [0.0053628  0.02095351 0.02544138 0.0068016  0.00813014 0.00977268
 0.00561026] 0.010669217053573734 -1.6791862810763853
GE loop errors =  -0.0132609577497836 [0.01846247 0.02807608 0.03487379 0.02284195 0.02818441 0.0336878
 0.00692878] 0.020510989925298536 -4.397820076080719
GE loop errors =  -0.0024595534736473485 [0.0053628  0.02095351 0.02544138 0.0068016  0.00813014 0.00977268
 0.00561026] 0.01066921704399149 -1.679186277953526
GE loop errors =  -0.002459553365265401 [0.0053628  0.02095351 0.02544138 0.0068016  0.00813014 0.00977268
 0.00561026] 0.01066921705364949 -1.679186281093744
GE loop errors =  -0.0024595533654134492 [0.0053628  0.02095351 0.02544138 0.0068016  0.00813014 0.00977268
 0.00561026] 0.010669217053631944 -1.6791862810863507
GE loop errors =  -0.0024595533658607442 [0.0053628  0.02095351 0.02544138 0.0068016  0.00813014 0.00977268
 0.00561026] 0.010669217053635754 -1.6791862810854967
GE loop errors =  -0.00245955336544609 [0.0053628  0.02095351 0.02544138 0.0068016  0.00813014 0.00977268
 0.00561026] 0.010669217053638862 -1.679186281088798
GE loop errors =  -0.002459553366108331 [0.0053628  0.02095351 0.02544138 0.0068016  0.00813014 0.00977268
 0.00561026] 0.010669217053654905 -1.6791862810914622
GE loop errors =  -0.0024595533667726467 [0.0053628  0.02095351 0.02544138 0.0068016  0.00813014 0.00977268
 0.00561026] 0.010669217053653438 -1.6791862810874991
GE loop errors =  -0.0024595533636706836 [0.0053628  0.02095351 0.02544138 0.0068016  0.00813014 0.00977268
 0.00561026] 0.01066921705357414 -1.6791862810734441
GE loop errors =  -0.0024595523506123172 [0.0053628  0.02095351 0.02544138 0.0068016  0.00813014 0.00977268
 0.00561026] 0.010669216959182968 -1.6791862385512952
GE loop errors =  -0.002459553363067673 [0.0053628  0.02095351 0.02544138 0.0068016  0.00813014 0.00977268
 0.00561026] 0.010669217053573727 -1.6791863541492957
GE loop errors =  0.035544849198103586 [0.00527551 0.01913618 0.02315342 0.0065434  0.00783586 0.00952119
 0.00512652] 0.006224965948063347 -0.3564584800857036
GE loop errors =  -0.002393650126462074 [0.00632796 0.01745782 0.02099582 0.0075357  0.00908869 0.01124337
 0.00467455] 0.005790401197969039 -0.20458050497806113
GE loop errors =  -0.03293162055530126 [0.01064476 0.02290333 0.02744037 0.01243657 0.01509812 0.01884101
 0.00610743] 0.008309024226307409 -1.0984143207318313
GE loop errors =  -0.00922003585653279 [0.00624534 0.01579063 0.01889788 0.007252   0.00875194 0.01090005
 0.00423913] 0.005028593351761581 -0.35043312815067384
GE loop errors =  -0.0023936518258872025 [0.00632796 0.01745782 0.02099582 0.0075357  0.00908869 0.01124337
 0.00467455] 0.005790401228917046 -0.20458051699875313
GE loop errors =  -0.002393650136164674 [0.00632796 0.01745782 0.02099582 0.0075357  0.00908869 0.01124337
 0.00467455] 0.005790401198191682 -0.20458050502483527
GE loop errors =  -0.002393650128559646 [0.00632796 0.01745782 0.02099582 0.0075357  0.00908869 0.01124337
 0.00467455] 0.0057904011980055865 -0.2045805049823075
GE loop errors =  -0.0023936501301580343 [0.00632796 0.01745782 0.02099582 0.0075357  0.00908869 0.01124337
 0.00467455] 0.005790401198027394 -0.20458050498249225
GE loop errors =  -0.0023936501354767242 [0.00632796 0.01745782 0.02099582 0.0075357  0.00908869 0.01124337
 0.00467455] 0.0057904011981308005 -0.20458050499869174
GE loop errors =  -0.002393650137830619 [0.00632796 0.01745782 0.02099582 0.0075357  0.00908869 0.01124337
 0.00467455] 0.005790401198169041 -0.2045805050020597
GE loop errors =  -0.0023936501417517603 [0.00632796 0.01745782 0.02099582 0.0075357  0.00908869 0.01124337
 0.00467455] 0.005790401198195939 -0.20458050498898572
GE loop errors =  -0.002393650126667299 [0.00632796 0.01745782 0.02099582 0.0075357  0.00908869 0.01124337
 0.00467455] 0.005790401197970252 -0.20458050497727953
GE loop errors =  -0.0023936477827561503 [0.00632796 0.01745782 0.02099582 0.0075357  0.00908869 0.01124337
 0.00467455] 0.005790401034519083 -0.20458046811257358
GE loop errors =  -0.002393650126462074 [0.00632796 0.01745782 0.02099582 0.0075357  0.00908869 0.01124337
 0.00467455] 0.005790401197969039 -0.2045805813413457
GE loop errors =  0.011113359666737224 [0.00596999 0.01637913 0.01970642 0.0071096  0.00857691 0.0105866
 0.00437344] 0.004588870118081432 -0.04954769972835393
GE loop errors =  0.009821763713621576 [0.00526692 0.01423638 0.01711006 0.00622813 0.00751252 0.00927619
 0.0038008 ] 0.003822202617337291 -0.01940063142645343
GE loop errors =  0.0010610547776035845 [0.00403011 0.01019794 0.01218839 0.00459011 0.00553197 0.00684104
 0.00272559] 0.002704517736529497 -0.08584928774255118
GE loop errors =  0.011987063971860681 [0.00425413 0.01178733 0.01419518 0.00510187 0.0061559  0.00759217
 0.00314372] 0.003081851391797395 0.012196729833415354
GE loop errors =  -0.0020219327750063826 [0.00329432 0.00807108 0.00961894 0.00367962 0.004433   0.00548493
 0.00215811] 0.0021781814052260744 -0.05419360637333526
GE loop errors =  0.012678785077570143 [0.00318621 0.00929781 0.01123917 0.00393615 0.00475183 0.0058529
 0.0024777 ] 0.002460609976412719 0.018352468081563877
GE loop errors =  0.011987061075926425 [0.00425413 0.01178733 0.01419518 0.00510187 0.0061559  0.00759217
 0.00314372] 0.0030818514553216016 0.012196713343301191
GE loop errors =  0.011987063945099921 [0.00425413 0.01178733 0.01419518 0.00510187 0.0061559  0.00759217
 0.00314372] 0.003081851392345958 0.012196729798390926
GE loop errors =  0.011987063952134128 [0.00425413 0.01178733 0.01419518 0.00510187 0.0061559  0.00759217
 0.00314372] 0.003081851392122331 0.012196729842968601
GE loop errors =  0.011987063951632612 [0.00425413 0.01178733 0.01419518 0.00510187 0.0061559  0.00759217
 0.00314372] 0.0030818513921039674 0.012196729855009636
GE loop errors =  0.01198706394731923 [0.00425413 0.01178733 0.01419518 0.00510187 0.0061559  0.00759217
 0.00314372] 0.0030818513921966155 0.012196729847516963
GE loop errors =  0.011987063941617318 [0.00425413 0.01178733 0.01419518 0.00510187 0.0061559  0.00759217
 0.00314372] 0.0030818513922806854 0.012196729854619726
GE loop errors =  0.011987063932461253 [0.00425413 0.01178733 0.01419518 0.00510187 0.0061559  0.00759217
 0.00314372] 0.003081851392350841 0.012196729895136649
GE loop errors =  0.011987063966795872 [0.00425413 0.01178733 0.01419518 0.00510187 0.0061559  0.00759217
 0.00314372] 0.0030818513918433495 0.012196729852561816
GE loop errors =  0.011987067034095925 [0.00425413 0.01178733 0.01419518 0.00510187 0.0061559  0.00759217
 0.00314372] 0.003081851179208568 0.012196742926386506
GE loop errors =  0.01198706397185996 [0.00425413 0.01178733 0.01419518 0.00510187 0.0061559  0.00759217
 0.00314372] 0.003081851391797428 0.012196652543400077
GE loop errors =  0.008079382724443362 [0.00391565 0.01074439 0.01292379 0.004668   0.00563144 0.00695975
 0.00287028] 0.0029085450554103085 -0.0038767008125111246
GE loop errors =  0.0054638748009117055 [0.00318717 0.00861621 0.01035155 0.00376323 0.00453892 0.00560758
 0.00230097] 0.002321054350106211 -0.018208384559748048
GE loop errors =  0.00797709238135784 [0.00347467 0.00959159 0.01154382 0.00415747 0.00501594 0.00619569
 0.0025613 ] 0.0025834276631849454 0.0014988057247391495
GE loop errors =  0.00434430720481245 [0.00278698 0.00751267 0.0090231  0.00328433 0.00396118 0.00489532
 0.00200681] 0.0020417375553974763 -0.015236574882621312
GE loop errors =  0.006862198975018566 [0.0030576  0.00847172 0.01019805 0.00366697 0.00442431 0.00546722
 0.00226309] 0.002304420176654682 0.0014404091500495042
GE loop errors =  0.003445428340060175 [0.00237122 0.00640335 0.00769067 0.0027959  0.00337206 0.00416852
 0.00171076] 0.0017556787288278298 -0.014137088104885898
GE loop errors =  0.005837609932782414 [0.00264277 0.00735732 0.00885903 0.00317916 0.00383591 0.00474231
 0.0019662 ] 0.0020217927955524188 0.0013902081023564605
GE loop errors =  0.002564723527653981 [0.00196427 0.00530273 0.00636768 0.00231447 0.00279133 0.00345175
 0.00141696] 0.00146704875928972 -0.01262986830011048
GE loop errors =  0.004898753999399444 [0.00222993 0.00624748 0.00752564 0.00269362 0.00325026 0.00402031
 0.00167038] 0.0017353860034911434 0.001322640533910402
GE loop errors =  0.0017386716036418698 [0.00156157 0.00420782 0.00505124 0.0018367  0.002215   0.00273995
 0.00112453] 0.001174683554426301 -0.011269882347094473
GE loop errors =  0.004040976529303353 [0.0018193  0.00514179 0.00619729 0.0022103  0.0026673  0.00330113
 0.0013755 ] 0.001444943779675495 0.0012266532050606571
GE loop errors =  0.0009563924977517846 [0.00116465 0.00311973 0.0037424  0.00136347 0.00164411 0.00203432
 0.00083372] 0.0008783950033968389 -0.010017079551417396
GE loop errors =  0.0032602313480077083 [0.00141135 0.00404018 0.00487378 0.00172935 0.00208717 0.00258489
 0.00108153] 0.0011501472028349925 0.0010945319194464176
GE loop errors =  0.000194110576636361 [0.00077673 0.00204136 0.00244409 0.00089686 0.00108113 0.00133778
 0.00054524] 0.0005782026402701788 -0.008825369419945162
GE loop errors =  0.002550444697139692 [0.00100702 0.00294315 0.00355554 0.00125126 0.00151048 0.00187224
 0.00078856] 0.0008506648585858024 0.0009147223872689736
GE loop errors =  -0.0006043455156326122 [0.000405   0.00097975 0.0011636  0.00044175 0.00053189 0.00065733
 0.00026087] 0.0002746943158881271 -0.00762378504468586
GE loop errors =  0.00189888558486985 [0.00060831 0.00185235 0.00224421 0.00077727 0.00093868 0.00116488
 0.00049698] 0.0005462038389380079 0.0006646202973534798
GE loop errors =  -0.0011981791178647533 [1.23239656e-04 1.54531473e-04 1.66896225e-04 9.16188960e-05
 1.09288297e-04 1.33256363e-04 3.96914060e-05] 3.271986849342698e-05 -0.005260642586820019
GE loop errors =  0.001221121722258925 [0.00022494 0.00077807 0.00095054 0.00031406 0.00037975 0.00047232
 0.00020934] 0.00023773741991574976 0.0001234398292835337
GE loop errors =  -0.000597461095888055 [6.67848965e-05 8.22488487e-05 8.89397392e-05 5.02567613e-05
 6.00324599e-05 7.35857950e-05 2.13796547e-05] 1.7943636510326516e-05 -0.0009642833530145722
GE loop errors =  0.00034480352843024864 [-4.32701009e-05 -5.16919074e-05 -5.59435185e-05 -3.29600525e-05
 -3.94370318e-05 -4.86361203e-05 -1.36632013e-05] -1.176928645707162e-05 -0.0007639362382345638
GE loop errors =  2.897827296641231e-05 [-3.77917621e-06 -4.36412305e-06 -4.70689240e-06 -2.85457326e-06
 -3.41655578e-06 -4.21711738e-06 -1.15973676e-06] -1.000599446476097e-06 -6.605489319255753e-05
GE loop errors =  3.863955226057758e-07 [-5.12944095e-08 -5.06844723e-08 -5.37458726e-08 -3.66429476e-08
 -4.38332490e-08 -5.39942913e-08 -1.37450149e-08] -1.2498010105999047e-08 -1.592083714641035e-06
GE loop errors =  -3.0625557270735904e-08 [3.84128833e-09 5.04515561e-09 5.49322192e-09 3.04861957e-09
 3.65020751e-09 4.50857958e-09 1.31581183e-09] 1.0999967915703568e-09 1.4701746575696006e-07
GE loop errors =  -4.318754354137866e-09 [5.36749128e-10 7.15334205e-10 7.78996621e-10 4.28738767e-10
 5.13374034e-10 6.34096230e-10 1.86013923e-10] 1.5521375157367956e-10 2.0794706401261465e-08
GE loop errors =  -4.831163247231984e-12 [3.56391999e-13 1.07550427e-12 1.16481824e-12 4.65315286e-13
 5.60183844e-13 6.96265962e-13 2.47212840e-13] 1.895844592425533e-13 3.602451670303708e-11
GE loop errors =  -5.321298957028375e-13 [5.53966595e-14 9.70924730e-14 1.04149328e-13 5.24580379e-14
 6.29669927e-14 7.80174536e-14 2.37683137e-14] 1.947747518826759e-14 2.801314735734195e-12
Iteration: 01  Distance:  2.696559652967859e-12
G components =  0.06497454989773595 0.015978119313422412 0.1236475612890091
Foreign debt holdings =  0.043861503997608776
Foreign capital holdings =  0.0
resource constraint:  -8.839248777370301e-13
Checking constraints on capital, labor, and consumption.
    There were no violations of the constraints on labor  supply.
    There were no violations of the constraints on  consumption.
Maximum error in labor FOC =  5.302425165609748e-13
Maximum error in savings FOC =  1.149302875091962e-12
Government spending breakpoints are tG1:  20 ; and tG2: 256
...

Note that I did not continue to run through TPI, but unless the budget parameters are way off and result in an explosion of government debt, I would expect the model to solve.

[Abhinav1109]

While trying to run the updated codes and ran the program for almost one whole day, no errors are shown but the program is entering in an infinite process as per the following.

Any suggestions as to what could the possible reasons for these warnings/messages be (computational power or some problems in initial parameters)?

[jdebacker]

@abhinav1109 I think the updates to the example script I’m proposing to add in the currently open PR should fix this. Can you try to run that script?

[jdebacker]

@abhinav1109. I just merged that PR so you can get the file by just syncing to the master. Please report back on how it works. I ran the full baseline and reform through TPI in about 4-5 hours. Everything solved.

[sebastiansajie]

Jason, I had to make some changes to taxcalc and the program ran, though in the end I got this message:-

[sebastiansajie]

Are there any graphs/visuals to show the results? Thanks.

[jdebacker]

@sebastiansajie I'm glad you were able to run the script through to completion. Everything looks to have solved.

All output is saved in directories OG-India/run_examples/OUTPUT_BASELINE and OG-India/run_examples/OUTPUT_REFORM.

To plot these, you can use the plotting utilities in OG-India. I noticed one typo in the dictionary used for variable labels by the plots. I just merged a PR to fix this. So to follow my example below, first:

1. Sync your OG-India repo
2. Update the ogindia package by running python setup.py develop from the OG-India/ directory and while you are in the ogindia-dev environment.

After you do this, you can follow this example to produce a bunch of standard output plots (you can generate other by using the other functions in ogindia.output_plots or by just writing your own custom plot scripts that read in the output saved from the model.

(base) A02655JDEBACKER:~ jason.debacker$ cd repos/OG-India/run_examples

(base) A02655JDEBACKER:~ jason.debacker$ mkdir plots

(base) A02655JDEBACKER:run_examples jason.debacker$ source activate ogindia-dev

(ogindia-dev) A02655JDEBACKER:run_examples jason.debacker$ ipython
Python 3.6.7 | packaged by conda-forge | (default, Jul  2 2019, 02:07:37)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.8.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from ogindia import output_plots as op

In [2]: op.plot_all('OUTPUT_BASELINE', 'OUTPUT_REFORM', 'plots')
No handles with labels found to put in legend.
No handles with labels found to put in legend.
No handles with labels found to put in legend.
No handles with labels found to put in legend.
/Users/jason.debacker/repos/OG-India/ogindia/output_plots.py:227: RuntimeWarning: invalid value encountered in true_divide
  var_to_plot = (reform_val - base_val) / base_val
No handles with labels found to put in legend.
No handles with labels found to put in legend.

This will save about 20 or so plots to the OG-India/run_examples/plots directory.

Here's one of them:

[sebastiansajie]

@jdebacker , as ipython was'nt working for me, I put those commands into a two line file and ran it and generated the plots. I also needed to add

if not os.path.exists(save_path):
       os.makedirs(save_path) 

to 'def plot_all()' in output_plots.py.

I shall write the detailed changes made to records.py to make the main program work.

Thanks.

[jdebacker]

@sebastiansajie Glad you got the plots. If you create the directory to save the plots to first, e.g.,

(base) A02655JDEBACKER:~ jason.debacker$ mkdir plots

Then no need to modify output_plots.py.

Yes - open a PR to the taxcalc repo with your changes. I had no issue and I'm seeing all taxcalc tests passing from the master branch, so I'd be interested in what you had to change.

[Abhinav1109]

Unfortunately, the code is still not running. Recounting the steps, they are as follows: a. Cloned the repository to a directory on the computer b. Navigated to the directory to which we had cloned this repository and run conda env create -f environment.yml c. Activated the ogindia-dev environment using the command activate ogindia-dev on Windows. d. Installed the packages by running the command by pip install -e . e. Navigated to ./run_examples f. Run the model using the file run_ogindia_experiment.py

Now, while running the updated codes, again got the error in reading GST records as below:

Since it is evident that GSTRecords could not be read, tried to open the file gst.csv in the taxcalc. However, while opening, got errors as

"The file format and extension of 'file.csv' don't match. The file could be corrupted or unsafe. Unless you trust its source, don't open it. Do you want to open it anyway?" and "Excel has detected that 'file.csv' is a SYLK file, but cannot load it. Either the file has errors or it is not a SYLK file format. Click OK to try to open the file in a different format."

Since this appears to be a common error in excel in files where the first column header starts with 'ID', introduced quotes around the same as below:

Though this should have corrected the error, we are still getting the same error of non-availability of GSTRecords. Any workaround which could be suggested for this issue.

[jdebacker]

@Abhinav1109 I heard from @sebastiansajie that he had the same error and, as he notes above, he's got some changes to the taxcalc package that should fix this. I was unable to recreate this error - my taxcalc package worked fine - I could work with it as normal and all tests pass from the master branch.

I have some guesses about the root of the problem, but they are just guesses. I look forward to @sebastiansajie opening a PR in the taxcalc repo to fix this.

[jdebacker]

@Abhinav1109 and @sebastiansajie this StackOverflow thread seems to address the issue.

A couple solutions: 1) rename ID_NO to REC_ID (note that this will require down stream changes in the taxcalc package) or 2) change the order of the columns in the csv file so that the first doesn't start with "ID".

[Abhinav1109]

@sebastiansajie Sir, I tried changing the order of the columns in the csv file but that did not work. Looking forward to knowing the changes you had made in the taxcalc to resolve the error.

[sebastiansajie]

@Abhinav1109, I was surprised by the error too and had to dig into the code. These are the changes I made to both gstrecords.py and records.py respectively. PLEASE NOTE: These changes I made directly in the package (to check if it works) which is embedded deep in the directory structure in my case this was as follows:-

I will ask Martin why the FIELD_VARS are not accessible outside the read_var_info function. I will then update the tax calc.

gstrecords.py

records.py

You can make the changes in taxcalc repository too but then you would need to rebuild the package.

[jdebacker]

@Abhinav1109 To be sure, after you made the changes to the gst.csv file, did you rebuild the taxcalc package?

To do this you want to be in the ogindia-dev environment and then do:

1. cd to the taxcalc directory.
2. From taxcalc/ run python setup.py develop

You can then go back to OG-India/run_examples and try the run_ogindia_experiment.py script. (alternatively, before doing this, you can check things worked by trying to run taxcalc by doing the following (it won't matter what directory you are in since you just built the taxcalc package above):

ogindia-dev) A02655JDEBACKER:~ jason.debacker$ ipython
Python 3.6.7 | packaged by conda-forge | (default, Jul  2 2019, 02:07:37)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.8.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import taxcalc

In [2]: from taxcalc import *

In [3]: recs = Records()

In [4]: grecs = GSTRecords()

In [5]: crecs = CorpRecords()

In [6]: pol = Policy()

In [7]: calc1 = Calculator(policy=pol, records=recs, gstrecords=grecs, corprecor
   ...: ds=crecs, verbose=False)

If you do not have iPython, you can install by condo install ipython or you can do the above in the iPython console in Spyder or in a Jupyter Notebook.

[kumar-ab]

I will ask Martin why the FIELD_VARS are not accessible outside the read_var_info function. I will then update the taxcalc.

I tried to check why FIELD_VARS are not accessible outside the read_var_info function and had an interesting observation

(I put some check print commands in the _blow_up method)

(and got this output)

This shows that the FIELD_VARS are accessible outside the read_var_info function also. But something goes amiss from the year 2020 onward. Will try to look into this.

@sebastiansajie @jdebacker @Abhinav1109

[Abhinav1109]

The code modification suggested by @sebastiansajie worked at getting past the GSTRecords error. Its now not encountering any issues on this count. However, now trying to understand the following errors i got in different runs of the same code.

a. Max age (s=0) insuff. data in year 2023. Fill in final ages with insuff. data with most recent successful estimate. b. ValueError: could not broadcast input array from shape (101,12) into shape (21,12) c. File "c:\users\tpru\desktop\work\og-india\ogindia\txfunc.py", line 1042, in tax_func_loop min_age = int(np.maximum(data['Age'].min(), s_min)) ValueError: cannot convert float NaN to integer

[sebastiansajie]

@Abhinav1109 could you check the dataset that is being read. At some point I had to manually copy pitSmalldata.csv directly into the site-package.

[sktpru]

yesterday I deleted all files on my computer(MacBook) and tried to run the program after fresh cloning. The program seemed to be running and has so far had not given errors but it was not stopping yesterday night, so I switched off the comp.

I have again started it and since last 2 hours it is still running. I will update once it is over.

few of the output files are made in OG-India/run_examples/OUTPUT_BASELINE and OG-India/run_examples/OUTPUT_REFORM also.

I will update once it is over.