search

EconForge / dolo.py

Economic modelling in python
BSD 2-Clause "Simplified" License
98 stars 72 forks source link

Find a way to cleanly install the optimally distributed grids. #2

Open albop opened 12 years ago

albop commented 12 years ago

The data is available at http://www.quantize.maths-fi.com/gaussian_database . We should provide a way to download it and install it in the good location.

Currently its path is hardcoded and works only on 1 computer.

sglyon commented 12 years ago

What do these grids contain?

Not sure if it is the same thing, but there is a function in scipy that can generate gauss-hermite quadrature nodes.

sglyon commented 12 years ago

Maybe we could embed this somewhere in setup.py

We could check if the user has an internet connection, then ask them if they would like to download these and if both pass we could do something like this http://code.activestate.com/recipes/496685-downloading-a-file-from-the-web/

albop commented 12 years ago

The optimally quantized data are meant to be used exactly like the gauss-hermite quadrature. They are supposed to behave better when the function to integrate is not smooth. Gauss-hermite is exact for polynomials up to order 'k'.

It is very easy to use once we have the precomputed nodes/weights (see dolo.numeric.quantization).

We also ship gauss-hermite quadrature using scipy in dolo.numeric.quadrature .

albop commented 12 years ago

There are basically two problems to solve:

1/ when do we download the data ?

the grids to download are very heavy (a few gigabytes). So it may not be very good to download them during the main installation. And a while ago I changed the default to gauss-hermite so users don't need it immediately.

2/ where ?

There are several places where program data can be stored at installation time (using options 'data_files' or 'package_data'). Another option would be to use standard data directory on linux ( ~/.local/.share ? ) and windows. This is probably better for people using virtual environements (they don't need to install the grids each time they setup a new environment).

Currently my preferred option would be to have some independent script to install it later in a data directory (there can be an option in the setup.py though). A good inspiration would be the way IPython installs mathjax on demand.

If we directly embed the script in the quantization file, we may contribute it later to the guy who made the quantization grids.

sglyon commented 12 years ago

Would it be more feasible to simply find the algorithm that created the nodes and write our own implementation of it?

Not sure how much computational overhead is created in generating the nodes, but if it isn't too bad maybe that's the way to do it.

albop commented 12 years ago

I don't think it is an option. The algorithm is quite complicated by itself (see research papers on the website) and it takes a long time to generate the grid. It would be a completely different project.