Migrate boto -> boto3 - Githubissues

turi-code / SFrame

SFrame: Scalable tabular and graph data-structures built for out-of-core data analysis and machine learning.

BSD 3-Clause "New" or "Revised" License

889 stars 331 forks source link

Migrate boto -> boto3 #336

Open hershaw opened 8 years ago

hershaw commented 8 years ago

Info

boto maintenance has staled and although boto is python 3 compatible system-wide boto plugins may not be which can cause an import of SFrame objects to fail at runtime. For example, this will happen on GCE ubuntu machines

boto is used in two places: file_util.py and test_gl_pickler.py

file_util.py is unit tested here

TODO

[ ] Change setup.py to remove boto and include boto3
[ ] Migrate file_util.py
- [ ] TODO: spec boto3 api usage
[ ] Migrate test_gl_picker.py
- [ ] TODO: spec boto3 api usage

hershaw commented 8 years ago

In my fork I have changed setup.py to use boto3 and then re-built the package:

source oss_local_scripts/python_env.sh debug
cd debug/oss_src/unity/python
make -j 4

But when I try to do an import of boto3 in file_util.py it's not available. Do I need to rebuild the miniconda package or something using the new setup.py?

@TobyRoseman

TobyRoseman commented 8 years ago

@hershaw - Good question. Changes to the setup.py affect the egg/wheel but not dev builds, which is what you're running. To change that, make the same change here: https://github.com/dato-code/SFrame/blob/fac7e34200e9105e9f54ab46d60c12a7643d070b/oss_local_scripts/pip_requirements.txt

Then you'll need to do a configure --cleanup. Then rerun configure and rerun make.

Let me know if you have any more questions.