Amazon credentials path

[lpillmann]

Error when running Jupyter notebooks in serenata-de-amor/develop saying it could not find config.ini file (Amazon credentials).

Could not find config.ini file.
using fetch from serenata_toolbox.datasets
`Could not find config.ini file.
You need Amzon section in it to interact with S3
(Check config.ini.example if you need a reference.)
—-------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-15-b724014e1ca7> in <module:angry:)
—--> 1 fetch('2017-04-21-sex-place-distances.xz', '../data')

/Users/renangohe/anaconda/envs/serenata_de_amor/lib/python3.6/site-packages/serenata_toolbox/datasets/__init__.py in fetch(filename, destination_path)
     72 
     73 def fetch(filename, destination_path):
—-> 74     datasets = Datasets(destination_path)
     75     return datasets.downloader.download(filename)
     76 

/Users/renangohe/anaconda/envs/serenata_de_amor/lib/python3.6/site-packages/serenata_toolbox/datasets/__init__.py in __init__(self, local_directory)
     53             local_directory,
     54             bucket=self.remote.bucket,
—-> 55             **self.remote.credentials
     56         )
     57 

TypeError: type object argument after ** must be a mapping, not NoneType

We are working on implementing a simple search for the config file in the project folder.

[cuducos]

Hi @luipillmann,

Can you provide more detail abou the code you're running?

Also I'm not sure what you mean by running Jupyter notebooks in serenata-de-amor/develop: if you're running jupyter notebook from that directory that is not how we thought things would work. Run jupyter notebook from the root, not from develop/ (or, if you really want to run from develop/, copy config.ini from root to there).

BTW do you have a config.ini in the root of the project?

[lpillmann]

Hi, @cuducos.

We were running jupyter notebook from the root, where the config.ini file was placed (its content was copied from the example .ini).

What I meant is that the .ipnb files were run from develop/. It seems the program expects config.ini to be in the same folder where notebooks are.

We suppose it has something to do with line 12 in datasets/remote.py

[cuducos]

Ow…got it. Makes sense.

This should fix it:

class RemoteDatasets:

    CONFIG = 'config.ini'

    def __init__(self, config=None):
        self.credentials = None
        self.config = config or self.CONFIG

    …

And replace self.CONFIG by self.config.

Sorry I can't open a PR right now, but probably something among these lines could fix. Feel free to open a PR.

On other news we could get rid of a config file and go for envvars. I initially opted to keep the config file because it was there already, but maybe this is not the best strategy. What do you think @jtemporal?

[caiopo]

Hi,

I talked to @luipillmann here at the sprint about having a default configuration file to be opened if a custom config.ini could not be found.

One of the main reasons of doing this would be that most toolbox's use cases don't require a custom config.ini, the default values work fine unless you need to upload something.

[cuducos]

One of the main reasons of doing this would be that most toolbox's use cases don't require a custom config.ini, the default values work fine unless you need to upload something.

👍

[jtemporal]

since #114 was merged I'm closing this one! Thanks everyone :)