search

uswitch / blueshift

Automate copying data from S3 into Amazon Redshift
Eclipse Public License 1.0
118 stars 19 forks source link

App does not recognize 'manifest.edn' file #14

Open ghost opened 9 years ago

ghost commented 9 years ago

I had this software working a few weeks ago, but now it does not work.

The S3 bucket scanner finds my two directories with 'manifest.edn' files, but does not add them to the "watched directories" list. The list stays length 0, and no files are uploaded to Postgres (Redshift).

The DEBUG printout shows that it finds the manifest.edn file. There is no printout about "I found a manifest.edn file but did not like it".

ghost commented 9 years ago

I set it up in a fresh bucket and now it works. I'm wondering if there is a timer feature to ignore buckets older than N days?

tgk commented 9 years ago

There's no feature that ignores old directories so it does sound a bit puzzling. I haven't observed anything like that myself. Were there any files apart from the manifest.edn files in the folders? Not that that should have any consequence but I'm trying to figure out what is happening.

Another wild idea: are you sure the rights on the bucket were right? Maybe your user didn't have the correct rights for the bucket.