Can't load plugin: sqlalchemy.dialects:redshift.psycopg2

[mjwillson]

Getting this with the latest SQLAlchemy, from

s.create_engine("redshift+psycopg2://foo:bar@localhost:5439/db")

However this works fine:

s.create_engine("redshift://foo:bar@localhost:5439/db")

Suspect it might be because you're registering yourself with a + here: https://github.com/binarydud/redshift_sqlalchemy/blob/master/redshift_sqlalchemy/__init__.py#L6 If the error message is anything to go by, looks like SQLAlchemy is converting that + into a . prior to lookup in the registry.

[briandailey]

@mjwillson what version of SQLAlchemy is this?

[faraocious]

Running into this error on SQLAlchemy 0.9.8

[Utsav2]

Same error here on 0.9.8

[graingert]

Can you try with https://pypi.python.org/pypi/sqlalchemy-redshift ?

[ziziermao]

Hey I am using redshift-sqlalchemy==0.4.1, but still get the same error. I have sqlalchemy==1.0.9 redshift-sqlalchemy==0.4.1 in my requirements.txt and have python-psycopg2 installed on the machine, may I ask what else I am missing?

[graingert]

@ziziermao please try with https://pypi.python.org/pypi/sqlalchemy-redshift/0.4.0

[ziziermao]

Thanks a lot graingert! But when I tried to use 0.4.0, I got DistributionNotFound: SQLAlchemy>=0.8.0, but I do have SQLAlchemy==1.0.9. And I was able to use redshift-sqlalchemy==0.4.1 on my laptop. Do you think I might missed some packages?

[graingert]

@ziziermao let's move this to IRC I'm graingert and I'll be in the #sqlalchemy channel on freenode

[ziziermao]

hey #graingert, thank you. I don't know where's IRC, could you please help with that? and I do see in redshift-sqlalchemy's dialect init, they have : from sqlalchemy.dialects import registry

registry.register("redshift", "redshift_sqlalchemy.dialect", "RedshiftDialect") registry.register('redshift+psycopg2', "redshift_sqlalchemy.dialect", "RedshiftDialect")

but how can the sqlalchemy doesn't recognize it?

[himanshigupta29]

I also faced the same issues.After a lot of trials and tweaks I found a solution. Initially I used virtual environment for installing the project dependencies like pscopg2 , redshift-SQLalchemy etc .But later I installed the dependencies on the system , not in virtual environment ..this solved the problem. May be there was some issue in loading the modules of dependencies from virtual environment

[graingert]

Can you post a Dockerfile or Vagrantfile that configures a virtualenv in the same way? On 4 Jun 2016 10:01, "himanshigupta29" notifications@github.com wrote:

I also faced the same issues.After a lot of trials and tweaks I found a solution. Initially I used virtual environment for installing the project dependencies like pscopg2 , redshift-SQLalchemy etc .But later I installed the dependencies on the system , not in virtual environment ..this solved the problem. May be there was some issue in loading the modules of dependencies from virtual environment

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/binarydud/redshift_sqlalchemy/issues/13#issuecomment-223745177, or mute the thread https://github.com/notifications/unsubscribe/AAZQTI2_NGZwcy9-hIM24L2gj6c9QzuDks5qIT7egaJpZM4B6IlK .

[aldencg]

For future searchers who are having this problem with Airflow: We had this same problem, and the issue was that the code in the dag file itself was attempting to create the Redshift connection rather than the code in the callable we passed to the operator. We had something like:

PythonOperator(
    task_id='do stuff',
    python_callable=Redshift().upload,
    dag=dag)

The constructor for the Redshift class was attempting to make the connection. Since it's in the DAG file, it was being executed on the master node, which clearly couldn't load sqlalchemy-redshift for some reason (maybe it was outside the virtual environment?). We fixed this by wrapping the callable in another function that wouldn't get executed until it got to the worker node.

def upload_wrapper(): Redshift().upload()

PythonOperator(
    task_id='do stuff',
    python_callable=upload_wrapper,
    dag=dag)

[davidyolchuyev]

@ziziermao please try with https://pypi.python.org/pypi/sqlalchemy-redshift/0.4.0

This worked for me. THANK YOU SO MUCH!