olemb / dbfread

Read DBF Files with Python
MIT License
224 stars 91 forks source link

Import into postgres -specific schema #63

Open andyfensham opened 2 years ago

andyfensham commented 2 years ago

How would I be able to import dbf data into a specific schema e.g. For e.g. if table was people.dbf I want to not import into the main public schema but want to import into import.people

ahmed-masud commented 8 months ago

Something like this:


from sqlalchemy import create_engine
#  etc....

def get_engine(db_name):
    # Create the connection to the database
    engine = create_engine(f'postgresql://postgres:postgres@db:5432/{db_name}')
    return engine

def import_table(dbf_file, table_name, db_name):
    try:
        # Attempt to read the dbf file with utf-8 encoding
        table = DBF(dbf_file, load=True)

        df = pd.DataFrame(iter(table))
        # # Print a preview of the dataframe
        logging.info(f"Preview of '{table_name}':\n{df.head()}")
        # # Convert DataFrame to SQL
        engine = get_engine(db_name)
        df.to_sql(table_name, con=engine, schema='import', if_exists='replace', index=False)
# --------------------------------------^^^^^^^^^^^^^--------------
    except Exception as e:
        # Broad exception handling to catch any other unexpected errors
        logging.error(f"Unexpected error during import of file {dbf_file}: {e}. Investigation needed.")

will work, and import the table(s) in a schema named import just create the schema in the database in your db before hand CREATE SCHEMA import IF NOT EXIST; or some such before hand.

A