chimpler / postgres-aws-s3

aws_s3 postgres extension to import/export data from/to s3 (compatible with aws_s3 extension on AWS RDS)
Apache License 2.0
141 stars 49 forks source link

ModuleNotFoundError: No module named 'boto3' Despite Installing 'boto3' #26

Open AkshitKhajuria opened 1 year ago

AkshitKhajuria commented 1 year ago

I'm running a Node server along with Postgres-12. I have save a stored function that uses query_export_to_s3 to generate a report and export it to s3. However I always run into this problem whenever the stored function gets executed. Here's the stack trace-

DatabaseError [SequelizeDatabaseError]: spiexceptions.ExternalRoutineException: error fetching next item from iterator
    at Query.formatError (/home/Sam/Documents/GitHub/cms-express-api/node_modules/sequelize/lib/dialects/postgres/query.js:386:16)
    at Query.run (/home/Sam/Documents/GitHub/cms-express-api/node_modules/sequelize/lib/dialects/postgres/query.js:87:18)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  parent: error: spiexceptions.ExternalRoutineException: error fetching next item from iterator
      at Parser.parseErrorMessage (/home/Sam/Documents/GitHub/cms-express-api/node_modules/pg-protocol/src/parser.ts:369:69)
      at Parser.handlePacket (/home/Sam/Documents/GitHub/cms-express-api/node_modules/pg-protocol/src/parser.ts:188:21)
      at Parser.parse (/home/Sam/Documents/GitHub/cms-express-api/node_modules/pg-protocol/src/parser.ts:103:30)
      at Socket.<anonymous> (/home/Sam/Documents/GitHub/cms-express-api/node_modules/pg-protocol/src/index.ts:7:48)
      at Socket.emit (events.js:314:20)
      at Socket.EventEmitter.emit (domain.js:483:12)
      at addChunk (_stream_readable.js:297:12)
      at readableAddChunk (_stream_readable.js:272:9)
      at Socket.Readable.push (_stream_readable.js:213:10)
      at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
    length: 349,
    severity: 'ERROR',
    code: '38000',
    detail: "ModuleNotFoundError: No module named 'boto3'",
    hint: undefined,
    position: undefined,
    internalPosition: undefined,
    internalQuery: undefined,
    where: 'Traceback (most recent call last):\n' +
      '  PL/Python function "query_export_to_s3", line 6, in <module>\n' +
      '    return plan.execute(\n' +
      'PL/Python function "query_export_to_s3"',
    schema: undefined,
    table: undefined,
    column: undefined,
    dataType: undefined,
    constraint: undefined,
    file: 'plpy_elog.c',
    line: '125',
    routine: 'PLy_elog_impl',
    sql: `begin; select * from public.stored_function_weather_data_export('select * from public."WeatherAA12"','04-11-2022/weather-report-1667546728087.csv'); fetch all in "cursor_data"; close cursor_data;commit;`,
    parameters: undefined
  },
  original: error: spiexceptions.ExternalRoutineException: error fetching next item from iterator
      at Parser.parseErrorMessage (/home/Sam/Documents/GitHub/cms-express-api/node_modules/pg-protocol/src/parser.ts:369:69)
      at Parser.handlePacket (/home/Sam/Documents/GitHub/cms-express-api/node_modules/pg-protocol/src/parser.ts:188:21)
      at Parser.parse (/home/Sam/Documents/GitHub/cms-express-api/node_modules/pg-protocol/src/parser.ts:103:30)
      at Socket.<anonymous> (/home/Sam/Documents/GitHub/cms-express-api/node_modules/pg-protocol/src/index.ts:7:48)
      at Socket.emit (events.js:314:20)
      at Socket.EventEmitter.emit (domain.js:483:12)
      at addChunk (_stream_readable.js:297:12)
      at readableAddChunk (_stream_readable.js:272:9)
      at Socket.Readable.push (_stream_readable.js:213:10)
      at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
    length: 349,
    severity: 'ERROR',
    code: '38000',
    detail: "ModuleNotFoundError: No module named 'boto3'",
    hint: undefined,
    position: undefined,
    internalPosition: undefined,
    internalQuery: undefined,
    where: 'Traceback (most recent call last):\n' +
      '  PL/Python function "query_export_to_s3", line 6, in <module>\n' +
      '    return plan.execute(\n' +
      'PL/Python function "query_export_to_s3"',
    schema: undefined,
    table: undefined,
    column: undefined,
    dataType: undefined,
    constraint: undefined,
    file: 'plpy_elog.c',
    line: '125',
    routine: 'PLy_elog_impl',
    sql: `begin; select * from public.stored_function_weather_data_export('select * from public."WeatherAA12"','04-11-2022/weather-report-1667546728087.csv'); fetch all in "cursor_data"; close cursor_data;commit;`,
    parameters: undefined
  },
  sql: `begin; select * from public.stored_function_weather_data_export('select * from public."WeatherAA12"','04-11-2022/weather-report-1667546728087.csv'); fetch all in "cursor_data"; close cursor_data;commit;`,
  parameters: undefined
}

I have already installed boto3 by executing - pip3 install boto3 but it simply doesn't seem to work. Lastly I also tried installing boto3 by cloning and building from it's source files following the instructions at boto3 No success there either. I'm new to python and AWS. Perhaps I'm making a rookie mistake so please bear with me.

Can someone please point out where I'm going wrong ?

zlianzhuang commented 1 year ago

https://github.com/radondb/multi-platform-postgresql/blob/main/image/postgresql/Dockerfile.template#L172

barman is also require boto3. i have install boto3 when support barman.

commit on multi_platform_postgresql: 98d15f7f5c498e15c931cafd6ff837d094d8e616