search

rapid7 / metasploit-framework

Metasploit Framework
https://www.metasploit.com/
Other
33.84k stars 13.92k forks source link

msfdb init failed in ubuntu #12098

Closed pastaoficial closed 3 years ago

pastaoficial commented 5 years ago

Steps to reproduce

Running msfdb init I have:

Found a database at /home/pasta/.msf4/db, checking to see if it is started
Using database port 5432 found in /home/pasta/.msf4/database.yml
Starting database at /home/pasta/.msf4/db...failed
2019-07-16 15:07:34.294 -03 [30693] LOG:  database system is shut down
[!] Your database may be corrupt. Try reinitializing.
MSF web service is already running as PID 28962

if append the debug parameter:

Found a database at /home/pasta/.msf4/db, checking to see if it is started
Using database port 5432 found in /home/pasta/.msf4/database.yml
run_cmd: cmd=pg_ctl -o "-p 5432" -D /home/pasta/.msf4/db status, input=, env={}
'pg_ctl -o "-p 5432" -D /home/pasta/.msf4/db status' returned 3
pg_ctl: no server running

Starting database at /home/pasta/.msf4/db...run_cmd: cmd=pg_ctl -o "-p 5432" -D /home/pasta/.msf4/db -l /home/pasta/.msf4/db/log start, input=, env={}
'pg_ctl -o "-p 5432" -D /home/pasta/.msf4/db -l /home/pasta/.msf4/db/log start' returned 1
waiting for server to start.... stopped waiting
pg_ctl: could not start server
Examine the log output.
run_cmd: cmd=pg_ctl -o "-p 5432" -D /home/pasta/.msf4/db status, input=, env={}
'pg_ctl -o "-p 5432" -D /home/pasta/.msf4/db status' returned 3
pg_ctl: no server running

failed
2019-07-16 15:07:41.726 -03 [30711] LOG:  database system is shut down
[!] Your database may be corrupt. Try reinitializing.
MSF web service is already running as PID 28962

also I changed the posgres's default port to 5432 in /home/pasta/.msf4/database.yml because it is the default port after to being installed with apt

I probe that because I see a similar solution in another issue

I'm going to paste the nmap output to corroborate the port:


Starting Nmap 7.60 ( https://nmap.org ) at 2019-07-16 15:12 -03
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00013s latency).
Not shown: 991 closed ports
PORT     STATE SERVICE
5432/tcp open  postgresql

Metasploit version

Framework Version: 5.0.37-dev-e6766c23a5

I installed Metasploit with:

I just clone the git repository and run the executables there (without install, ./msfconsole, ./msfdb)

OS

What OS are you running Metasploit on?

Ubuntu 18.04.2 LTS

log file

attached msf-ws.log

TomSellers commented 5 years ago

Dealing with similar.

Based on the database log I thought it was a permissions problem.

$ cat ~/.msf4/db/log 
2019-08-03 09:16:50.974 CDT [2678] LOG:  listening on IPv4 address "127.0.0.1", port 5433
2019-08-03 09:16:50.982 CDT [2678] FATAL:  could not create lock file "/var/run/postgresql/.s.PGSQL.5433.lock": Permission denied
2019-08-03 09:16:50.982 CDT [2678] LOG:  database system is shut down
2019-08-03 09:16:53.416 CDT [2745] LOG:  listening on IPv4 address "127.0.0.1", port 5433
2019-08-03 09:16:53.424 CDT [2745] FATAL:  could not create lock file "/var/run/postgresql/.s.PGSQL.5433.lock": Permission denied
2019-08-03 09:16:53.424 CDT [2745] LOG:  database system is shut down

So I added myself to the group but it didn't seem to help.

sudo adduser $USER postgres

Nmap output

PORT     STATE  SERVICE    VERSION
5432/tcp open   postgresql PostgreSQL DB 10.0 - 10.1
| ssl-cert: Subject: commonName=ubuntu
| Subject Alternative Name: DNS:ubuntu
| Not valid before: 2019-04-27T23:04:46
|_Not valid after:  2029-04-24T23:04:46
|_ssl-date: TLS randomness does not represent time
5433/tcp closed pyrrho
5434/tcp closed sgi-arrayd

Environment

$ uname -a
Linux spectre 5.0.0-23-generic #24~18.04.1-Ubuntu SMP Mon Jul 29 16:12:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

 $ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"

$ psql --version
psql (PostgreSQL) 10.9 (Ubuntu 10.9-0ubuntu0.18.04.1)

Debug output

Output from my initial run, before I added myself to the postgres group.

$ ./msfdb init --debug
Creating database at /home/tom/.msf4/db
run_cmd: cmd=initdb --auth-host=trust --auth-local=trust -E UTF8 /home/tom/.msf4/db, input=, env={}
run_cmd: cmd=pg_ctl -o "-p 5433" -D /home/tom/.msf4/db status, input=, env={}
'pg_ctl -o "-p 5433" -D /home/tom/.msf4/db status' returned 3
pg_ctl: no server running

Starting database at /home/tom/.msf4/db...run_cmd: cmd=pg_ctl -o "-p 5433" -D /home/tom/.msf4/db -l /home/tom/.msf4/db/log start, input=, env={}
'pg_ctl -o "-p 5433" -D /home/tom/.msf4/db -l /home/tom/.msf4/db/log start' returned 1
waiting for server to start.... stopped waiting
pg_ctl: could not start server
Examine the log output.
run_cmd: cmd=pg_ctl -o "-p 5433" -D /home/tom/.msf4/db status, input=, env={}
'pg_ctl -o "-p 5433" -D /home/tom/.msf4/db status' returned 3
pg_ctl: no server running

failed
2019-08-03 09:16:50.982 CDT [2678] LOG:  database system is shut down
[!] Your database may be corrupt. Try reinitializing.
Creating database users
psql -p 5433 -c "create user msf with password 'aiOhC/f9WitjWJATSiAu6Ug4MjokXiWN01F3kjTxWVg=';" postgres
run_cmd: cmd=psql -p 5433 -c "create user msf with password 'aiOhC/f9WitjWJATSiAu6Ug4MjokXiWN01F3kjTxWVg=';" postgres, input=, env={}
'psql -p 5433 -c "create user msf with password 'aiOhC/f9WitjWJATSiAu6Ug4MjokXiWN01F3kjTxWVg=';" postgres' returned 2

psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5433"?
psql -p 5433 -c "create user msftest with password 's83B/NOG1GPx66vLX2Krf5OjN61/zJsOrzOVl9kwIvY=';" postgres
run_cmd: cmd=psql -p 5433 -c "create user msftest with password 's83B/NOG1GPx66vLX2Krf5OjN61/zJsOrzOVl9kwIvY=';" postgres, input=, env={}
'psql -p 5433 -c "create user msftest with password 's83B/NOG1GPx66vLX2Krf5OjN61/zJsOrzOVl9kwIvY=';" postgres' returned 2

psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5433"?
psql -p 5433 -c "alter role msf createdb;" postgres
run_cmd: cmd=psql -p 5433 -c "alter role msf createdb;" postgres, input=, env={}
'psql -p 5433 -c "alter role msf createdb;" postgres' returned 2

psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5433"?
psql -p 5433 -c "alter role msftest createdb;" postgres
run_cmd: cmd=psql -p 5433 -c "alter role msftest createdb;" postgres, input=, env={}
'psql -p 5433 -c "alter role msftest createdb;" postgres' returned 2

psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5433"?
psql -p 5433 -c "alter role msf with password 'aiOhC/f9WitjWJATSiAu6Ug4MjokXiWN01F3kjTxWVg=';" postgres
run_cmd: cmd=psql -p 5433 -c "alter role msf with password 'aiOhC/f9WitjWJATSiAu6Ug4MjokXiWN01F3kjTxWVg=';" postgres, input=, env={}
'psql -p 5433 -c "alter role msf with password 'aiOhC/f9WitjWJATSiAu6Ug4MjokXiWN01F3kjTxWVg=';" postgres' returned 2

psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5433"?
psql -p 5433 -c "alter role msftest with password 's83B/NOG1GPx66vLX2Krf5OjN61/zJsOrzOVl9kwIvY=';" postgres
run_cmd: cmd=psql -p 5433 -c "alter role msftest with password 's83B/NOG1GPx66vLX2Krf5OjN61/zJsOrzOVl9kwIvY=';" postgres, input=, env={}
'psql -p 5433 -c "alter role msftest with password 's83B/NOG1GPx66vLX2Krf5OjN61/zJsOrzOVl9kwIvY=';" postgres' returned 2

psql: could not connect to server: No such file or directory
    Is the server running locally and accepting
    connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5433"?
run_cmd: cmd=createdb -p 5433 -O msf -h 127.0.0.1 -U msf -E UTF-8 -T template0 msf, input=aiOhC/f9WitjWJATSiAu6Ug4MjokXiWN01F3kjTxWVg=
aiOhC/f9WitjWJATSiAu6Ug4MjokXiWN01F3kjTxWVg=
, env={}
'createdb -p 5433 -O msf -h 127.0.0.1 -U msf -E UTF-8 -T template0 msf' returned 1

createdb: could not connect to database template1: could not connect to server: Connection refused
    Is the server running on host "127.0.0.1" and accepting
    TCP/IP connections on port 5433?
run_cmd: cmd=createdb -p 5433 -O msftest -h 127.0.0.1 -U msftest -E UTF-8 -T template0 msftest, input=s83B/NOG1GPx66vLX2Krf5OjN61/zJsOrzOVl9kwIvY=
s83B/NOG1GPx66vLX2Krf5OjN61/zJsOrzOVl9kwIvY=
, env={}
'createdb -p 5433 -O msftest -h 127.0.0.1 -U msftest -E UTF-8 -T template0 msftest' returned 1

createdb: could not connect to database template1: could not connect to server: Connection refused
    Is the server running on host "127.0.0.1" and accepting
    TCP/IP connections on port 5433?
Writing client authentication configuration file /home/tom/.msf4/db/pg_hba.conf
run_cmd: cmd=pg_ctl -o "-p 5433" -D /home/tom/.msf4/db status, input=, env={}
'pg_ctl -o "-p 5433" -D /home/tom/.msf4/db status' returned 3
pg_ctl: no server running

Database is no longer running at /home/tom/.msf4/db
run_cmd: cmd=pg_ctl -o "-p 5433" -D /home/tom/.msf4/db status, input=, env={}
'pg_ctl -o "-p 5433" -D /home/tom/.msf4/db status' returned 3
pg_ctl: no server running

Starting database at /home/tom/.msf4/db...run_cmd: cmd=pg_ctl -o "-p 5433" -D /home/tom/.msf4/db -l /home/tom/.msf4/db/log start, input=, env={}
'pg_ctl -o "-p 5433" -D /home/tom/.msf4/db -l /home/tom/.msf4/db/log start' returned 1
waiting for server to start.... stopped waiting
pg_ctl: could not start server
Examine the log output.
run_cmd: cmd=pg_ctl -o "-p 5433" -D /home/tom/.msf4/db status, input=, env={}
'pg_ctl -o "-p 5433" -D /home/tom/.msf4/db status' returned 3
pg_ctl: no server running

failed
2019-08-03 09:16:53.424 CDT [2745] LOG:  database system is shut down
[!] Your database may be corrupt. Try reinitializing.
Creating initial database schema
run_cmd: cmd=bundle exec rake db:migrate, input=, env={}
'bundle exec rake db:migrate' returned 1

/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activesupport-4.2.11.1/lib/active_support/core_ext/object/duplicable.rb:111: warning: BigDecimal.new is deprecated; use BigDecimal() method instead.
rake aborted!
PG::ConnectionBad: could not connect to server: Connection refused
    Is the server running on host "127.0.0.1" and accepting
    TCP/IP connections on port 5433?
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `initialize'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `new'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/postgresql_adapter.rb:651:in `connect'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/postgresql_adapter.rb:242:in `initialize'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/postgresql_adapter.rb:44:in `new'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/postgresql_adapter.rb:44:in `postgresql_connection'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:438:in `new_connection'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:448:in `checkout_new_connection'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:422:in `acquire_connection'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:349:in `block in checkout'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:348:in `checkout'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:263:in `block in connection'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:262:in `connection'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:571:in `retrieve_connection'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_handling.rb:113:in `retrieve_connection'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/connection_handling.rb:87:in `connection'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/migration.rb:941:in `initialize'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/migration.rb:823:in `new'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/migration.rb:823:in `up'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/migration.rb:801:in `migrate'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/tasks/database_tasks.rb:139:in `migrate'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/activerecord-4.2.11.1/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/bin/ruby_executable_hooks:24:in `eval'
/home/tom/.rvm/gems/ruby-2.6.2@metasploit-framework/bin/ruby_executable_hooks:24:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
[?] Initial MSF web service account username? [tom]:
TomSellers commented 5 years ago

@pastaoficial - It appears that you either need to log out for a while or reboot to allow the group change to take effect on Ubuntu 18.04 of you are logged into an interactive session in Gnome.

A restart solved my issue. 

 $ ./msfdb init
Creating database at /home/tom/.msf4/db
Starting database at /home/tom/.msf4/db...success
Creating database users
Writing client authentication configuration file /home/tom/.msf4/db/pg_hba.conf
Stopping database at /home/tom/.msf4/db
Starting database at /home/tom/.msf4/db...success
Creating initial database schema

At this point I'm not sure if the original postgres install added me to the correct groups or if I still would have needed to add myself to the postgres group manually.

Reference: https://askubuntu.com/questions/1045993/after-adding-a-group-logoutlogin-is-not-enough-in-18-04

y0k4i-1337 commented 4 years ago

This can be caused when the script tries to create the socket for the respective db instance. Depending on distribution, the default could be at /var/run/postgres, /run/postgres or /tmp. Yours are considering /var/run/postgres so when you run msfdb init under ordinary user you don't have permissions to do that.

You could change this by editing the template postgres.conf (mine is at /usr/share/postgresql/postgresql.conf.sample) looking for unix_socket_directories entry and set it to unix_socket_directories = '/tmp'. This way, new databases will use that for creating sockets.

Beware though that commands like psql could still look for the original path, so you might have to set PGHOST environment variable accordingly like:

export PGHOST=/tmp

before runing msfdb.

The script should works after that. When running psql by itself, it is sufficient to provide -h /tmp in options.

todb-r7 commented 4 years ago

Just bumping this because the wiki leads the user down a primrose path of sadness. Here's the full repro (after a failed attempt, but the results are the same). Source install on Ubuntu 20 LTS.

Note one fix: Adding the path to the pg_ctl and initdb correctly in my .profile is reflected below. My .profile is edited to include:

# Include PostgreSQL's correct path
if [ -d "/usr/lib/postgresql/12/bin" ] ; then
    PATH="$PATH:/usr/lib/postgresql/12/bin"
fi

Why the PostgreSQL package does this is a mystery, and I suspect there are other problems in there.

todb@azazel:~$ cd git/metasploit-framework
todb@azazel:~/git/metasploit-framework$ sudo apt update && sudo apt-get install -y postgresql postgresql-client
[sudo] password for todb: 
Hit:1 http://us.archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://us.archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:3 http://security.ubuntu.com/ubuntu focal-security InRelease    
Hit:4 http://ppa.launchpad.net/rael-gc/rvm/ubuntu focal InRelease   
Hit:5 http://us.archive.ubuntu.com/ubuntu focal-backports InRelease
Reading package lists... Done
Building dependency tree       
Reading state information... Done
All packages are up to date.
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  postgresql-12 postgresql-client-12 postgresql-client-common
  postgresql-common
Suggested packages:
  postgresql-doc postgresql-doc-12 libjson-perl
The following NEW packages will be installed:
  postgresql postgresql-12 postgresql-client postgresql-client-12
  postgresql-client-common postgresql-common
0 upgraded, 6 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/14.7 MB of archives.
After this operation, 45.7 MB of additional disk space will be used.
Preconfiguring packages ...
Selecting previously unselected package postgresql-client-common.
(Reading database ... 131844 files and directories currently installed.)
Preparing to unpack .../0-postgresql-client-common_214_all.deb ...
Unpacking postgresql-client-common (214) ...
Selecting previously unselected package postgresql-client-12.
Preparing to unpack .../1-postgresql-client-12_12.2-4_amd64.deb ...
Unpacking postgresql-client-12 (12.2-4) ...
Selecting previously unselected package postgresql-common.
Preparing to unpack .../2-postgresql-common_214_all.deb ...
Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common'
Unpacking postgresql-common (214) ...
Selecting previously unselected package postgresql-12.
Preparing to unpack .../3-postgresql-12_12.2-4_amd64.deb ...
Unpacking postgresql-12 (12.2-4) ...
Selecting previously unselected package postgresql.
Preparing to unpack .../4-postgresql_12+214_all.deb ...
Unpacking postgresql (12+214) ...
Selecting previously unselected package postgresql-client.
Preparing to unpack .../5-postgresql-client_12+214_all.deb ...
Unpacking postgresql-client (12+214) ...
Setting up postgresql-client-common (214) ...
Setting up postgresql-client-12 (12.2-4) ...
update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode
Setting up postgresql-common (214) ...

Creating config file /etc/postgresql-common/createcluster.conf with new version
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
  en_us
Removing obsolete dictionary files:
Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service → /lib/systemd/system/postgresql.service.
Setting up postgresql-12 (12.2-4) ...
Creating new PostgreSQL cluster 12/main ...
/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/12/main ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... America/Chicago
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

Success. You can now start the database server using:

    pg_ctlcluster 12 main start

Ver Cluster Port Status Owner    Data directory              Log file
12  main    5432 down   postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log
update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode
Setting up postgresql-client (12+214) ...
Setting up postgresql (12+214) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for systemd (245.4-4ubuntu3) ...
todb@azazel:~/git/metasploit-framework$ sudo service postgresql start && sudo update-rc.d postgresql enable
todb@azazel:~/git/metasploit-framework$ ./msfdb init
Missing requirement: PostgreSQL does not appear to be installed or 'pg_ctl' is not in the environment path
Missing requirement: PostgreSQL does not appear to be installed or 'initdb' is not in the environment path
todb@azazel:~/git/metasploit-framework$ . ~/.profile 
todb@azazel:~/git/metasploit-framework$ ./msfdb init
[?] Would you like to delete your existing data and configurations?: yes
Found a database at /home/todb/.msf4/db, checking to see if it is started
Starting database at /home/todb/.msf4/db...failed
2020-04-27 17:18:02.136 CDT [11451] LOG:  database system is shut down
[!] Your database may be corrupt. Try reinitializing.
[?] Initial MSF web service account username? [todb]: 
[?] Initial MSF web service account password? (Leave blank for random password): 
Generating SSL key and certificate for MSF web service
Attempting to start MSF web service...failed
[!] MSF web service appears to be started, but may not operate as expected.

Please see /home/todb/.msf4/logs/msf-ws.log for additional details.
todb@azazel:~/git/metasploit-framework$ ./msfconsole -qx "db_status; exit"
[-] ***
[-] * WARNING: No database support: could not connect to server: Connection refused
    Is the server running on host "127.0.0.1" and accepting
    TCP/IP connections on port 5433?

[-] ***
[*] postgresql selected, no connection
todb@azazel:~/git/metasploit-framework$
gwillcox-r7 commented 4 years ago

Adding confirmed label as @todb-r7 was able to replicate this issue.

Keskebeu commented 4 years ago

this happens because the msfdb command could not be issued as root. postgreslq has root executable privileges and this sucks. This would be solved if msfbd was to be executed with elevated privileges.

gwillcox-r7 commented 4 years ago

Can confirm that I also ran into this issue and that @mchoji's solution worked for me for resolving the issue and it allows a normal user to execute the msfdb init command.

I do however agree with @todb-r7 that there could be more weirdness going on here as haven't tested this extensively, however it appears to be a workaround for the time being.

Edit: Deleted original comment, seems like others are also encountering other issues as shown at https://github.com/rapid7/metasploit-framework/issues/13084, so @todb-r7's comment is likely right.

gwillcox-r7 commented 4 years ago

Oh also in case anyone is interested, here is the logs from my approach:

$ ./msfdb init
/home/gwillcox/git/metasploit-framework/lib/metasploit/framework/version.rb:22: warning: Insecure world writable dir /home/gwillcox/.rvm/gems/ruby-2.6.6@metasploit-framework/bin in PATH, mode 040777
Creating database at /home/gwillcox/.msf4/db
Starting database at /home/gwillcox/.msf4/db...failed
2020-07-14 11:37:58.784 CDT [6541] LOG:  database system is shut down
[!] Your database may be corrupt. Try reinitializing.
Creating database users
Writing client authentication configuration file /home/gwillcox/.msf4/db/pg_hba.conf
Database is no longer running at /home/gwillcox/.msf4/db
Starting database at /home/gwillcox/.msf4/db...failed

2020-07-14 11:38:01.394 CDT [6608] LOG:  database system is shut down
[!] Your database may be corrupt. Try reinitializing.

Creating initial database schema
[?] Initial MSF web service account username? [gwillcox]: ^CTraceback (most recent call last):
        6: from ./msfdb:1084:in `<main>'

        5: from ./msfdb:1084:in `each'
        4: from ./msfdb:1085:in `block in <main>'
        3: from ./msfdb:984:in `invoke_command'
        2: from ./msfdb:472:in `init_web_service'
        1: from ./msfdb:384:in `ask_value'
./msfdb:384:in `gets': Interrupt

$ ./msfdb delete
/home/gwillcox/git/metasploit-framework/lib/metasploit/framework/version.rb:22: warning: Insecure world writable dir /home/gwillcox/.rvm/gems/ruby-2.6.6@metasploit-framework/bin in PATH, mode 040777
[?] Would you like to delete your existing data and configurations?: Y
Database is no longer running at /home/gwillcox/.msf4/db
Deleting all data at /home/gwillcox/.msf4/db
MSF web service is no longer running
$ cd /usr/share/postgresql/
$ dir
12
$ cd 12
$ ls
catalog_version  information_schema.sql  pg_service.conf.sample  postgresql.conf.sample  system_views.sql
contrib          man                     postgres.bki            psqlrc.sample           timezonesets
errcodes.txt     pg_hba.conf.sample      postgres.description    snowball_create.sql     tsearch_data
extension        pg_ident.conf.sample    postgres.shdescription  sql_features.txt
$ nano postgres
postgres.bki            postgres.description    postgres.shdescription  postgresql.conf.sample
$ nano postgresql.conf.sample
$ sudo nano postgresql.conf.sample
[sudo] password for gwillcox:
$ export PGHOST=/tmp
$  ./msfdb init
-bash: ./msfdb: No such file or directory
$ cd ~
$ cd git/
$ cd metasploit-framework
$ ./msfdb init
/home/gwillcox/git/metasploit-framework/lib/metasploit/framework/version.rb:22: warning: Insecure world writable dir /home/gwillcox/.rvm/gems/ruby-2.6.6@metasploit-framework/bin in PATH, mode 040777
Creating database at /home/gwillcox/.msf4/db
Starting database at /home/gwillcox/.msf4/db...success
Creating database users
Writing client authentication configuration file /home/gwillcox/.msf4/db/pg_hba.conf
Stopping database at /home/gwillcox/.msf4/db
Starting database at /home/gwillcox/.msf4/db...success
Creating initial database schema
[?] Initial MSF web service account username? [gwillcox]: *redacted*
[?] Initial MSF web service account password? (Leave blank for random password):
Generating SSL key and certificate for MSF web service
Attempting to start MSF web service...success
MSF web service started and online
Creating MSF web service user *redacted*

    ############################################################
    ##              MSF Web Service Credentials               ##
    ##                                                        ##
    ##        Please store these credentials securely.        ##
    ##    You will need them to connect to the webservice.    ##
    ############################################################

MSF web service username: *redacted*
MSF web service password: *redacted*
MSF web service user API token: *redacted*

MSF web service configuration complete
The web service has been configured as your default data service in msfconsole with the name "local-https-data-service"

If needed, manually reconnect to the data service in msfconsole using the command:
db_connect --token *redacted* --cert *redacted* --skip-verify https://localhost:5443

The username and password are credentials for the API account:
https://localhost:5443/api/v1/auth/account
gwillcox-r7 commented 4 years ago

Okay so did some more testing on this today, seems like I still ran into problems cause I had to run export PGHOST=/tmp again as it hadn't stuck around.

What is weirder is that /home/gwillcox/.msf4/db/postgresql.conf seems to have the line unix_socket_directories = '/tmp' # comma-separated list of directories uncommented but its not being honored for some reason until that export PGHOST=/tmp command is run. Very weird.

akshaydrunk commented 3 years ago

mv /usr/lib/postgresql/12/bin/initdb /usr/local/bin/

mark-adams commented 3 years ago

Setting export PGHOST=/tmp worked for me but only after editing the postgresql.conf.sample and setting unix_socket_directories = '/tmp'.

gwillcox-r7 commented 3 years ago

Once https://github.com/rapid7/metasploit-framework/pull/15213 lands this PR should be resolved, the root issue is now resolved with #15196 and #15054, however there is still an issue when reinit'ing the database due to a requirement on the existence of a certain file that is deleted when deleting the database. #15213 should fix this issue.

gwillcox-r7 commented 3 years ago

This should now be resolved now that #15213 has been landed. @pastaoficial, @mchoji please let me know if you guys are still encountering errors after removing your workarounds and upgrading to the latest version of Metasploit with these fixes included and I'll be happy to reopen this issue and investigate further if the issue is still occurring.