search

zonkyio / embedded-postgres

Java embedded PostgreSQL component for testing
Apache License 2.0
341 stars 41 forks source link

embedded-postgres issue in Apple Silicon (M1)

 #115

Closed Vivek-M-08 closed 1 year ago

Vivek-M-08 commented 1 year ago

The project i am working on is using the following dependency:

<dependency>
    <groupId>io.zonky.test</groupId>
    <artifactId>embedded-postgres</artifactId>
    <version>1.2.6</version>
    <scope>test</scope>
</dependency>

But while executing the test cases i am getting the following errors Error: Missing postgres binaries

Exception encountered when invoking run on a nested suite - Missing postgres binaries
java.lang.IllegalStateException: Missing postgres binaries

Attempted Solution: Dependency Upgrade

To resolve the missing binaries issue, an attempt was made to upgrade the embedded-postgres dependency to the latest version:

<dependency>
    <groupId>io.zonky.test</groupId>
    <artifactId>embedded-postgres</artifactId>
    <version>2.0.4</version>
    <scope>test</scope>
</dependency>

New Error: Process failed during initdb

After upgrading the dependency, a new error is encountered:

Exception encountered when invoking run on a nested suite - Process [/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-95baeb96ab6751ea71c17b280cad95e9/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6492089817289658487, -E, UTF-8] failed
java.lang.IllegalStateException: Process [/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-95baeb96ab6751ea71c17b280cad95e9/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6492089817289658487, -E, UTF-8] failed

Please help to resolve this issue.

tomix26 commented 1 year ago

Try to add this dependency: https://mvnrepository.com/artifact/io.zonky.test.postgres/embedded-postgres-binaries-darwin-arm64v8

Vivek-M-08 commented 1 year ago

@tomix26 The issue still persist even after adding the above mentioned dependency. 

Exception encountered when invoking run on a nested suite - Process [/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg16314842684659684511, -E, UTF-8] failed
java.lang.IllegalStateException: Process [/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg16314842684659684511, -E, UTF-8] failed
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.system(EmbeddedPostgres.java:633)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.initdb(EmbeddedPostgres.java:250)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.<init>(EmbeddedPostgres.java:157)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres$Builder.start(EmbeddedPostgres.java:584)
    at org.ekstep.analytics.util.EmbeddedPostgresql$.start(EmbeddedPostgresql.scala:15)
    at org.ekstep.analytics.exhaust.TestOnDemandDruidExhaustJob.beforeAll(TestOnDemandDruidExhaustJob.scala:49)
    at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
    at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
    at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
    at org.ekstep.analytics.util.BaseSpec.run(BaseSpec.scala:8)
    at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
    at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1346)
    at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1340)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1340)
    at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:1031)
    at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:1010)
    at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1506)
    at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
    at org.scalatest.tools.Runner$.run(Runner.scala:850)
    at org.scalatest.tools.Runner.run(Runner.scala)
    at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2or3(ScalaTestRunner.java:43)
    at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:26)
Vivek-M-08 commented 1 year ago

@tomix26 I can see an error in this file.

Screenshot 2023-07-06 at 9 42 09 PM
tomix26 commented 1 year ago

It seems to me like an issues with a postgres driver. What version of the driver are you using?

Nevertheless, first I would recommend to try the previous working version of embedded postgres 1.2.6 together with the embedded-postgres-binaries-darwin-arm64v8 dependency. After that step you can try to upgrade to a newer version of embedded postgres. There is a changelog here: https://github.com/zonkyio/embedded-postgres/releases/tag/v2.0.0

If you encounter any other problems, please send me the full log, not images, thank you.

Vivek-M-08 commented 1 year ago

Hi @tomix26

I have adding both the dependency as suggested.

<dependency>
    <groupId>io.zonky.test</groupId>
    <artifactId>embedded-postgres</artifactId>
    <version>2.0.4</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>io.zonky.test.postgres</groupId>
    <artifactId>embedded-postgres-binaries-darwin-arm64v8</artifactId>
    <version>15.3.0</version>
</dependency>

However, I continue to see the same error; please view the following error log. 

Exception encountered when invoking run on a nested suite - Process [/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg2326912685824662426, -E, UTF-8] failed
java.lang.IllegalStateException: Process [/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg2326912685824662426, -E, UTF-8] failed
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.system(EmbeddedPostgres.java:633)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.initdb(EmbeddedPostgres.java:250)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.<init>(EmbeddedPostgres.java:157)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres$Builder.start(EmbeddedPostgres.java:584)
    at org.ekstep.analytics.util.EmbeddedPostgresql$.start(EmbeddedPostgresql.scala:15)
    at org.ekstep.analytics.exhaust.TestOnDemandDruidExhaustJob.beforeAll(TestOnDemandDruidExhaustJob.scala:49)
    at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
    at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
    at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
    at org.ekstep.analytics.util.BaseSpec.run(BaseSpec.scala:8)
    at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
    at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1346)
    at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1340)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1340)
    at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:1031)
    at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:1010)
    at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1506)
    at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
    at org.scalatest.tools.Runner$.run(Runner.scala:850)
    at org.scalatest.tools.Runner.run(Runner.scala)
    at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2or3(ScalaTestRunner.java:43)
    at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:26)

By visiting this link, https://github.com/zonkyio/embedded-postgres/issues/78, I too attempted to comprehend the problem. appending the command line output for a couple commands for the directory on my local that is causing the problem.

user@PWA177 ~ % cd /private/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin
user@PWA177 bin % ls
initdb      pg_ctl      postgres
user@PWA177 bin % ./postgres
postgres does not know where to find the server configuration file.
You must specify the --config-file or -D invocation option or set the PGDATA environment variable.
user@PWA177 bin % file postgres
postgres: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64]
postgres (for architecture x86_64): Mach-O 64-bit executable x86_64
postgres (for architecture arm64):  Mach-O 64-bit executable arm64
user@PWA177 bin % uname -a
Darwin PWA177.local 22.5.0 Darwin Kernel Version 22.5.0: Thu Jun  8 22:22:19 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8103 arm64
user@PWA177 bin % ./initdb -V
initdb (PostgreSQL) 15.3
user@PWA177 bin % ./pg_ctl -V
pg_ctl (PostgreSQL) 15.3
tomix26 commented 1 year ago

Have you tried embedded-postgres 1.2.6 ?

Vivek-M-08 commented 1 year ago

Yes i have tried embedded-postgres 1.2.6 with embedded-postgres-binaries-darwin-arm64v8 dependency and got the same error as mentioned above.

But when i try embedded-postgres 1.2.6 dependency alone. I get the following error

Exception encountered when invoking run on a nested suite - Missing postgres binaries
java.lang.IllegalStateException: Missing postgres binaries
tomix26 commented 1 year ago

Ok, great, good to know.

Try to run the binaries manually with the following command in the /private/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin directory: pg_ctl -D ./data start

The output should give you more information about what is wrong and why the database is not starting.

Vivek-M-08 commented 1 year ago

I tried to execute the above mentioned command in the specified directory, it says that command not found.

user@PWA177 ~ % cd /private/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin

user@PWA177 bin % ls
initdb      pg_ctl      postgres

user@PWA177 bin % pg_ctl -D ./data start
zsh: command not found: pg_ctl

As all the three files[initdb, pg_ctl, postgres] inside this directory are Unix Executable Files. I tried with different commands.

user@PWA177 bin %  ./pg_ctl -V       
pg_ctl (PostgreSQL) 15.3

user@PWA177 bin %  ./pg_ctl start    
pg_ctl: no database directory specified and environment variable PGDATA unset
Try "pg_ctl --help" for more information.

user@PWA177 bin % ./pg_ctl -D ./data start
pg_ctl: directory "data" does not exist
tomix26 commented 1 year ago

What about mkdir ./data && ./pg_ctl -D ./data start ?

Vivek-M-08 commented 1 year ago

Should i add any config related to database cluster in the data directory 

user@PWA177 bin % mkdir ./data && ./pg_ctl -D ./data start
pg_ctl: directory "data" is not a database cluster directory

user@PWA177 bin % ls                                      
data        initdb      pg_ctl      postgres
tomix26 commented 1 year ago

Yes, you're right. It's been a while since I last tried it. The following command should work: mkdir ./data && ./bin/initdb -D ./data && ./bin/pg_ctl -D ./data start

Vivek-M-08 commented 1 year ago

Both commands were successful, and I attached the console logs as well. what can be the next steps?

user@PWA177 bin % ./initdb -D ./data 
The files belonging to this database system will be owned by user "user".
This user must also own the server process.

The database cluster will be initialized with this locale configuration:
  provider:    libc
  LC_COLLATE:  C
  LC_CTYPE:    UTF-8
  LC_MESSAGES: C
  LC_MONETARY: C
  LC_NUMERIC:  C
  LC_TIME:     C
The default database encoding has accordingly been set to "UTF8".
initdb: could not find suitable text search configuration for locale "UTF-8"
The default text search configuration will be set to "simple".

Data page checksums are disabled.

fixing permissions on existing directory data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Asia/Kolkata
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    pg_ctl -D ./data -l logfile start
user@PWA177 bin % ./pg_ctl -D ./data start 
waiting for server to start....2023-07-12 22:37:21.341 IST [11590] LOG:  starting PostgreSQL 15.3 on x86_64-apple-darwin20.6.0, compiled by Apple clang version 12.0.0 (clang-1200.0.32.29), 64-bit
2023-07-12 22:37:21.342 IST [11590] LOG:  listening on IPv6 address "::1", port 5432
2023-07-12 22:37:21.342 IST [11590] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2023-07-12 22:37:21.343 IST [11590] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2023-07-12 22:37:21.345 IST [11593] LOG:  database system was shut down at 2023-07-12 22:35:28 IST
2023-07-12 22:37:21.347 IST [11590] LOG:  database system is ready to accept connections
 done
server started
tomix26 commented 1 year ago

Ok, so the database is starting fine. Now run your tests again and send me the full log. If there are any potentially sensitive information in the log you can send it to my private email: tomix26@gmail.com

Vivek-M-08 commented 1 year ago

Still i can see the same error log.

Testing started at 9:02 pm ...

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/user/.m2/repository/org/sunbird/analytics-job-driver/2.0/analytics-job-driver-2.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/user/.m2/repository/org/slf4j/slf4j-log4j12/1.7.30/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/user/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/Users/user/.m2/repository/org/apache/spark/spark-unsafe_2.12/3.1.0/spark-unsafe_2.12-3.1.0.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Exception encountered when invoking run on a nested suite - Process [/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6929773607312554948, -E, UTF-8] failed
java.lang.IllegalStateException: Process [/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6929773607312554948, -E, UTF-8] failed
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.system(EmbeddedPostgres.java:633)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.initdb(EmbeddedPostgres.java:250)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.<init>(EmbeddedPostgres.java:157)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres$Builder.start(EmbeddedPostgres.java:584)
    at org.ekstep.analytics.util.EmbeddedPostgresql$.start(EmbeddedPostgresql.scala:15)
    at org.ekstep.analytics.exhaust.TestOnDemandDruidExhaustJob.beforeAll(TestOnDemandDruidExhaustJob.scala:49)
    at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
    at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
    at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
    at org.ekstep.analytics.util.BaseSpec.run(BaseSpec.scala:8)
    at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
    at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1346)
    at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1340)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1340)
    at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:1031)
    at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:1010)
    at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1506)
    at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
    at org.scalatest.tools.Runner$.run(Runner.scala:850)
    at org.scalatest.tools.Runner.run(Runner.scala)
    at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2or3(ScalaTestRunner.java:43)
    at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:26)

Process finished with exit code 0

I checked the server status as well

user@PWA177 bin % ./pg_ctl -D ./data start 
waiting for server to start....2023-07-13 21:10:09.030 IST [15430] LOG:  starting PostgreSQL 15.3 on x86_64-apple-darwin20.6.0, compiled by Apple clang version 12.0.0 (clang-1200.0.32.29), 64-bit
2023-07-13 21:10:09.031 IST [15430] LOG:  listening on IPv6 address "::1", port 5432
2023-07-13 21:10:09.031 IST [15430] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2023-07-13 21:10:09.032 IST [15430] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2023-07-13 21:10:09.035 IST [15433] LOG:  database system was shut down at 2023-07-13 21:02:25 IST
2023-07-13 21:10:09.039 IST [15430] LOG:  database system is ready to accept connections
 done
server started

user@PWA177 bin % ./pg_ctl -D ./data status
pg_ctl: server is running (PID: 15430)
/private/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/postgres "-D" "data"
tomix26 commented 1 year ago

Run this command: /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/initdb -A trust -U postgres -D /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6929773607312554948 -E UTF-8

Vivek-M-08 commented 1 year ago

This command successful created database cluster configuration.

user@PWA177 bin % /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/initdb -A trust -U postgres -D /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6929773607312554948 -E UTF-8
The files belonging to this database system will be owned by user "user".
This user must also own the server process.

The database cluster will be initialized with this locale configuration:
  provider:    libc
  LC_COLLATE:  C
  LC_CTYPE:    UTF-8
  LC_MESSAGES: C
  LC_MONETARY: C
  LC_NUMERIC:  C
  LC_TIME:     C
initdb: could not find suitable text search configuration for locale "UTF-8"
The default text search configuration will be set to "simple".

Data page checksums are disabled.

fixing permissions on existing directory /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6929773607312554948 ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Asia/Kolkata
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

Success. You can now start the database server using:

    /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/pg_ctl -D /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6929773607312554948 -l logfile start

Below command started the server. 

user@PWA177 bin % /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/pg_ctl -D /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6929773607312554948 -l logfile start
waiting for server to start.... done
server started

user@PWA177 bin % ls
data        initdb      logfile     pg_ctl      postgres

user@PWA177 bin % cat logfile 
2023-07-13 22:40:23.744 IST [17313] LOG:  starting PostgreSQL 15.3 on x86_64-apple-darwin20.6.0, compiled by Apple clang version 12.0.0 (clang-1200.0.32.29), 64-bit
2023-07-13 22:40:23.745 IST [17313] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2023-07-13 22:40:23.745 IST [17313] LOG:  listening on IPv6 address "::1", port 5432
2023-07-13 22:40:23.746 IST [17313] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2023-07-13 22:40:23.748 IST [17316] LOG:  database system was shut down at 2023-07-13 22:35:11 IST
2023-07-13 22:40:23.749 IST [17313] LOG:  database system is ready to accept connections

But still i am not able to execute the test case

Testing started at 10:43 pm ...

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/user/.m2/repository/org/sunbird/analytics-job-driver/2.0/analytics-job-driver-2.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/user/.m2/repository/org/slf4j/slf4j-log4j12/1.7.30/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/user/.m2/repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/Users/user/.m2/repository/org/apache/spark/spark-unsafe_2.12/3.1.0/spark-unsafe_2.12-3.1.0.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Exception encountered when invoking run on a nested suite - Process [/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6221457694258232048, -E, UTF-8] failed
java.lang.IllegalStateException: Process [/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6221457694258232048, -E, UTF-8] failed
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.system(EmbeddedPostgres.java:633)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.initdb(EmbeddedPostgres.java:250)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres.<init>(EmbeddedPostgres.java:157)
    at io.zonky.test.db.postgres.embedded.EmbeddedPostgres$Builder.start(EmbeddedPostgres.java:584)
    at org.ekstep.analytics.util.EmbeddedPostgresql$.start(EmbeddedPostgresql.scala:15)
    at org.ekstep.analytics.exhaust.TestOnDemandDruidExhaustJob.beforeAll(TestOnDemandDruidExhaustJob.scala:49)
    at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
    at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
    at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
    at org.ekstep.analytics.util.BaseSpec.run(BaseSpec.scala:8)
    at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
    at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1346)
    at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1340)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1340)
    at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:1031)
    at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:1010)
    at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1506)
    at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
    at org.scalatest.tools.Runner$.run(Runner.scala:850)
    at org.scalatest.tools.Runner.run(Runner.scala)
    at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2or3(ScalaTestRunner.java:43)
    at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:26)

Process finished with exit code 0
tomix26 commented 1 year ago

Then I guess it is probably a permissions issue. Are the tests running under the same user as the successfully completed commands above?

Vivek-M-08 commented 1 year ago

Yes both test-cases and successfully completed commands above are under same user.

Vivek-M-08 commented 1 year ago

@tomix26 can you help me here.

tomix26 commented 1 year ago

Next, you can try to call the command directly from the code, like below. Because that is what the library does. There is the corresponding code here: EmbeddedPostgres.java#L241

ProcessBuilder builder = new ProcessBuilder("/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/embedded-pg/PG-16f255bae154432086b16236cfb59f2e/bin/initdb", "-A", "trust", "-U", "postgres", "-D", "/var/folders/f7/kt5z7zpj61v5b757h_wxcyh80000gn/T/epg6929773607312554948", "-E", "UTF-8");
Process process = builder.start();
// Extract and verify the process output, there is more information here: https://docs.oracle.com/javase/8/docs/api/java/lang/ProcessBuilder.html

You can also try to check out this repo and run the included tests to rule out that Scala is the cause of the problem. Because Scala is not officially supported yet and some problems have been reported in relation to the SBT server https://github.com/zonkyio/embedded-postgres/pull/65. Maybe the SBT server runs with a different user with different permissions. I don't know, I don't use Scala, I have limited experience in this area, so I can't help more.

As a last resort, I would try to debug the code. EmbeddedPostgres.java#L241 is a good starting point. I guess that if you put a thread based breakpoint here EmbeddedPostgres.java#L632, the database error could be printed out to the log. Because logging is done asynchronously in a different thread and probably does not have enough time to print out the error.

Vivek-M-08 commented 1 year ago

Hi @tomix26, this issue has been resloved.

You were correct that the test cases were not running under the same user with administrative access; Every time I run IntelliJ using the Mac launchpad, it was unable to obtain admin access. As a result, I was not able to execute the test case.

Solution: After starting IntelliJ by executing the .sh file located at this path: cd /Applications/IntelliJ IDEA CE.app/Contents/MacOS/ via terminal. I was able to execute the test cases.

Thank you for your support.

tomix26 commented 1 year ago

Glad to hear that. Thanks for letting me know.