search

yandex-qatools / postgresql-embedded

Embedded PostgreSQL Server
Other
494 stars 90 forks source link

AbstractPGProcess.<init> throws NullPointerException on Windows because PackagePaths.getFileSet returns incorrect pattern #81

Closed KoichiSenada closed 7 years ago

KoichiSenada commented 7 years ago

I have tested it with postgresql-embedded-2.0 on Windows 10 x64, which downloads the following ZIP file. http://get.enterprisedb.com/postgresql/postgresql-9.5.5-1-windows-x64-binaries.zip

On Windows method ru.yandex.qatools.embed.postgresql.PackagePaths.getFileSet() creates incorrect pattern ^.*pgsql\\bin\\postgres.exe$

at de.flapdoodle.embed.process.config.store.FileSet$Builder.addEntry(FileSet.java:119)
at ru.yandex.qatools.embed.postgresql.PackagePaths.getFileSet(PackagePaths.java:51)
at de.flapdoodle.embed.process.store.CachedPostgresArtifactStore.extractFileSet(CachedPostgresArtifactStore.java:46)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:57)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:129)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

Then method de.flapdoodle.embed.process.extract.AbstractExtractor.extract() extracts a org.apache.commons.compress.archivers.zip.ZipFile$Entry which ZipArchiveEntry.name field correctly equals to pgsql/bin/postgres.exe

at de.flapdoodle.embed.process.extract.AbstractExtractor.extract(AbstractExtractor.java:71)
at de.flapdoodle.embed.process.store.PostgresArtifactStore.extractFileSet(PostgresArtifactStore.java:76)
at de.flapdoodle.embed.process.store.CachedPostgresArtifactStore.extractFileSet(CachedPostgresArtifactStore.java:60)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:57)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:129)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

But that is a mismatch because of which the de.flapdoodle.embed.process.store.PostgresFilesToExtract$1.type() method returns FileType.Library instead of the expected FileType.Executable

at de.flapdoodle.embed.process.store.PostgresFilesToExtract$1.type(PostgresFilesToExtract.java:105)
at de.flapdoodle.embed.process.extract.AbstractExtractor.extract(AbstractExtractor.java:75)
at de.flapdoodle.embed.process.store.PostgresArtifactStore.extractFileSet(PostgresArtifactStore.java:76)
at de.flapdoodle.embed.process.store.CachedPostgresArtifactStore.extractFileSet(CachedPostgresArtifactStore.java:60)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:57)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:129)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

Thus, the de.flapdoodle.embed.process.extract.AbstractExtractor.extract() method has created an instance of de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet$Builder with the _executable field still being null, because of which the ImmutableExtractedFileSet instance constructor throws NullPointerException

at java.lang.NullPointerException.<init>(NullPointerException.java:70)
at de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet.<init>(ImmutableExtractedFileSet.java:45)
at de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet$Builder.build(ImmutableExtractedFileSet.java:122)
at de.flapdoodle.embed.process.extract.AbstractExtractor.extract(AbstractExtractor.java:92)
at de.flapdoodle.embed.process.store.PostgresArtifactStore.extractFileSet(PostgresArtifactStore.java:76)
at de.flapdoodle.embed.process.store.CachedPostgresArtifactStore.extractFileSet(CachedPostgresArtifactStore.java:60)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:57)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:129)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

The NullPointerException is caught by the de.flapdoodle.embed.process.store.PostgresArtifactStore.extractFileSet() method, which then returns de.flapdoodle.embed.process.store.EmptyFileSet instead of the expected de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet

at de.flapdoodle.embed.process.store.PostgresArtifactStore.extractFileSet(PostgresArtifactStore.java:80)
at de.flapdoodle.embed.process.store.CachedPostgresArtifactStore.extractFileSet(CachedPostgresArtifactStore.java:60)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:57)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:129)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

That is how the EmbeddedPostgres.start() method creates an instance of ru.yandex.qatools.embed.postgresql.PostgresExecutable with field de.flapdoodle.embed.process.runtime.Executable.executable that contains the unexpected EmptyFileSet instead of the expected ImmutableExtractedFileSet, then it attempts to execute it.

at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:130)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

Because of that the ru.yandex.qatools.embed.postgresql.AbstractPGProcess instance constructor runs the EmptyFileSet.executable() method which always returns null instead of a java.io.File value, which is actually always a bad coding style.

at de.flapdoodle.embed.process.runtime.AbstractProcess.<init>(AbstractProcess.java:72)
at ru.yandex.qatools.embed.postgresql.AbstractPGProcess.<init>(AbstractPGProcess.java:19)
at ru.yandex.qatools.embed.postgresql.PostgresProcess.<init>(PostgresProcess.java:67)
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:25)
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:13)
at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:101)
- locked <0x29fb> (a ru.yandex.qatools.embed.postgresql.PostgresExecutable)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:130)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

The received null value is then passed to the de.flapdoodle.embed.process.runtime.AbstractProcess.pidFile() method which was not designed to accept it, so it throws the originally mentioned NullPointerException which is never caught anywhere since then.

at java.lang.NullPointerException.<init>(NullPointerException.java:60)
at de.flapdoodle.embed.process.runtime.AbstractProcess.pidFile(AbstractProcess.java:125)
at de.flapdoodle.embed.process.runtime.AbstractProcess.<init>(AbstractProcess.java:72)
at ru.yandex.qatools.embed.postgresql.AbstractPGProcess.<init>(AbstractPGProcess.java:19)
at ru.yandex.qatools.embed.postgresql.PostgresProcess.<init>(PostgresProcess.java:67)
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:25)
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:13)
at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:101)
- locked <0x29fb> (a ru.yandex.qatools.embed.postgresql.PostgresExecutable)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:130)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

So I suggest that you fix the incorrect executable zip file entry pattern generation in your method ru.yandex.qatools.embed.postgresql.PackagePaths.getFileSet() and probably consider replacing or fixing the bad code style de.flapdoodle.embed.process library which has methods that return null values and passes it to its own methods while they are not designed for accepting null values.

smecsia commented 7 years ago

@KoichiSenada Thanks! Can you please check the version from #82 ?

KoichiSenada commented 7 years ago

@smecsia , thank you, I have checked out the pull request #82

git fetch origin refs/pull/82/head:pull_82
git checkout pull_82

But the mvn install command fails with the same NullPointerException at the same method de.flapdoodle.embed.process.runtime.AbstractProcess.pidFile called from the ru.yandex.qatools.embed.postgresql.AbstractPGProcess.<init> constructor.

2017-05-15 12:42:25 DEBUG Files:120 - could delete C:\Users\UserName\AppData\Local\Temp\postgresql-embed-22609af5-4735-402c-9bc2-a09f5c5cabaf\db-content-f7cf8ccf-41f5-4db6-842d-c42a6d856211
2017-05-15 12:42:25 WARN  AbstractProcess:174 - Could not delete pid file: C:\Users\UserName\AppData\Local\Temp\postgresql-embed-22609af5-4735-402c-9bc2-a09f5c5cabaf\pgsql\bin\pg_ctl.pid
2017-05-15 12:42:26 ERROR CachedPostgresArtifactStore:63 - Failed to extract file set
java.lang.NullPointerException: executable is NULL
    at de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet.<init>(ImmutableExtractedFileSet.java:45)[de.flapdoodle.embed.process-2.0.1.jar:]
    at de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet$Builder.build(ImmutableExtractedFileSet.java:122)[de.flapdoodle.embed.process-2.0.1.jar:]
    at de.flapdoodle.embed.process.store.CachedPostgresArtifactStore.extractFileSet(CachedPostgresArtifactStore.java:58)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:57)[de.flapdoodle.embed.process-2.0.1.jar:]
    at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)[de.flapdoodle.embed.process-2.0.1.jar:]
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.runCmd(PostgresProcess.java:117)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.shutdownPostgres(PostgresProcess.java:132)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.sendStopToPostgresqlInstance(PostgresProcess.java:170)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.stopInternal(PostgresProcess.java:144)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at de.flapdoodle.embed.process.runtime.AbstractProcess.stop(AbstractProcess.java:170)[de.flapdoodle.embed.process-2.0.1.jar:]
    at de.flapdoodle.embed.process.runtime.AbstractProcess$JobKiller.run(AbstractProcess.java:243)[de.flapdoodle.embed.process-2.0.1.jar:]
    at java.lang.Thread.run(Thread.java:748)[:1.8.0_131]
2017-05-15 12:42:26 TRACE PostgresProcess:134 - Failed to stop postgres by pg_ctl!
java.lang.NullPointerException
    at de.flapdoodle.embed.process.runtime.AbstractProcess.pidFile(AbstractProcess.java:125)[de.flapdoodle.embed.process-2.0.1.jar:]
    at de.flapdoodle.embed.process.runtime.AbstractProcess.<init>(AbstractProcess.java:72)[de.flapdoodle.embed.process-2.0.1.jar:]
    at ru.yandex.qatools.embed.postgresql.AbstractPGProcess.<init>(AbstractPGProcess.java:19)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at ru.yandex.qatools.embed.postgresql.PgCtlProcess.<init>(PgCtlProcess.java:21)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at ru.yandex.qatools.embed.postgresql.PgCtlExecutable.start(PgCtlExecutable.java:24)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at ru.yandex.qatools.embed.postgresql.PgCtlExecutable.start(PgCtlExecutable.java:14)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:101)[de.flapdoodle.embed.process-2.0.1.jar:]
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.runCmd(PostgresProcess.java:118)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.shutdownPostgres(PostgresProcess.java:132)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.sendStopToPostgresqlInstance(PostgresProcess.java:170)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.stopInternal(PostgresProcess.java:144)[file:/C:/Users/UserName/Documents/github.com/yandex-qatools/postgresql-embedded/target/classes/:]
    at de.flapdoodle.embed.process.runtime.AbstractProcess.stop(AbstractProcess.java:170)[de.flapdoodle.embed.process-2.0.1.jar:]
    at de.flapdoodle.embed.process.runtime.AbstractProcess$JobKiller.run(AbstractProcess.java:243)[de.flapdoodle.embed.process-2.0.1.jar:]
    at java.lang.Thread.run(Thread.java:748)[:1.8.0_131]

Though, I have managed to install the snapshot while skipping tests with command mvn install -DskipTests, the originally mentioned code with it now runs fine. But the exception in ru.yandex.qatools.embed.postgresql.PgCtlExecutable and ru.yandex.qatools.embed.postgresql.PgCtlProcess is still a bug. Can you please fix it as well?

smecsia commented 7 years ago

@KoichiSenada can you please check version 2.2?

KoichiSenada commented 7 years ago

@smecsia , thank you, that helped! This case is now solved.