Open ajaypaul-ibm opened 3 months ago
I think the root of this issue has to do with how EclipseLink saves date/time data to PostgreSQL. For the above test case the create query for the table was:
[eclipselink.sql] CREATE TABLE DEMOGRAPHICINFO (ID BIGINT NOT NULL, COLLECTEDON VARCHAR(255), INTRAGOVERNMENTALDEBT DECIMAL(38), NUMFULLTIMEWORKERS BIGINT, PUBLICDEBT DECIMAL(38), PRIMARY KEY (ID))
The instant collectedOn
was saved in PostgreSQL as VARCHAR(255)
instead of as a TIMESTAMP
.
So I can see two solutions:
Java Type | Column Type |
---|---|
Instant | TIMESTAMP |
LocalDate | DATE |
LocalTime | TIME |
OffsetTime | TIME WITH TIME ZONE |
OffsetDateTime | TIMESTAMP WITH TIME ZONE |
ZonedDateTime | TIMESTAMP WITH TIME ZONE |
NOTE: in this case ZonedDateTime stores zone offset and zone id data, PostgreSQL can store one or the other but not both.
Dates and Times | Java Type | PostgreSQL |
---|---|---|
Instant | EXTRACT( YEAR FROM COLLECTEDON::TIMESTAMP ) | |
LocalDate | EXTRACT( YEAR FROM COLLECTEDON::DATE ) | |
LocalTime | EXTRACT( HOUR FROM COLLECTEDON::TIME ) | |
OffsetTime | EXTRACT( HOUR FROM COLLECTEDON::TIME WITH TIME ZONE ) | |
OffsetDate | EXTRACT( YEAR FROM COLLECTEDON::TIMESTAMP WITH TIME ZONE ) | |
ZonedDateTime | IMPOSSIBLE |
NOTE: In this case ZonedDateTime is stored as 2007-12-03T10:15:30+01:00 Europe/Paris
which PostgreSQL cannot parse because it only supports the use of an offset or zoneId but not both.
@lukasj So EclipseLink does not support the EXTRACT keyword of JPQL without the PostgreSQL extension?
I would have expected EclipseLink to support basic JPQL with just the core library, and for the PostgreSQL extension to support data types specific to PostgreSQL. Similar to how the Oracle Extension supports things like oracle.sql.json.OracleJsonValue
Instant, LocalDate, LocalTime, OffsetTime, and OffsetData should be supported data types for time without the need for the PostgreSQL extension, IMHO.
@KyleAure Have I said anything about extract being or not being supported without the PostgreSQL extension?
So EclipseLink does not support the EXTRACT keyword of JPQL without the PostgreSQL extension?
Have you tried the same use-case on DB different from PostgreSQL?
Instant, LocalDate, LocalTime, OffsetTime, and OffsetData should be supported data types for time without the need for the PostgreSQL extension, IMHO.
Can all these be represented by the same datatypes in all supported DBs or are there differences?
Anyway, I do see two possible things to check here:
...both items are DB specific, so the DB specific support class seems like the best starting point for digging in. WDYT?
Have I said anything about extract being or not being supported without the PostgreSQL extension?
No sorry, what I meant by that statement was that it was not supported without an extension, but the exception thrown by EclipseLink does not make that clear.
The DatabaseException thrown by EclipseLink makes it seem like the EXTRACT
part of the SQL Query was malformed. Which it was because it was impossible to execute on the target database.
Therefore, any user would assume that EclipseLink is not correctly supporting EXTRACT
on PostgreSQL.
Have you tried the same use-case on DB different from PostgreSQL?
Yeah, this works on Derby, DB2, and MS SQLServer without any additional extensions. It seems this fails on Oracle and PostgreSQL without any additional extensions.
I ran the same test on MS SQLServer and found that the table is created with Instant
being stored as a VARCHAR(255)
similar to PostgreSQL:
CREATE TABLE DEMOGRAPHICINFO (ID NUMERIC(28) NOT NULL, COLLECTEDON VARCHAR(255) NULL, INTRAGOVERNMENTALDEBT NUMERIC(28) NULL, NUMFULLTIMEWORKERS NUMERIC(28) NULL, PUBLICDEBT NUMERIC(28) NULL, PRIMARY KEY (ID))
The SQL Query that is run is:
SELECT (PUBLICDEBT / NUMFULLTIMEWORKERS) FROM DEMOGRAPHICINFO WHERE (DATEPART(YEAR,COLLECTEDON) = ?)
Are java.time types mapped to DB types on PostgreSQL as expected?
No, but now that I am learning more about EclipseLink I do not think they should when not using the PostgreSQL extension.
When not using the extension I would expect the java.time types to be stored as VARCHAR255
in the database.
Which is what is currently happening.
Is there a need for PostgreSQL specific syntax for EXTRACT function?
Yes, because the database needs to cast the VARCHAR into a DATE/TIME object for time based functions to work. Therefore, the EXTRACT function needs to be customized for PostgreSQL (similar to what SQLServer does above).
But I would argue that this functionality should not be sequestered to the extension but rather be included in the base org.eclipse.persistence.platform.database.PostgreSQLPlatform class.
Thanks for sticking with me while I learn more about EclipseLink and JPA :D
This issue is seen in Oracle DB , hence updating the title.
This issue is seen in Oracle DB , hence updating the title. @ajaypaul-ibm There are some
...EXTRACT
tests https://github.com/eclipse-ee4j/eclipselink/blob/master/jpa/eclipselink.jpa.test.jse/src/it/java/org/eclipse/persistence/jpa/test/query/TestDateTimeFunctions.java Could You please be more specific about issue in Oracle DB. In the attachment You can find test results from JPA JSE test module and everything passing. jpa-jse-test-report.tar.gz
@rfelcman Similar to PostgreSQL, EclipseLink will save timestamp data to an Oracle DB as VARCHAR2(255) [When not using the extension]:
[eclipselink.query] Execute query DataModifyQuery(sql="CREATE TABLE DEMOGRAPHICINFO (ID NUMBER(38) NOT NULL, COLLECTEDON VARCHAR2(255) NULL, INTRAGOVERNMENTALDEBT NUMBER(38) NULL, NUMFULLTIMEWORKERS NUMBER(38) NULL, PUBLICDEBT NUMBER(38) NULL, PRIMARY KEY (ID))")
Then when attempting to extract the year from the date column providing the following JPQL query:
SELECT this.publicDebt / this.numFullTimeWorkers FROM DemographicInfo WHERE EXTRACT (YEAR FROM this.collectedOn) = ?1
Eclipselink generates an incorrect SQL query:
SELECT (PUBLICDEBT / NUMFULLTIMEWORKERS) FROM DEMOGRAPHICINFO WHERE (EXTRACT(YEAR FROM COLLECTEDON) = ?)
Resulting in the following exception:
Exception [EclipseLink-4002] (Eclipse Persistence Services - 5.0.0-B03.v202409121024-4a7149f0cd04d7466837d70f68abb743c88acb83): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: ORA-30076: invalid extract field for extract source
https://docs.oracle.com/error-help/db/ora-30076/
Error Code: 30076
Call: SELECT (PUBLICDEBT / NUMFULLTIMEWORKERS) FROM DEMOGRAPHICINFO WHERE (EXTRACT(YEAR FROM COLLECTEDON) = ?)
bind => [2024]
Query: ReportQuery(referenceClass=DemographicInfo sql="SELECT (PUBLICDEBT / NUMFULLTIMEWORKERS) FROM DEMOGRAPHICINFO WHERE (EXTRACT(YEAR FROM COLLECTEDON) = ?)")
[10/2/24, 8:42:05:508 CDT] 00000050 id=00000000 eclipselink 3 throwable
Local Exception Stack:
Exception [EclipseLink-4002] (Eclipse Persistence Services - 5.0.0-B03.v202409121024-4a7149f0cd04d7466837d70f68abb743c88acb83): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: ORA-30076: invalid extract field for extract source
https://docs.oracle.com/error-help/db/ora-30076/
Error Code: 30076
Call: SELECT (PUBLICDEBT / NUMFULLTIMEWORKERS) FROM DEMOGRAPHICINFO WHERE (EXTRACT(YEAR FROM COLLECTEDON) = ?)
bind => [2024]
Query: ReportQuery(referenceClass=DemographicInfo sql="SELECT (PUBLICDEBT / NUMFULLTIMEWORKERS) FROM DEMOGRAPHICINFO WHERE (EXTRACT(YEAR FROM COLLECTEDON) = ?)")
at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:346)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:702)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:569)
at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:2053)
at org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:611)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:282)
The correct query should have been:
SELECT (PUBLICDEBT / NUMFULLTIMEWORKERS)
FROM DEMOGRAPHICINFO
WHERE (EXTRACT(YEAR FROM(TO_DATE( COLLECTEDON , 'yyyy-mm-dd"T"hh24:mi:ss"Z"' ))) = ?)
I re-tested DDL generation against Oracle 23C and PostgreSQL with following entity:
package com.oracle.jpa.bugtest;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import java.time.LocalDateTime;
import java.util.UUID;
@Entity
@Table(name = "TEST_TAB_DDL")
public class DDLEntity {
private UUID id;
private LocalDateTime simpleLocalDateTime;
public DDLEntity() {
}
public DDLEntity(UUID id, LocalDateTime simpleLocalDateTime) {
this.id = id;
this.simpleLocalDateTime = simpleLocalDateTime;
}
@Id
public UUID getId() {
return id;
}
public void setId(UUID id) {
this.id = id;
}
public LocalDateTime getSimpleLocalDateTime() {
return simpleLocalDateTime;
}
public void setSimpleLocalDateTime(LocalDateTime simpleLocalDateTime) {
this.simpleLocalDateTime = simpleLocalDateTime;
}
}
and for the Oracle 23C DDL script is
CREATE TABLE TEST_TAB_DDL (ID VARCHAR2(255) NOT NULL, SIMPLELOCALDATETIME TIMESTAMP(9) NULL, PRIMARY KEY (ID))
for the PostgreSQL DDL script is
CREATE TABLE TEST_TAB_DDL (ID VARCHAR(255) NOT NULL, SIMPLELOCALDATETIME TIMESTAMP, PRIMARY KEY (ID))
so I don't see any issue there as expected TIMESTAMP
DB table column type is generated.
@rfelcman
Where you using drop-and-create
on your persistence unit?
Where you using the Oracle (org.eclipse.persistence.oracle) and PostgreSQL (org.eclipse.persistence.pgsql) extensions?
It's about persistence unit in persistence.xml
<persistence-unit name="test-jpa-pu-ddl" transaction-type="RESOURCE_LOCAL">
<provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
<class>com.oracle.jpa.bugtest.DDLEntity</class>
<exclude-unlisted-classes>true</exclude-unlisted-classes>
<properties>
<property name="eclipselink.logging.level" value="FINEST"/>
<property name="eclipselink.logging.level.jpars" value="FINEST"/>
<property name="eclipselink.logging.parameters" value="true"/>
<property name="jakarta.persistence.schema-generation.database.action" value="drop-and-create"/>
<property name="jakarta.persistence.schema-generation.scripts.action" value="drop-and-create"/>
<property name="jakarta.persistence.schema-generation.create-source" value="metadata-then-script"/>
<property name="jakarta.persistence.schema-generation.drop-source" value="metadata-then-script"/>
<property name="jakarta.persistence.schema-generation.scripts.create-target" value="target/create_ddlentity.sql"/>
<property name="jakarta.persistence.schema-generation.scripts.drop-target" value="target/drop_ddlentity.sql"/>
<!--POSTGRESQL-->
<property name="jakarta.persistence.jdbc.driver" value="org.postgresql.Driver"/>
<property name="jakarta.persistence.jdbc.url" value="jdbc:postgresql://localhost:5432/ecltests"/>
<property name="jakarta.persistence.jdbc.user" value="********"/>
<property name="jakarta.persistence.jdbc.password" value="********"/>
<!--ORACLE-->
<!--
<property name="jakarta.persistence.jdbc.driver" value="oracle.jdbc.OracleDriver"/>
<property name="jakarta.persistence.jdbc.url" value="jdbc:oracle:thin:@localhost:1521/FREEPDB1"/>
<property name="jakarta.persistence.jdbc.user" value="********"/>
<property name="jakarta.persistence.jdbc.password" value="********"/>
-->
</properties>
</persistence-unit>
property https://eclipse.dev/eclipselink/documentation/4.0/jpa/extensions/jpa-extensions.html#target-database like
<property name="eclipselink.target-database" value="Oracle23"/>
or
<property name="eclipselink.target-database" value="org.eclipse.persistence.platform.database.Oracle23"/>
wasn't needed as auto-detection works see log messages like
....
[EL Finest]: connection: 2024.10.17 19:59:39.887--Thread(Thread[#1,main,5,main])--Database platform: org.eclipse.persistence.platform.database.MariaDBPlatform, regular expression: (?i)mariadb.*
[EL Finest]: connection: 2024.10.17 19:59:39.887--Thread(Thread[#1,main,5,main])--Database platform: org.eclipse.persistence.platform.database.Informix11Platform, regular expression: (?i)(informix.*)|(ids.*)
[EL Finest]: connection: 2024.10.17 19:59:39.887--Thread(Thread[#1,main,5,main])--Database platform: org.eclipse.persistence.platform.database.PostgreSQL10Platform, regular expression: (?is)postgresql.*14.*
[EL Finest]: connection: 2024.10.17 19:59:39.887--Thread(Thread[#1,main,5,main])--Database platform: org.eclipse.persistence.platform.database.PostgreSQL10Platform, regular expression: (?is)postgresql.*13.*
[EL Fine]: connection: 2024.10.17 19:59:39.888--Thread(Thread[#1,main,5,main])--Detected database platform: org.eclipse.persistence.platform.database.PostgreSQL10Platform
....
@rfelcman By using ddl files to create the schema:
<property name="jakarta.persistence.schema-generation.scripts.create-target" value="target/create_ddlentity.sql"/>
<property name="jakarta.persistence.schema-generation.scripts.drop-target" value="target/drop_ddlentity.sql"/>
You are not replicating this issue. Here is the persistence unit I am trying to use:
<persistence ...>
<persistence-unit name="RecreatePersistenceUnit">
...
<properties>
<!-- EclipseLink should create the database schema automatically -->
<property name="jakarta.persistence.schema-generation.database.action" value="drop-and-create" />
<property name="eclipselink.logging.parameters" value="true"/>
</properties>
</persistence-unit>
</persistence>
Eclipse link can correctly identify the target database for both Oracle and PostgreSQL. But what EclipseLink fails to do is to generate a valid SQL query given the type of column it is using to store timestamp data.
Sorry I don't understand Your comment about:
<property name="jakarta.persistence.schema-generation.scripts.create-target" value="target/create_ddlentity.sql"/>
<property name="jakarta.persistence.schema-generation.scripts.drop-target" value="target/drop_ddlentity.sql"/>
These properties are for the script output and generated by EclipseLink. Not as the input to generate DB schema. Properties
<property name="jakarta.persistence.schema-generation.create-source" value="metadata-then-script"/>
<property name="jakarta.persistence.schema-generation.drop-source" value="metadata-then-script"/>
says drop/generate DB schema and additionally scripts specified above.
https://jakarta.ee/specifications/persistence/3.2/jakarta-persistence-spec-3.2#a12384
Thanks for the information. I was just providing information on how you might replicate this issue. I am not sure why the tables being created using DDL vs the table that is created in my environment are different.
EclipseLink is generated a SQL query from JPQL to PostgreSQL that is rejected by the database.
For example, the JPQL SELECT this.publicDebt / this.numFullTimeWorkers FROM DemographicInfo WHERE EXTRACT (YEAR FROM this.collectedOn) = ?1
The PostgreSQL JDBC driver throws the exception: org.postgresql.util.PSQLException: ERROR: function pg_catalog.extract(unknown, character varying) does not exist
Full exception stack: