search

vmware-archive / pxf-field

Prototype PXF extensions for HAWQ
Apache License 2.0
36 stars 15 forks source link

Compiling PXF-JSON for PXF 2.3.2.0 and Hadoop 2.2.0-gphd-3.1.0.0 #19

Closed kdunn-pivotal closed 8 years ago

kdunn-pivotal commented 8 years ago

Trying to compile this for a customer using PXF 2.3.2.0 and PHD 2.1 (Hadoop 2.2.0-gphd-3.1.0.0) and getting an error about a missing class in pxf-service:

[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/kdunn/Desktop/pxfBuild/pxf-field/pxf-test/src/main/java/com/pivotal/pxf/PxfUnit.java:[36,31] cannot find symbol
  symbol:   class FragmentsResponse
  location: package com.pivotal.pxf.service
[ERROR] /Users/kdunn/Desktop/pxfBuild/pxf-field/pxf-test/src/main/java/com/pivotal/pxf/PxfUnit.java:[365,17] cannot find symbol
  symbol:   class FragmentsResponse
  location: class com.pivotal.pxf.PxfUnit
[ERROR] /Users/kdunn/Desktop/pxfBuild/pxf-field/pxf-test/src/main/java/com/pivotal/pxf/PxfUnit.java:[366,33] cannot find symbol
  symbol:   method formatResponse(java.util.List<com.pivotal.pxf.api.Fragment>,java.lang.String)
  location: class com.pivotal.pxf.service.FragmentsResponseFormatter

Any idea if/where the 2.3.2.0 code is available and can I simply add that class and compile?

tzolov commented 8 years ago

Hi @kdunn-pivotal, I've put together some release process and maven repository. Unfortunately for you the oldest release is for PHD3.0. From the commit history it seems that you can revert to commit: b30c0725fb8bfc29dc18d36873d57a610023749f to get to the right PXF version. Something like:

git reset --hard b30c0725fb8bfc29dc18d36873d57a610023749f

Please let me know if it worked for you?

kdunn-pivotal commented 8 years ago

Hey @tzolov :+1: Between reverting to the commit you mentioned and making some small changes the pom.xml I was able to compile and pass the tests. Since this is water under the bridge for most (old PHD versions), I'm putting the pom.xml in below as reference - PR doesn't make sense here.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.pivotal</groupId>
    <artifactId>json-pxf-ext</artifactId>
    <version>2.0.1.0-0</version>

    <properties>
        <hadoop-version>2.2.0-gphd-3.1.0.0</hadoop-version>
        <pxf-version>2.3.2.0</pxf-version>
        <pxf-test-version>2.0.1.0-1</pxf-test-version>
    </properties>

    <repositories>
        <repository>
            <id>org.springsource.maven.milestone</id>
            <name>SpringSource Maven Release Repository</name>
            <url>http://repo.springsource.org/libs-release</url>
        </repository>
    </repositories>
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.0</version>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
    <dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-auth</artifactId>
            <version>${hadoop-version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>${hadoop-version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-mapreduce-client-common</artifactId>
            <version>${hadoop-version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-mapreduce-client-core</artifactId>
            <version>${hadoop-version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.avro</groupId>
            <artifactId>avro</artifactId>
            <version>1.7.4</version>
        </dependency>
        <dependency>
            <groupId>commons-codec</groupId>
            <artifactId>commons-codec</artifactId>
            <version>1.4</version>
        </dependency>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>11.0.2</version>
        </dependency>
        <dependency>
            <groupId>com.pivotal</groupId>
            <artifactId>pxf-hdfs</artifactId>
            <version>${pxf-version}</version>
        </dependency>
        <dependency>
            <groupId>com.pivotal</groupId>
            <artifactId>pxf-test</artifactId>
            <version>${pxf-test-version}</version>
        </dependency>
        <dependency>
            <groupId>commons-cli</groupId>
            <artifactId>commons-cli</artifactId>
            <version>1.2</version>
        </dependency>
        <dependency>
            <groupId>commons-logging</groupId>
            <artifactId>commons-logging</artifactId>
            <version>1.1.3</version>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17</version>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
        </dependency>
        <dependency>
            <groupId>commons-configuration</groupId>
            <artifactId>commons-configuration</artifactId>
            <version>1.6</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-io</artifactId>
            <version>1.3.2</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.5</version>
        </dependency>
        <dependency>
            <groupId>org.codehaus.jackson</groupId>
            <artifactId>jackson-mapper-asl</artifactId>
            <version>1.8.8</version>
        </dependency>
        <dependency>
            <groupId>org.codehaus.jackson</groupId>
            <artifactId>jackson-core-asl</artifactId>
            <version>1.8.8</version>
        </dependency>
    </dependencies>
</project>
tzolov commented 8 years ago

Glad you've solved it @kdunn-pivotal and tanks for the pom!