search

ipfs-shipyard / java-ipfs-http-client

A Java implementation of the HTTP IPFS API
MIT License
538 stars 243 forks source link

Update to ipfs v0.5.0 #170

Closed ianopolous closed 4 years ago

ianopolous commented 4 years ago

This fixes all the breaking changes in ipfs v0.5.0.

As well as upgrading dependencies.

@lidel Does this qualify for the two bounties?

NadeeraM commented 4 years ago

@lidel I just pull this branch and compile it . mvn compile is passed. but mvn test not success full. I am getting following error. I am running tests against provided docker IPFS in this project.

`pub => sub took 2 pub => sub took 0 pub => sub took 1 child1: zb2rhXvZfBVfxehiE7VfgL5LbJpUXjkY64bms6hVqpE86zEff root1: zdpuAte25GgAQNVY9pgWHTj1a33wmWVFmyhLzELamTwAq7eBS java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168) at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140) at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:351) at java.base/sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:552) at java.base/sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:609) at java.base/sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:696) at java.base/java.io.FilterInputStream.read(FilterInputStream.java:133) at java.base/sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3505) at java.base/sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3498) at io.ipfs.api.IPFS.getObjectStream(IPFS.java:743) at io.ipfs.api.IPFS.retrieveAndParseStream(IPFS.java:682) at io.ipfs.api.IPFS.access$400(IPFS.java:15) at io.ipfs.api.IPFS$Pubsub.sub(IPFS.java:298) at io.ipfs.api.APITest.lambda$pubsubSynchronous$6(APITest.java:416) at java.base/java.lang.Thread.run(Thread.java:834)

Tests run: 39, Failures: 0, Errors: 3, Skipped: 2, Time elapsed: 216.74 sec <<< FAILURE! - in io.ipfs.api.APITest swarmTest(io.ipfs.api.APITest) Time elapsed: 0.052 sec <<< ERROR! java.lang.IllegalStateException: Unknown Multihash type: 0 at io.ipfs.api.APITest.swarmTest(APITest.java:663)

pinUpdate(io.ipfs.api.APITest) Time elapsed: 0.036 sec <<< ERROR! java.lang.IllegalStateException: Unknown Multibase type: / at io.ipfs.api.APITest.pinUpdate(APITest.java:253)

rawLeafNodePinUpdate(io.ipfs.api.APITest) Time elapsed: 0.011 sec <<< ERROR! java.lang.IllegalStateException: Unknown Multibase type: / at io.ipfs.api.APITest.rawLeafNodePinUpdate(APITest.java:287)

Results :

Tests in error: APITest.pinUpdate:253 » IllegalState Unknown Multibase type: / APITest.rawLeafNodePinUpdate:287 » IllegalState Unknown Multibase type: / APITest.swarmTest:663 » IllegalState Unknown Multihash type: 0 AddTest.add:37 IllegalState Error returned from IPFS: {"Name":"html/chap/ch01....

Tests run: 50, Failures: 0, Errors: 4, Skipped: 2

[INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:44 min [INFO] Finished at: 2020-07-30T10:02:47+12:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project java-ipfs-http-client: There are test failures. [ERROR] [ERROR] Please refer to /home/nadeera/Desktop/Work/ipfs/java-ipfs-http-client/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException `

Thanks.

ianopolous commented 4 years ago

That looks like maven is still using the old dependencies. I don't actually use maven myself, but I'll have a look.

ianopolous commented 4 years ago

@NadeeraM I've just fixed the maven file I think. Are you able to try again?

NadeeraM commented 4 years ago

@NadeeraM I've just fixed the maven file I think. Are you able to try again?

@ianopolous yes I will check

NadeeraM commented 4 years ago

@ianopolous now mvn build also failing. Problem is v1.4.0</version.multiaddr> is still not in mvn repo. Hope you have to add it manually.

[INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 0.228 s [INFO] Finished at: 2020-07-30T11:24:54+12:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project java-ipfs-http-client: Could not resolve dependencies for project com.github.ipfs:java-ipfs-http-client:jar:v1.3.0: Failure to find com.github.multiformats:java-multiaddr:jar:v1.4.0 in https://jitpack.io was cached in the local repository, resolution will not be reattempted until the update interval of jitpack.io has elapsed or updates are forced -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

NadeeraM commented 4 years ago

@ianopolous actually dependency is downloading from Jitpack. but getting following error when mvn compile.

ts/java-multiaddr/v1.4.0/java-multiaddr-v1.4.0.pom Downloading from central: https://repo.maven.apache.org/maven2/com/github/multiformats/java-multiaddr/v1.4.0/java-multiaddr-v1.4.0.pom [WARNING] The POM for com.github.multiformats:java-multiaddr:jar:v1.4.0 is missing, no dependency information available Downloading from jitpack.io: https://jitpack.io/com/github/multiformats/java-multiaddr/v1.4.0/java-multiaddr-v1.4.0.jar Downloading from central: https://repo.maven.apache.org/maven2/com/github/multiformats/java-multiaddr/v1.4.0/java-multiaddr-

v1.4.0.jar [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 5.693 s [INFO] Finished at: 2020-07-30T12:15:05+12:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project java-ipfs-http-client: Could not resolve dependencies for project com.github.ipfs:java-ipfs-http-client:jar:v1.3.0: Could not find artifact com.github.multiformats:java-multiaddr:jar:v1.4.0 in jitpack.io (https://jitpack.io) -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

ianopolous commented 4 years ago

@NadeeraM jitpack.io had got itself into a strange state, but I've bumped a few versions in the dependencies for maven and it is working now.

NadeeraM commented 4 years ago

@ianopolous yes can compile the code successfully. But Still tests are failing. This is the tests that fail

    public void pubsubSynchronous() throws Exception {
        String topic = "topic" + System.nanoTime();
        List<Map<String, Object>> res = Collections.synchronizedList(new ArrayList<>());
        new Thread(() -> {
            try {
                ipfs.pubsub.sub(topic, res::add, t -> t.printStackTrace());
            } catch (IOException e) {
                throw new RuntimeException(e);}
        }).start();```

Stack trace Error:

```pub => sub took 1
child1: zb2rhXvZfBVfxehiE7VfgL5LbJpUXjkY64bms6hVqpE86zEff
root1: zdpuAte25GgAQNVY9pgWHTj1a33wmWVFmyhLzELamTwAq7eBS
java.net.SocketTimeoutException: Read timed out
        at java.base/java.net.SocketInputStream.socketRead0(Native Method)
        at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
        at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
        at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:292)
        at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:351)
        at java.base/sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:552)
        at java.base/sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:609)
        at java.base/sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:696)
        at java.base/java.io.FilterInputStream.read(FilterInputStream.java:133)
        at java.base/sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3505)
        at java.base/sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3498)
        at io.ipfs.api.IPFS.getObjectStream(IPFS.java:743)
        at io.ipfs.api.IPFS.retrieveAndParseStream(IPFS.java:682)
        at io.ipfs.api.IPFS.access$400(IPFS.java:15)
        at io.ipfs.api.IPFS$Pubsub.sub(IPFS.java:298)
        at io.ipfs.api.APITest.lambda$pubsubSynchronous$6(APITest.java:416)
        at java.base/java.lang.Thread.run(Thread.java:834)

Tests run: 39, Failures: 0, Errors: 2, Skipped: 2, Time elapsed: 243.865 sec <<< FAILURE! - in io.ipfs.api.APITest
pinUpdate(io.ipfs.api.APITest)  Time elapsed: 0 sec  <<< ERROR!
java.lang.IllegalStateException: Unknown Multibase type: /
        at io.ipfs.api.APITest.pinUpdate(APITest.java:253)

rawLeafNodePinUpdate(io.ipfs.api.APITest)  Time elapsed: 0.021 sec  <<< ERROR!
java.lang.IllegalStateException: Unknown Multibase type: /
        at io.ipfs.api.APITest.rawLeafNodePinUpdate(APITest.java:287)

Results :

Tests in error: 
  APITest.pinUpdate:253 » IllegalState Unknown Multibase type: /
  APITest.rawLeafNodePinUpdate:287 » IllegalState Unknown Multibase type: /
  AddTest.add:37 IllegalState Error returned from IPFS: {"Name":"html/chap/ch01....

Tests run: 50, Failures: 0, Errors: 3, Skipped: 2

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  04:09 min
[INFO] Finished at: 2020-07-31T09:48:55+12:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project java-ipfs-http-client: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/nadeera/Desktop/Work/ipfs/java-ipfs-http-client/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
ianopolous commented 4 years ago

The tests all run fine in CI, and if I run them locally using maven against ipfs v0.5.0 then they all pass:

Tests run: 50, Failures: 0, Errors: 0, Skipped: 2

The errors you're getting, @NadeeraM look like your running against an older version of ipfs, 0.4.23 or earlier (when pin update returned multiaddr rather than multihashes). Could you confirm the ipfs version you are testing against?

NadeeraM commented 4 years ago

The tests all run fine in CI, and if I run them locally using maven against ipfs v0.5.0 then they all pass:

Tests run: 50, Failures: 0, Errors: 0, Skipped: 2

The errors you're getting, @NadeeraM look like your running against an older version of ipfs, 0.4.23 or earlier (when pin update returned multiaddr rather than multihashes). Could you confirm the ipfs version you are testing against?

@ianopolous Yes i am running local ipfs using docker-compose file in this project. Version is "Version": "0.4.16". Hope better to update that file version as well. :)

https://github.com/ipfs-shipyard/java-ipfs-http-client/blob/fc82516d715a64b42df0532166d0408412b3116b/docker-compose.yml#L4

Thank you

ianopolous commented 4 years ago

@lidel Any other comments on this? I think it's good to go now.

ianopolous commented 4 years ago

@ianopolous I noticed tests did not run against bundled junit and cid.jar (?) and after adding both in f092411 CI no longer passed (example).

It includes the cid jar via the ipfs.jar manifest.

I've re-run the job and it is now green – any idea why previous run was flaky?

Some of the tests are not strict unit tests because they impose a time limit and are thus more of a performance test of ipfs, like the pubsubSynchronous test. I've rewritten that to ignore the time now and instead require 95% delivery of messages (ipfs pubsub isn't reliable even on the same instance it seems).

lidel commented 4 years ago

Thank you @ianopolous, expecting 95% delivery of messages should work better than time-based one :+1:

:point_right: I've confirmed this PR works for me locally and satisfies requirements of both bounties. I believe this closes the double bounty listed in #157 – please merge this PR and make a new release of this library :pray:

We will finalize bounty details in #157