Unable to build Mongodb-async driver with maven

[adnavare]

Hi,

I have maven version 3.6.0, MongoDB version 4.0.5. I am trying to build mongodb package with "mvn -pl com.yahoo.ycsb:mongodb-binding -am clean package" and I am seeing the error "Could not resolve dependencies for project com.yahoo.ycsb:mongodb-bindin g:jar:0.16.0-SNAPSHOT: Failed to collect dependencies at com.allanbank:mongodb-async-driver:jar:2.10.1: Failed to read artifact des criptor for com.allanbank:mongodb-async-driver:jar:2.10.1: Could not transfer artifact com.allanbank:mongodb-async-driver:pom:2.10. 1 from/to allanbank (http://www.allanbank.com/repo/): Connect to www.allanbank.com:80 [www.allanbank.com/206.210.70.161] failed: "

I am running on Clear Linux OS. Can someone please point me what might be going wrong. Thanks

[allanbank]

It is trying to pull the wrong version. The version should be 2.0.1 not 2.10.1.

Did you modify the mongodb.async.version version in the parent directory's pom.xml?

[adnavare]

Yes i did modify the pom.xml from the home directory of YCSB, because i tried earlier with 2.0.1 and it failed with the same error. So i read in some forums that 2.10.1 might work so i changed it. Even if i try synchronous driver i.e. mongodb it complains the same. Any idea what might be going wrong?

[allanbank]

Can you connect to this website from your browser?

http://www.allanbank.com/mongodb-async-driver/index.html

I just tried to do the build from my machine and it worked fine:

Downloading from allanbank: http://www.allanbank.com/repo/com/allanbank/mongodb-async-driver/2.0.1/mongodb-async-driver-2.0.1.pom Downloaded from allanbank: http://www.allanbank.com/repo/com/allanbank/mongodb-async-driver/2.0.1/mongodb-async-driver-2.0.1.pom (17 kB at 59 kB/s) Downloading from allanbank: http://www.allanbank.com/repo/com/allanbank/mongodb-async-driver/2.0.1/mongodb-async-driver-2.0.1.jar Downloaded from allanbank: http://www.allanbank.com/repo/com/allanbank/mongodb-async-driver/2.0.1/mongodb-async-driver-2.0.1.jar (552 kB at 1.9 MB/s)

Do those links work in your browser?

Rob.

[adnavare]

I am running the commands from terminal, and when I curl to http://www.allanbank.com/mongodb-async-driver/index.html I get 403 Forbidden error.

When I try to wget above links, I am getting 403 Forbidden as well. Am i missing some authentication file or something?

[allanbank]

The server is blocking the curl user agent to stop script kiddie bots and such.

If you use an agent other than "curl.*" the connection will work:

curl -A "let_me_in"  http://www.allanbank.com/mongodb-async-driver/index.html 

[adnavare]

But bin/ycsb isn't it automatically tries to get the artifact? If I am able to get http://www.allanbank.com/mongodb-async-driver/index.html with any other agent, how should I proceed? I think I have to point to the jar/pom file?

[adnavare]

What do you suggest to do right now?

[allanbank]

Can you run the following command from the mongodb subdirectory and then post the full output to pastebin or somewhere similar?

mvn  --debug compile 

[adnavare]

Sure. I ran the command from inside YCSB/mongodb. And now I went back to the mongodb-async version 2.0.1 (reverted pom.xml to the original). Attached is the output file. Please let me know if you are unable to open it or any other issue.

output_mvn.txt

This is a pastebin, https://pastebin.com/iA2Y5Rbm

Thank you for your inputs.

[adnavare]

@allanbank If you want, we can share screen, I am struggling with this issues for a while.

[risdenk]

@adnavare it is like you have issue connecting to pull down the mvn dependency. Do you have a proxy or something else to access the internet?

[risdenk]

I am unable to reproduce the issue you are having:

mvn --version
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T14:41:47-04:00)
Maven home: /usr/local/Cellar/maven/3.6.0/libexec
Java version: 1.8.0_192, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.14", arch: "x86_64", family: "mac"

mvn -pl mongodb -am compile

[INFO] Reactor Summary for YCSB Root 0.16.0-SNAPSHOT:
[INFO]
[INFO] YCSB Root .......................................... SUCCESS [  4.067 s]
[INFO] Core YCSB .......................................... SUCCESS [  4.148 s]
[INFO] Per Datastore Binding descriptor ................... SUCCESS [  0.163 s]
[INFO] YCSB Datastore Binding Parent ...................... SUCCESS [  0.140 s]
[INFO] MongoDB Binding .................................... SUCCESS [  0.396 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  9.421 s
[INFO] Finished at: 2019-01-17T13:21:28-05:00
[INFO] ------------------------------------------------------------------------

I even removed the dependency and checked that I could download it explicitly.

mvn dependency:get -DremoteRepositories='allan::default::http://www.allanbank.com/repo/' -Dartifact=com.allanbank:mongodb-async-driver:2.0.1

[adnavare]

@risdenk : Yes I am behind a proxy, but I have set the proxies in my environment, and also added inside maven/settings.xml. I know this might be some connectivity issue, but that's what I am trying to understand where else I may have to add the proxies.

My mv version mvn -version Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T11:41:47-07:00) Maven home: /usr/local/maven Java version: 1.8.0-internal, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-1.8.0-openjdk/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "4.19.13-680.native", arch: "amd64", family: "unix"

[adnavare]

The other option I am thinking is, if I download this http://www.allanbank.com/repo/com/allanbank/mongodb-async-driver/2.0.1/mongodb-async-driver-2.0.1.pom separately, do I need any other files that mvn will need? What I am thinking here is may be i will download the needed files somewhere local, and then inside pom.xml direct the local files, instead of the file available on the internet? Is it possible to direct pom.xml to chose local files

[adnavare]

@risdenk @allanbank : I have started the mongd server and is listening on 27017. Here is the output of it https://pastebin.com/2reqsK0t, just in case. I need to start the server if I am correct?

[adnavare]

OMG wow, it worked. I changed the settings.xml, to have an ID. I have two proxies set now, one has an id http_proxy and other has an id https_proxy. Earlier I had given in one of the proxy as https, thinking that tag means the name of the proxy. But the proxy I have even for https_proxy it use the protocol http. So I think the correct protocol was an issue, or it was getting confused with what proxy to use because there were no IDs

[adnavare]

I was able to build the source with mvn -pl com.yahoo.ycsb:mongodb-binding -am clean package Then ran, ./bin/ycsb load mongodb-async -s -P workloads/workloada and ran the test ./bin/ycsb run mongodb-async -s -P workloads/workloada > outputRun.txt

All worked, thank you so much for your time and inputs @allanbank @risdenk.

[simplynaveen20]

I encountered the similar issue when building mongo with error below Failed to read artifact descriptor for com.allanbank:mongodb-async-driver:jar:2.0.1: Could not transfer artifact com.allanbank:mongodb-async-driver:pom:2.0.1 from/to maven-default-http-blocker (http://0.0.0.0/):

Solution - I have maven 3.8.3 , in setting.xml under maven conf I commented out the below blocker

     <mirror>
      <id>maven-default-http-blocker</id>
      <mirrorOf>external:http:*</mirrorOf>
      <name>Pseudo repository to mirror external repositories initially using HTTP.</name>
      <url>http://0.0.0.0/</url>
      <blocked>true</blocked>
    </mirror>