search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.07k stars 5.11k forks source link

Fail to pull in MediaPipe's external dependencies due to unstable internet connection #3

Closed chenloveheimei closed 5 years ago

chenloveheimei commented 5 years ago

Hello,My name is Long. Description: version:newest(2aaf469) bazel:0.27.0 ubuntu:16.04 ndk:r17c opencv:3.4.1(from source code include origin and contrib)

this is my Error: long@long:/media/long/data/android/mediapipe$ bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu ERROR: /media/long/data/android/mediapipe/third_party/BUILD:24:1: no such package '@android_opencv//': java.io.IOException: Error downloading [https://sourceforge.net/projects/opencvlibrary/files/4.0.1/opencv-4.0.1-android-sdk.zip/downl oad] to /home/long/.cache/bazel/_bazel_long/2855d1951dc2684b853376a37e34b96b/external/android_opencv/download.zip: connect timed out and referenced by '//third_party:opencv' ERROR: Analysis of target '//mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu:facedetectiongpu' failed; build aborted: Analysis failed INFO: Elapsed time: 16.197s INFO: 0 processes. FAILED: Build did NOT complete successfully (156 packages loaded, 6354 targets configured) currently loading: @org_tensorflow//tensorflow/lite/experimental/ruy Fetching @com_google_common_flogger; fetching 7s Fetching @com_google_common_flogger_system_backend; fetching 7s

Can I use version 3.4.1 of opencv? Is it possible not to upgrade? thx.

jiuqiant commented 5 years ago

Hi Long, the error you pasted above is more like a network issue. More specifically, your machine is not able to download the pre-compiled OpenCV android SDK from https://sourceforge.net/projects/opencvlibrary/files/4.0.1/opencv-4.0.1-android-sdk.zip/download. Unfortunately, we use the pre-compiled OpenCV Android SDK directly, rather than building the libraries from source.

Could you verify that you can access https://sourceforge.net/projects/opencvlibrary/files/4.0.1/opencv-4.0.1-android-sdk.zip/download? If so, can you retry the build command?

chenloveheimei commented 5 years ago

Hi Long, the error you pasted above is more like a network issue. More specifically, your machine is not able to download the pre-compiled OpenCV android SDK from https://sourceforge.net/projects/opencvlibrary/files/4.0.1/opencv-4.0.1-android-sdk.zip/download. Unfortunately, we use the pre-compiled OpenCV Android SDK directly, rather than building the libraries from source.

Could you verify that you can access https://sourceforge.net/projects/opencvlibrary/files/4.0.1/opencv-4.0.1-android-sdk.zip/download? If so, can you retry the build command?

i can acces your website. now i try it again. and generate a new Error: INFO: Call stack for the definition of repository 'FP16' which is a third_party_http_archive (rule definition at /home/long/.cache/bazel/_bazel_long/2855d1951dc2684b853376a37e34b96b/external/org_tensorflow/third_party/repo.bzl:206:28):

I am in China. I can't open the VPN recent.

jiuqiant commented 5 years ago

Hi Long, sorry about the inconvenience. https://mirror.bazel.build/ is a google owned website, and it is likely to be blocked by the Great Firewall. But, https://github.com/ should not be blacklisted. If you don't have VPN, the best solution might be retry the Bazel build command later. Bazel caches the downloaded libraries, so you don't need to worry about losing them by retrying the same build command. Hope this helps.

chenloveheimei commented 5 years ago

Hi Long, sorry about the inconvenience. https://mirror.bazel.build/ is a google owned website, and it is likely to be blocked by the Great Firewall. But, https://github.com/ should not be blacklisted. If you don't have VPN, the best solution might be retry the Bazel build command later. Bazel caches the downloaded libraries, so you don't need to worry about losing them by retrying the same build command. Hope this helps.

My windows PC can opens VPN, but the Ubuntu can't.so I think I'll download the libraries which project needs from Windows and put them into the Ubuntu to setup?

jiuqiant commented 5 years ago

Yes, it's possible to cache the external dependencies and use the cached version later. We are testing the following workflow:

# On the workstation that has the full access to the Internet, fetch all the dependencies to a cache directory by doing:
$ git clone https://github.com/google/mediapipe.git
$ cd mediapipe
$ bazel fetch //...  --keep_going --repository_cache=<your cache directory> 

# Then, copy the cache directory to the machine that has restricted Internet access and do:
$  bazel build -c opt --config=android_arm64  mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu --distdir=<your cache directory>

If the workflow works, we may host an archive that contains all the MediaPipe external dependencies for the users in China.

Stay tuned.

chenloveheimei commented 5 years ago

Yes, it's possible to cache the external dependencies and use the cached version later. We are testing the following workflow:

# On the workstation that has the full access to the Internet, fetch all the dependencies to a cache directory by doing:
$ git clone https://github.com/google/mediapipe.git
$ cd mediapipe
$ bazel fetch //...  --keep_going --repository_cache=<your cache directory> 

# Then, copy the cache directory to the machine that has restricted Internet access and do:
$  bazel build -c opt --config=android_arm64  mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu --distdir=<your cache directory>

If the workflow works, we may host an archive that contains all the MediaPipe external dependencies for the users in China.

Stay tuned.

thanks.I hope have a good news and can you @me when you update the new version?

camillol commented 5 years ago

@jiuqiant it seems like this would be a common issue for projects using Bazel. There are similar issues on the TensorFlow and Bazel trackers; we should check in with them and see if there's an existing workaround or plan.

jiuqiant commented 5 years ago

@chenloveheimei and I tested the current version of MediaPipe. We verified that MediaPipe demo apps can be successfully built in China now. However, due to the unstable/slow internet connection, we prefer to ask Bazel to cache the external repositories locally and search for the local archives before accessing the network.

For the users with unstable/slow internet connection, you may need to try the following build command several times to download all the needed external repositories:

$ bazel build -c opt --config=android_arm64 \ 
  --distdir=<your cache directory> \
  --repository_cache=<your cache directory> \
  --keep_going \
  mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu
chenloveheimei commented 5 years ago

@chenloveheimei and I tested the current version of MediaPipe. We verified that MediaPipe demo apps can be successfully built in China now. However, due to the unstable/slow internet connection, we prefer to ask Bazel to cache the external repositories locally and search for the local archives before accessing the network.

For the users with unstable/slow internet connection, you may need to try the following build command several times to download all the needed external repositories:

$ bazel build -c opt --config=android_arm64 \ 
  --distdir=<your cache directory> \
  --repository_cache=<your cache directory> \
  --keep_going \
  mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu

I am now able to build object and face detection demos from Shenzhen, China. The problem is solved. Thank you , you are good guy.

hippyk commented 4 years ago

Yes, it's possible to cache the external dependencies and use the cached version later. We are testing the following workflow:

# On the workstation that has the full access to the Internet, fetch all the dependencies to a cache directory by doing:
$ git clone https://github.com/google/mediapipe.git
$ cd mediapipe
$ bazel fetch //...  --keep_going --repository_cache=<your cache directory> 

# Then, copy the cache directory to the machine that has restricted Internet access and do:
$  bazel build -c opt --config=android_arm64  mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu --distdir=<your cache directory>

If the workflow works, we may host an archive that contains all the MediaPipe external dependencies for the users in China.

Stay tuned.

Any others work but the 'opencv-3.4.3-android-sdk.zip'.

Papageno2 commented 4 years ago

Yes, it's possible to cache the external dependencies and use the cached version later. We are testing the following workflow:

# On the workstation that has the full access to the Internet, fetch all the dependencies to a cache directory by doing:
$ git clone https://github.com/google/mediapipe.git
$ cd mediapipe
$ bazel fetch //...  --keep_going --repository_cache=<your cache directory> 

# Then, copy the cache directory to the machine that has restricted Internet access and do:
$  bazel build -c opt --config=android_arm64  mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu --distdir=<your cache directory>

If the workflow works, we may host an archive that contains all the MediaPipe external dependencies for the users in China.

Stay tuned.

hi, thanks for the above hints.

I want to install MediaPipe on CentOS7, but the internet connection error occurs (Beijing now), so I try to fetch the cache dependencies on my PC(macOS), but the above workflow didn't work on my PC, anywhere I can download all the MediaPipe externel dependencies directly ?

the following is part of the warnings and errors on PC fetching(*** is my username):

WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/unicode-org/icu/archive/release-64-2.zip failed: class java.io.IOException connect timed out
WARNING: Download from https://github.com/unicode-org/icu/archive/release-64-2.zip failed: class javax.net.ssl.SSLProtocolException Read timed out
ERROR: An error occurred during the fetch of repository 'icu':
   java.io.IOException: Error downloading [https://storage.googleapis.com/mirror.tensorflow.org/github.com/unicode-org/icu/archive/release-64-2.zip, https://github.com/unicode-org/icu/archive/release-64-2.zip] to /private/var/tmp/_bazel_***/392f0b43f8832c7859e2f502bc85241d/external/icu/release-64-2.zip: Read timed out
ERROR: /private/var/tmp/_bazel_***/392f0b43f8832c7859e2f502bc85241d/external/org_tensorflow/third_party/icu/data/BUILD.bazel:39:1: no such package '@icu//': java.io.IOException: Error downloading [https://storage.googleapis.com/mirror.tensorflow.org/github.com/unicode-org/icu/archive/release-64-2.zip, https://github.com/unicode-org/icu/archive/release-64-2.zip] to /private/var/tmp/_bazel_***/392f0b43f8832c7859e2f502bc85241d/external/icu/release-64-2.zip: Read timed out and referenced by '@org_tensorflow//third_party/icu/data:conversion_data'
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/open-source-parsers/jsoncpp/archive/1.8.4.tar.gz failed: class java.io.IOException connect timed out
WARNING: Download from https://github.com/open-source-parsers/jsoncpp/archive/1.8.4.tar.gz failed: class java.io.IOException connect timed out
ERROR: An error occurred during the fetch of repository 'jsoncpp_git':
   java.io.IOException: Error downloading [https://storage.googleapis.com/mirror.tensorflow.org/github.com/open-source-parsers/jsoncpp/archive/1.8.4
Samuel-wei commented 3 years ago

@chenloveheimei and I tested the current version of MediaPipe. We verified that MediaPipe demo apps can be successfully built in China now. However, due to the unstable/slow internet connection, we prefer to ask Bazel to cache the external repositories locally and search for the local archives before accessing the network. For the users with unstable/slow internet connection, you may need to try the following build command several times to download all the needed external repositories:

$ bazel build -c opt --config=android_arm64 \ 
  --distdir=<your cache directory> \
  --repository_cache=<your cache directory> \
  --keep_going \
  mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectiongpu

I am now able to build object and face detection demos from Shenzhen, China. The problem is solved. Thank you , you are good guy.

I had a similar issues like as you encounter issues what about above. I had workflow above, but I am unsuccessful. Would you give me some advises? thank you

Lr-2002 commented 3 years ago

HI,could i download the dependencies via firefox and the copy it into the fire storage ? things doesn't work well,It seems bazel making different dictionary every time?

11676670 commented 3 years ago

if you network blocked by the Great Firewall. you can change your repositories in WORKSPACE file like this. repositories = [ "https://maven.aliyun.com/repository/google", "https://maven.aliyun.com/repository/central", "https://maven.aliyun.com/repository/public", ], Good luck.

DLGL99GL commented 3 years ago

如果你的网络被防火长城。 您可以更改您的存储库工作区中的文件。 存储库= ( “https://maven.aliyun.com/repository/google”, “https://maven.aliyun.com/repository/central”, “https://maven.aliyun.com/repository/public”, ), 祝你好运。

May I ask which file path is modified specifically？

11676670 commented 3 years ago

git clone https://github.com/google/mediapipe.git

cd mediapipe

vi  WORKSPACE       //this WORKSPACE  file

------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2021年9月16日(星期四) 下午4:29 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [google/mediapipe] Fail to pull in MediaPipe's external dependencies due to unstable internet connection (#3)

如果你的网络被防火长城。 您可以更改您的存储库工作区中的文件。 存储库= ( “https://maven.aliyun.com/repository/google”, “https://maven.aliyun.com/repository/central”, “https://maven.aliyun.com/repository/public”, ), 祝你好运。

May I ask which file path is modified specifically？

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.