search

claffin / cloudproxy

Hide your scrapers IP behind the cloud. Provision proxy servers across different cloud providers to improve your scraping success.
https://cloudproxy.io/
MIT License
1.4k stars 80 forks source link

Digitalocean Droplet created but not showing in the API #62

Closed mrahmadt closed 2 years ago

mrahmadt commented 2 years ago

Hello

I launched the container as per the documentation, but I'm still not seeing anything in the UI nor the API, I can see it created the droplets and keep removing/adding new droplets every few minutes

What I'm missing?

export DIGITALOCEAN_ENABLED=True
export DIGITALOCEAN_ACCESS_TOKEN="XXXXX"
export DIGITALOCEAN_MIN_SCALING=2
export DIGITALOCEAN_MAX_SCALING=2
export DIGITALOCEAN_SIZE="s-1vcpu-512mb-10gb"
export DIGITALOCEAN_REGION="fra1"
export AGE_LIMIT="1200"
export USERNAME="XXX"
export PASSWORD='XXXX'

docker run -e USERNAME=$USERNAME \
    -e PASSWORD=$PASSWORD \
    -e DIGITALOCEAN_ENABLED=$DIGITALOCEAN_ENABLED \
    -e DIGITALOCEAN_ACCESS_TOKEN=$DIGITALOCEAN_ACCESS_TOKEN \
    -e DIGITALOCEAN_MIN_SCALING=$DIGITALOCEAN_MIN_SCALING \
    -e DIGITALOCEAN_MAX_SCALING=$DIGITALOCEAN_MAX_SCALING \
    -e DIGITALOCEAN_SIZE=$DIGITALOCEAN_SIZE \
    -e DIGITALOCEAN_REGION=$DIGITALOCEAN_REGION \
    -e AGE_LIMIT=$AGE_LIMIT \
    -it -p 8000:8000 laffin/cloudproxy:latest
mrahmadt commented 2 years ago

BTW, no error reported in the console

2022-07-14 22:07:54.967 | INFO     | apscheduler.executors.base:run_job:144 - Job "do_manager (trigger: interval[0:00:20], next run at: 2022-07-14 22:08:05 UTC)" executed successfully
2022-07-14 22:07:56.059 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:07:56.064 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:07:58.703 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:07:58.716 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:01.701 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:01.705 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:04.715 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:04.727 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:05.395 | INFO     | apscheduler.executors.base:run_job:123 - Running job "do_manager (trigger: interval[0:00:20], next run at: 2022-07-14 22:08:25 UTC)" (scheduled at 2022-07-14 22:08:05.383825+00:00)
2022-07-14 22:08:07.731 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:07.737 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:08.697 | INFO     | cloudproxy.providers.digitalocean.main:do_deployment:29 - Minimum DO Droplets met
2022-07-14 22:08:10.844 | INFO     | cloudproxy.providers.digitalocean.main:do_check_alive:61 - Waiting: DO -> 178.62.53.129
2022-07-14 22:08:10.878 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:10.882 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:12.942 | INFO     | cloudproxy.providers.digitalocean.main:do_check_alive:61 - Waiting: DO -> 178.62.44.62
2022-07-14 22:08:12.951 | INFO     | apscheduler.executors.base:run_job:144 - Job "do_manager (trigger: interval[0:00:20], next run at: 2022-07-14 22:08:25 UTC)" executed successfully
2022-07-14 22:08:15.278 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:15.285 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:18.286 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:18.295 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:21.294 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:21.302 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:24.481 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET / HTTP/1.1" 200
2022-07-14 22:08:24.689 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:24.698 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:25.388 | INFO     | apscheduler.executors.base:run_job:123 - Running job "do_manager (trigger: interval[0:00:20], next run at: 2022-07-14 22:08:45 UTC)" (scheduled at 2022-07-14 22:08:25.383825+00:00)
2022-07-14 22:08:25.625 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /ui/ HTTP/1.1" 304
2022-07-14 22:08:25.727 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:25.731 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:26.182 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /ui/ HTTP/1.1" 304
2022-07-14 22:08:26.278 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:26.284 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:26.703 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /ui/ HTTP/1.1" 304
2022-07-14 22:08:26.812 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:26.819 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:28.198 | INFO     | cloudproxy.providers.digitalocean.main:do_deployment:29 - Minimum DO Droplets met
2022-07-14 22:08:29.820 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57148 - "GET /destroy HTTP/1.1" 200
2022-07-14 22:08:29.824 | INFO     | uvicorn.protocols.http.h11_impl:send:461 - 172.17.0.1:57150 - "GET /providers HTTP/1.1" 200
2022-07-14 22:08:31.265 | INFO     | cloudproxy.providers.digitalocean.main:do_check_alive:61 - Waiting: DO -> 178.62.53.129
claffin commented 2 years ago

Are the username and password being used alphanumeric? Using characters like $;'#/., etc can cause issues.

Bayoslav commented 2 years ago

I can confirm this is still active and I'm using a letters-only password

claffin commented 2 years ago

Sorry, is this still an issue?

Bayoslav commented 2 years ago

Yes, sort of, I've been using version 0.63 of cloudproxy and that one works well. Think it has to do with firewall settings which might not be related to cloudproxy itself