search

online-bridge-hackathon / DDS

An api that returns DDS results for a given deal or partial deal
https://bit.ly/bridge-hackathon
Apache License 2.0
4 stars 7 forks source link

Added scaffolding to access DDS via a Google Cloud Function. #84

Closed tameware closed 4 years ago

tameware commented 4 years ago

Also added a new sample deal.

I had some trouble with my first push, and rebase -i for once did not help me, so this one is not as clean as I'd like. There's not a lot here, though. Let me know if you'd like me to clean it up - I can just copy everything to a new client.

I need additional permissions to test the targets that deploy and curl against test and prod. Once those are working I'll remove the "aw" targets.

tameware commented 4 years ago

I pushed some changes. You can now set up and connect to your own test deployment, e.g.:

make -f Makefile_gcf deploy_dev ENDPOINT=dds_hk
make -f Makefile_gcf curl_dev ENDPOINT=dds_hk
tameware commented 4 years ago

I'll update our docs for this if we decide to switch. You need to execute this before starting a local server:

pip install functions-framework

tameware commented 4 years ago

Was your testing successful, @kiat-ebed ?

kiat-ebed commented 4 years ago

Hey Adam, just wrapping up a project tonight that I have been heavily involved in for the past few months. I'll get back to being more responsive asap...

On Fri, 4 Sep 2020 at 21:03, Adam Wildavsky notifications@github.com wrote:

Was your testing successful, @kiat-ebed https://github.com/kiat-ebed ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/online-bridge-hackathon/DDS/pull/84#issuecomment-687354828, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQPWWYU5ADW34IZUMCJGWQLSEFB2HANCNFSM4QQCGMZA .

-- Kind regards, Kiat

Kiat Huang 07526-810553 ebu.co.uk/biographies/kiat-huang www.linkedin.com/in/kiathuang

Come teach or learn bridge at the..

kiat-ebed commented 4 years ago

Works well. With the new code I could deploy from my Linux machine and get results as expected in around 2 seconds.

{kiat@newton:~/dev/hack/DDS ?} cat /etc/lsb-release && uname -a DISTRIB_ID=Ubuntu DISTRIB_RELEASE=20.04 DISTRIB_CODENAME=focal DISTRIB_DESCRIPTION="Ubuntu 20.04.1 LTS" Linux newton 5.4.0-45-generic #49-Ubuntu SMP Wed Aug 26 13:38:52 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

{kiat@newton:~/dev/hack/DDS ?} time make -f Makefile_gcf curl_prod curl \ --header "Content-Type: application/json" \ --data "@./data/sample_deal.json" \ https://us-central1-globalbridge-app.cloudfunctions.net/dds {"C":{"E":4,"N":9,"S":8,"W":4},"D":{"E":6,"N":7,"S":7,"W":6},"H":{"E":6,"N":6,"S":6,"W":7},"N":{"E":6,"N":7,"S":6,"W":6},"S":{"E":5,"N":8,"S":7,"W":5}}

real 0m2.006s user 0m0.027s sys 0m0.010s {kiat@newton:~/dev/hack/DDS ?} time make -f Makefile_gcf curl_dev curl \ --header "Content-Type: application/json" \ --data "@./data/sample_deal.json" \ https://us-central1-online-bridge-hackathon-2020.cloudfunctions.net/dds {"C":{"E":4,"N":9,"S":8,"W":4},"D":{"E":6,"N":7,"S":7,"W":6},"H":{"E":6,"N":6,"S":6,"W":7},"N":{"E":6,"N":7,"S":6,"W":6},"S":{"E":5,"N":8,"S":7,"W":5}}

real 0m2.064s user 0m0.018s sys 0m0.019s

kiat-ebed commented 4 years ago

LGTM

kiat-ebed commented 4 years ago

Future responses from my usual kiat.huang@gmail.com email address.

tameware commented 4 years ago

Grand!

Did you also try make -f Makefile_gcf deploy_dev? I'm curious whether that will produce a working version when run from a Linux box.

kiat-ebed commented 4 years ago

It did not work unfortunately, details below.

make -f Makefile_gcf deploy_dev

git submodule update --init --recursive
make -C libdds/.build
make[1]: Entering directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
make[2]: Entering directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
make[3]: Entering directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
make[3]: Leaving directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
[100%] Built target dds
make[2]: Leaving directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
make[1]: Leaving directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
gcloud functions delete dds --project=online-bridge-hackathon-2020 --quiet || true
Waiting for operation to finish...done.                                        
Deleted [projects/online-bridge-hackathon-2020/locations/us-central1/functions/dds].
gcloud functions deploy dds --entry-point=dds --memory=2048MB --runtime python37 --trigger-http --allow-unauthenticated --project=online-bridge-hackathon-2020
Deploying function (may take a while - up to 2 minutes)...⠹                    
For Cloud Build Stackdriver Logs, visit: https://console.cloud.google.com/logs/viewer?project=online-bridge-hackathon-2020&advancedFilter=resource.type%3Dbuild%0Aresource.labels.build_id%3D48154790-3b56-4060-b385-91800f395c6b%0AlogName%3Dprojects%2Fonline-bridge-hackathon-2020%2Flogs%2Fcloudbuild
Deploying function (may take a while - up to 2 minutes)...done.                
availableMemoryMb: 2048
buildId: 48154790-3b56-4060-b385-91800f395c6b
entryPoint: dds
httpsTrigger:
  url: https://us-central1-online-bridge-hackathon-2020.cloudfunctions.net/dds
ingressSettings: ALLOW_ALL
labels:
  deployment-tool: cli-gcloud
name: projects/online-bridge-hackathon-2020/locations/us-central1/functions/dds
runtime: python37
serviceAccountEmail: online-bridge-hackathon-2020@appspot.gserviceaccount.com
sourceUploadUrl: https://storage.googleapis.com/gcf-upload-us-central1-de76b72a-95a5-4cf7-b200-6ec269510e99/006f9fe5-3e39-42a9-9208-fa30a9f5cf55.zip?GoogleAccessId=service-928665799533@gcf-admin-robot.iam.gserviceaccount.com&Expires=1599901125&Signature=uuQIDDaWaLSvhve8Ma455sGv4bZgtYF%2Fbv9hviMDojaH0ohFY9YbpR%2B6Lx%2F6MxBMPYPmL%2FFN7TUhPS2XMcEHnW%2FiMSkBtX8mpt7hrsr6HiPnXuL1qVu4x4BnaT%2Frfy1ONB8aoHzdt5YrbooJveTKQ7fseDxTuua%2FNrEIRMDxMiaCBZQRm7eUTdMRqd7%2BEY96Q2kJhqebCx1U%2BQkx4tggzzvicAVVqFy4eGEn6WZYDd8Zw2%2FP0CkodTXTJcHXQn%2BrRcgPi3J7vOkUfUFpHdTGW1OLqZqTMg4o1kqSzZ7KGnYOIyxXZV81IHG8%2BHxHl6yErElEbDTB%2F%2FA8JKvmec%2Fyiw%3D%3D
status: ACTIVE
timeout: 60s
updateTime: '2020-09-12T08:30:26.021Z'
versionId: '1'
{kiat@newton:~/hack/DDS ?}

make -f Makefile_gcf curl_dev

{kiat@newton:~/hack/DDS ?} make -f Makefile_gcf curl_dev 
curl \
--header "Content-Type: application/json" \
--data "@./data/sample_deal.json" \
https://us-central1-online-bridge-hackathon-2020.cloudfunctions.net/dds
Error: could not handle the request
{kiat@newton:~/hack/DDS ?} 
{kiat@newton:~/hack/DDS ?} make -f Makefile_gcf curl_dev 
curl \
--header "Content-Type: application/json" \
--data "@./data/sample_deal.json" \
https://us-central1-online-bridge-hackathon-2020.cloudfunctions.net/dds
Error: could not handle the request
{kiat@newton:~/hack/DDS ?}
kiat-ebed commented 4 years ago

Now curl_prod also does not work, though deploy_prod is successful

make -f Makefile_gcf deploy_prod

{kiat@newton:~/hack/DDS ?} make -f Makefile_gcf deploy_prod 
git submodule update --init --recursive
make -C libdds/.build
make[1]: Entering directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
make[2]: Entering directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
make[3]: Entering directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
make[3]: Leaving directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
[100%] Built target dds
make[2]: Leaving directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
make[1]: Leaving directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'
gcloud functions delete dds --project=globalbridge-app --quiet || true
ERROR: (gcloud.functions.delete) ResponseError: status=[404], code=[Not Found], message=[Function dds in region us-central1 in project globalbridge-app does not exist]
gcloud functions deploy dds --entry-point=dds --memory=2048MB --runtime python37 --trigger-http --allow-unauthenticated --project=globalbridge-app
Deploying function (may take a while - up to 2 minutes)...⠛                    
For Cloud Build Stackdriver Logs, visit: https://console.cloud.google.com/logs/viewer?project=globalbridge-app&advancedFilter=resource.type%3Dbuild%0Aresource.labels.build_id%3D290f5c2f-4420-4466-8b67-16f87b4ac13c%0AlogName%3Dprojects%2Fglobalbridge-app%2Flogs%2Fcloudbuild
Deploying function (may take a while - up to 2 minutes)...done.                
availableMemoryMb: 2048
buildId: 290f5c2f-4420-4466-8b67-16f87b4ac13c
entryPoint: dds
httpsTrigger:
  url: https://us-central1-globalbridge-app.cloudfunctions.net/dds
ingressSettings: ALLOW_ALL
labels:
  deployment-tool: cli-gcloud
name: projects/globalbridge-app/locations/us-central1/functions/dds
runtime: python37
serviceAccountEmail: globalbridge-app@appspot.gserviceaccount.com
sourceUploadUrl: https://storage.googleapis.com/gcf-upload-us-central1-120d6d56-5ac1-4f33-8c5e-d92ef0796c7c/a6e93c9e-ae2b-4a0e-8e74-198c072e7a83.zip?GoogleAccessId=service-34968331715@gcf-admin-robot.iam.gserviceaccount.com&Expires=1599902920&Signature=IAyHDRRIlZVxiYfkl89iJlaQdFPgej9O0PpZ%2Fyf39nORCkHRzc28UamqG0IvbZjxPZnujAu9j2KXGymSmGmhcHZcnnPB6BlLHQn%2BqkgJscR%2Fn9ubxknvdaUCpGPUfLvqfWepTHtdL9zqWMv45gQWXXZhIQ1%2BB8dR108vCFDJHSEVx7fgOKmrdUefZ2bDz0F95rXO1Wg86%2FohzU1Odh0FngycKn%2BgMHWkl6cdygsM4j%2BEdtGZMdxwT%2BI5%2B%2FQNXMbmcCOMgeTLDLqV6CwuPUz8LXcgS2taT9wpiVQUEQIrvtHpf1%2Bx8o0WgZ1f3tzXN74Ats1xWGyNgRm4UnCkWLaQEQ%3D%3D
status: ACTIVE
timeout: 60s
updateTime: '2020-09-12T09:00:21.105Z'
versionId: '1'

make -f Makefile_gcf curl_prod 


curl \
--header "Content-Type: application/json" \
--data "@./data/sample_deal.json" \
https://us-central1-globalbridge-app.cloudfunctions.net/dds
Error: could not handle the request
{kiat@newton:~/hack/DDS ?}```
tameware commented 4 years ago

My theory is that the binary you build is not compatible with the GCP execution environment. The same happens in my Mac. Please try deploying from the Cloud Shell and let me know what happens.

Also, you should be able to start a local server and use the curl_local target.

On Sat, Sep 12, 2020 at 5:03 AM Kiat Huang notifications@github.com wrote:

Now curl_prod also does not work, though deploy_prod is successful

make -f Makefile_gcf deploy_prod

{kiat@newton:~/hack/DDS ?} make -f Makefile_gcf deploy_prod

git submodule update --init --recursive

make -C libdds/.build

make[1]: Entering directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'

make[2]: Entering directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'

make[3]: Entering directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'

make[3]: Leaving directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'

[100%] Built target dds

make[2]: Leaving directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'

make[1]: Leaving directory '/home/kiat/Bridge/Hackathon/code/DDS/libdds/.build'

gcloud functions delete dds --project=globalbridge-app --quiet || true

ERROR: (gcloud.functions.delete) ResponseError: status=[404], code=[Not Found], message=[Function dds in region us-central1 in project globalbridge-app does not exist]

gcloud functions deploy dds --entry-point=dds --memory=2048MB --runtime python37 --trigger-http --allow-unauthenticated --project=globalbridge-app

Deploying function (may take a while - up to 2 minutes)...⠛

For Cloud Build Stackdriver Logs, visit: https://console.cloud.google.com/logs/viewer?project=globalbridge-app&advancedFilter=resource.type%3Dbuild%0Aresource.labels.build_id%3D290f5c2f-4420-4466-8b67-16f87b4ac13c%0AlogName%3Dprojects%2Fglobalbridge-app%2Flogs%2Fcloudbuild

Deploying function (may take a while - up to 2 minutes)...done.

availableMemoryMb: 2048

buildId: 290f5c2f-4420-4466-8b67-16f87b4ac13c

entryPoint: dds

httpsTrigger:

url: https://us-central1-globalbridge-app.cloudfunctions.net/dds

ingressSettings: ALLOW_ALL

labels:

deployment-tool: cli-gcloud

name: projects/globalbridge-app/locations/us-central1/functions/dds

runtime: python37

serviceAccountEmail: globalbridge-app@appspot.gserviceaccount.com

sourceUploadUrl: https://storage.googleapis.com/gcf-upload-us-central1-120d6d56-5ac1-4f33-8c5e-d92ef0796c7c/a6e93c9e-ae2b-4a0e-8e74-198c072e7a83.zip?GoogleAccessId=service-34968331715@gcf-admin-robot.iam.gserviceaccount.com&Expires=1599902920&Signature=IAyHDRRIlZVxiYfkl89iJlaQdFPgej9O0PpZ%2Fyf39nORCkHRzc28UamqG0IvbZjxPZnujAu9j2KXGymSmGmhcHZcnnPB6BlLHQn%2BqkgJscR%2Fn9ubxknvdaUCpGPUfLvqfWepTHtdL9zqWMv45gQWXXZhIQ1%2BB8dR108vCFDJHSEVx7fgOKmrdUefZ2bDz0F95rXO1Wg86%2FohzU1Odh0FngycKn%2BgMHWkl6cdygsM4j%2BEdtGZMdxwT%2BI5%2B%2FQNXMbmcCOMgeTLDLqV6CwuPUz8LXcgS2taT9wpiVQUEQIrvtHpf1%2Bx8o0WgZ1f3tzXN74Ats1xWGyNgRm4UnCkWLaQEQ%3D%3D

status: ACTIVE

timeout: 60s

updateTime: '2020-09-12T09:00:21.105Z'

versionId: '1'

make -f Makefile_gcf curl_prod

curl \

--header "Content-Type: application/json" \

--data "@./data/sample_deal.json" \

https://us-central1-globalbridge-app.cloudfunctions.net/dds

Error: could not handle the request

{kiat@newton:~/hack/DDS ?}```

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/online-bridge-hackathon/DDS/pull/84#issuecomment-691450332, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABC4PYHNDKKCCHDPTFLYD6DSFM2MZANCNFSM4QQCGMZA .

kiat-huang commented 4 years ago

Yep, you are right, on GCP both prod & curl (make -f Makefile_gcf curl_prod) work for me. I had repeated errors with running the local server though (functions-framework isn't working on my system)


curl \
--header "Content-Type: application/json" \
--data "@./data/sample_deal.json" \
https://us-central1-globalbridge-app.cloudfunctions.net/dds
{"C":{"E":4,"N":9,"S":8,"W":4},"D":{"E":6,"N":7,"S":7,"W":6},"H":{"E":6,"N":6,"S":6,"W":7},"N":{"E":6,"N":7,"S":6,"W":6},"S":{"E":5,"N":8,"S":7,"W":5}}
kiat_huang@cloudshell:~/code/DDS (globalbridge-app)$```
tameware commented 4 years ago

That's good news r.e. the Cloud Shell - thanks for testing.

I'm surprised to hear about problems running functions-framework locally (it was fine on my Mac), but I don't think it should keep us from using Google Cloud Functions since Google nicely provides free Cloud Shell access for the asking.