Upload straight to S3

[suhlrich]

We will now upload videos from the phone straight to S3, instead of the current solution phone -->API. @xhardcorex it'd be great if you could implement the following python code in place of the current Alamofire upload

In summary, instead of the 1 POST request, there will be 3 requests: 1) GET an S3 link and credentials from API 2) POST the video to the S3 link and credentials gotten from above 3) PATCH the video link to the same upload url as used before, put the video link in parameters, don't upload the video itself

sessionURL = self.sessionStatusUrl # this is what Swift is pinging, but remove the 'status/?device_id=...' from the end
url_link= sessionURL + '/get_presigned_url/'

r_getLink = requests.get(url_link).json()

# wherever the video is on the iphone
videoFile = {'file': open('C:/Users/suhlr/Downloads/testMovie.mov', 'rb')}

# this should upload the video to S3, with credentials in the ['fields'] field
r_S3 = requests.post(r_getLink['url'], data=r_getLink['fields'], files=videoFile)

# once finished uploading to S3, send the link and parameters to the api similar to before
# videoURL is the url that we are currently using in AF.upload request
parameters = {'fov': myFOV, 'model': myModel, 'video_url':r_getLink['fields']['key']}
requests.patch(videoURL,data=parameters)

The updated API functionality is only implemented in the dev api (https://dev.opencap.ai). You can test by logging onto dev.app.opencap.ai. Slack us if you don't have credentials.

[xhardcorex]

Got it. I will implement this task.

[xhardcorex]

Hi there! I am getting the following error. Is there something wrong on my side?

<?xml version="1.0" encoding="UTF-8"?>

<Error>
    <Code>AccessDenied</Code>
    <Message>Invalid according to Policy: Policy expired.</Message>
    <RequestId>FR8H4MM93QZTPPVW</RequestId>
    <HostId>BzJiMRweMJGtLbO658nSZl4jPslquDwRhPNLTGG/4HYfUyAXebmwue72hyNmA43O2VpWSZ3+wEs=</HostId>
</Error>

curl --location 'https://mc-mocap-video-storage.s3.amazonaws.com/' \ --header 'Host: mc-mocap-video-storage.s3.amazonaws.com' \ --header 'Connection: keep-alive' \ --header 'Accept: /' \ --header 'User-Agent: OpenCap/1.6 (com.stanfordnmbl.OpenCap1; build:21; iOS 16.3.1) Alamofire/5.4.0' \ --header 'Accept-Language: en-UA;q=1.0, ru-UA;q=0.9, uk-UA;q=0.8' \ --header 'Pragma: no-cache' \ --header 'Cache-Control: no-cache' \ --form 'key="\"e51d6f1d-924e-415d-9a6c-2eed311eccd0.mov"' \ --form 'AWSAccessKeyId="AKIAZTRKQGHOES33JEGA"' \ --form 'policy="eyJleHBpcmF0aW9uIjogIjIwMjMtMDQtMTdUMTQ6MzE6MDJaIiwgImNvbmRpdGlvbnMiOiBbeyJidWNrZXQiOiAibWMtbW9jYXAtdmlkZW8tc3RvcmFnZSJ9LCB7ImtleSI6ICJlNTFkNmYxZC05MjRlLTQxNWQtOWE2Yy0yZWVkMzExZWNjZDAubW92In1dfQ=="' \ --form 'signature="abBPxlw+n8xql2pYrIr2zAG4eoA="' \ --form 'file=@"BtX4vIgrR/recording.mov"'

[antoinefalisse]

@xhardcorex is this something that used to work or is it related to #26?

[suhlrich]

it's on the new upload straight to S3. I'm trying to duplicate...will be back with you soon.

[xhardcorex]

Looks like request /get_presigned_url stopped working now and returns 404 but worked properly yesterday. Could you please take a look?

curl 'https://dev.opencap.ai/sessions/638db547-4a86-406a-a993-4ab7538fc5bb/get_presigned_url/' \ -H 'Host: dev.opencap.ai' \ -H 'Accept: /' \ -H 'Accept-Language: en-GB,en;q=0.9' \ -H 'Connection: keep-alive' \ -H 'User-Agent: OpenCap/21 CFNetwork/1404.0.5 Darwin/22.3.0' \ -H 'Pragma: no-cache' \ -H 'Cache-Control: no-cache' \ --proxy http://localhost:9090

[suhlrich]

sorry. I messed something up on the dev api. it is working again now. sorry for the inconvenience.

[xhardcorex]

@suhlrich I am still getting this error on uploading video to S3. I atttached error message and curl to reproduce. Could you please check policy configs? Thanks!

curl --location 'https://mc-mocap-video-storage.s3.amazonaws.com/' \
--header 'Host: mc-mocap-video-storage.s3.amazonaws.com' \
--header 'Connection: keep-alive' \
--header 'Accept: */*' \
--header 'User-Agent: OpenCap/1.6 (com.stanfordnmbl.OpenCap1; build:21; iOS 16.3.1) Alamofire/5.4.0' \
--header 'Accept-Language: en-UA;q=1.0, ru-UA;q=0.9, uk-UA;q=0.8' \
--header 'Pragma: no-cache' \
--header 'Cache-Control: no-cache' \
--form 'key="1602be65-e8a0-491a-a6ef-4d1119e4eff4.mov"' \
--form 'AWSAccessKeyId="AKIAZTRKQGHOES33JEGA"' \
--form 'policy="eyJleHBpcmF0aW9uIjogIjIwMjMtMDQtMTlUMDg6MjU6MDFaIiwgImNvbmRpdGlvbnMiOiBbeyJidWNrZXQiOiAibWMtbW9jYXAtdmlkZW8tc3RvcmFnZSJ9LCB7ImtleSI6ICIxNjAyYmU2NS1lOGEwLTQ5MWEtYTZlZi00ZDExMTllNGVmZjQubW92In1dfQ=="' \
--form 'signature="6cXSySlz9WgO/TK4kcw/MvXBnqY="' \
--form 'file=@"GVWcaYV_5/IMG_1465.MOV"'

<Error>
    <Code>AccessDenied</Code>
    <Message>Invalid according to Policy: Policy expired.</Message>
    <RequestId>M3Z5NYBNNCEG9B3D</RequestId>
    <HostId>Lal3g8cX/CRqPUpNQiiCil/N97rhnSdPjWrq3HeOJS4sB2UEl/sNgCSE1xQ3kdJ4AC2tZyM1Tno=</HostId>
</Error>

[xhardcorex]

I finished this task. Please take a look new build 1.6 (22)