search

redguava / cliniko-api

The API for Cliniko
74 stars 39 forks source link

Need help attaching files to a patient, receiving an error #290

Closed BrianJustice closed 4 years ago

BrianJustice commented 4 years ago

I need someone to help, we are attempting to attach files to a patient record via the api but getting an error message. support@cliniko.com says the API request looks good and recommended I reach out here. You can reach me at brian.justice@justicetechnologygroup.com. Thanks.

hagen commented 4 years ago

Hi @BrianJustice

could you supply some more information about where you're having trouble, and maybe some requests/responses so we can help you troubleshoot?

Cheers!

BrianJustice commented 4 years ago

Yeah sure thing The details are below:

URL: POST https://api.cliniko.com/v1/patient_attachments

Request body: { "description": "Patient Form", "patient_id": 59329514, "upload_url": " https://cliniko-files-production-ap-southeast-2.s3.ap-southeast-2.amazonaws.com/62657/patients/F59329514/attachments/temp/e06946c8-9cb2-4251-aedb-22a362f0dfe6/BrianJustice.pdf " }

API Response: { "message": "Validation Failed", "errors": { "temp_attachment_url": "is invalid" } }

Please let me know if more information is needed.

On Sun, Dec 15, 2019 at 9:43 PM Hagen notifications@github.com wrote:

Hi @BrianJustice https://github.com/BrianJustice

could you supply some more information about where you're having trouble, and maybe some requests/responses so we can help you troubleshoot?

Cheers!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/redguava/cliniko-api/issues/290?email_source=notifications&email_token=AFRTWD4LBUVF6G5FHAPYKODQY3TNPA5CNFSM4J3DJ4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG5K67A#issuecomment-565882748, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFRTWD5F5FLXBNLAXN22K73QY3TNPANCNFSM4J3DJ4TA .

-- Brian Justice Managing Director (239) 823-1912 http://justicetechnologygroup.com

hagen commented 4 years ago

Hi @BrianJustice - couple things. That upload_url you're using has a few issues:

<PostResponse>
  <Location>https://cliniko-files.s3.amazonaws.com/1%2Fpatients%2F123456789%2Fattachments%2Ftemp%2Fb325fe81-91b3-49b4-a0d8-1ea4dbe69293%2Ff2674105b21b0e6e9795a2535d71d53c.jpeg</Location>
  <Bucket>cliniko-files</Bucket>
  <Key>1/patients/123456789/attachments/temp/b325fe81-91b3-49b4-a0d8-1ea4dbe69293/f2674105b21b0e6e9795a2535d71d53c.jpeg</Key>
  <ETag>"893195af71c3cf544a147d717d3c31d8"</ETag>
</PostResponse>

Have you seen the guide for this? There are three distinct steps to upload patient attachments.

Cheers!

BrianJustice commented 4 years ago

Thanks for you response. I have seen the guide, that is what we followed to get to this point. I believe that is the URL that is being returned from the presigned endpoint, however, we will double check.

Additionally, I think overall it would help greatly to have more detailed messaging returned from the API. Simply having 'Invalid URL' or 'Validation failed' is not actionable from a developer standpoint and does not help me troubleshoot what the issue is. I've seen these generic error messages on various endpoints throughout the API and have seen other developers via the group have similar issues and are forced to reach out via email, I think more detailed messaging could go a long way in helping developers integrate with your API and lower the volume of emails you see coming in.

On Mon, Dec 16, 2019 at 5:44 PM Hagen notifications@github.com wrote:

Hi @BrianJustice https://github.com/BrianJustice - couple things. That upload_url you're using has a few issues:

  • our S3 bucket is not cliniko-files-production-ap-southeast-2, it's just cliniko-files. Make sure you're using the URL returned in your attachment_presigned_post request as the base, as this will have the correct bucket URL. For example, see the url prop in this response:

{ "url":"https://cliniko-files.s3.amazonaws.com", "fields":{ "key":"1/patients/123456789/attachments/temp/b325fe81-91b3-49b4-a0d8-1ea4dbe69293/${filename}", ... } }

  • the path in that URL 62657/patients/F59329514/... is also wrong - we do not have any non-numeric characters in our patient IDs (so the letter F shouldn't be there). I think you've somehow kept a partially encoded / character, which is %2F. Make sure you're using the exact Key value returned from S3 when you upload the file to its temp bucket location. That response from S3 will look something like this:
https://cliniko-files.s3.amazonaws.com/1%2Fpatients%2F123456789%2Fattachments%2Ftemp%2Fb325fe81-91b3-49b4-a0d8-1ea4dbe69293%2Ff2674105b21b0e6e9795a2535d71d53c.jpeg cliniko-files 1/patients/123456789/attachments/temp/b325fe81-91b3-49b4-a0d8-1ea4dbe69293/f2674105b21b0e6e9795a2535d71d53c.jpeg "893195af71c3cf544a147d717d3c31d8"

Have you seen the guide https://github.com/redguava/cliniko-api/blob/master/guides/uploading_patient_attachments.md for this? There are three distinct steps to upload patient attachments.

Cheers!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/redguava/cliniko-api/issues/290?email_source=notifications&email_token=AFRTWD46FXCCXBDLYTBBG2LQZAADDA5CNFSM4J3DJ4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHAMGSY#issuecomment-566281035, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFRTWD7CUBECX6OIG7LVWOLQZAADDANCNFSM4J3DJ4TA .

-- Brian Justice Managing Director (239) 823-1912 http://justicetechnologygroup.com

hagen commented 4 years ago

Thanks for the suggestion @BrianJustice - appreciate the feedback!

On the URL validation for patient attachments specifically, that error is what we get from S3 when we try to fetch the temp file. The flow is:

  1. request presigned post from Cliniko; Cliniko gives you the temp location
  2. Upload directly to S3 temp location; S3 tells you the exact Key for the file
  3. Tell Cliniko the exact file URL (bucket + key); Cliniko goes to fetch file from S3.

In that last step, 3, Cliniko is asking S3 for the file you've specified in the request. S3 - not wanting to divulge anything more than it has to about incorrect requests - simply tells us that the URL is invalid (essentially a 404) if that URL isn't exactly correct. So that's all we can send back to you.

Many of our endpoints do have descriptive errors - try sending a malformed email to the /patients endpoint and you'll get a specific error back. That said, if you've got suggestions about which endpoints we could provide better errors on (and which errors), please let us know! 😊

Thanks!

hagen commented 4 years ago

Sorry - didn't mean to close!

BrianJustice commented 4 years ago

Yeah no worries, I'm simply stating what I think would help me avoid having to reach out.

However, on your note on the file url, here is a response to a request I sent tonight when uploading a file to S3:

On Wed, Dec 18, 2019 at 4:48 PM Hagen notifications@github.com wrote:

Thanks for the suggestion @BrianJustice https://github.com/BrianJustice

  • appreciate the feedback!

On the URL validation for patient attachments specifically, that error is what we get from S3 when we try to fetch the temp file. The flow is:

  1. request presigned post from Cliniko; Cliniko gives you the temp location
  2. Upload directly to S3 temp location; S3 tells you the exact Key for the file
  3. Tell Cliniko the exact file URL (bucket + key); Cliniko goes to fetch file from S3.

In that last step, 3, Cliniko is asking S3 for the file you've specified in the request. S3 - not wanting to divulge anything more than it has to about incorrect requests - simply tells us that the URL is invalid (essentially a 404) if that URL isn't exactly correct. So that's all we can send back to you.

Many of our endpoints do have descriptive errors - try sending a malformed email to the /patients endpoint and you'll get a specific error back. That said, if you've got suggestions about which endpoints we could provide better errors on (and which errors), please let us know! 😊

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/redguava/cliniko-api/issues/290?email_source=notifications&email_token=AFRTWD2R2BBIHFA5F4RMI4LQZKLDNA5CNFSM4J3DJ4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHHTKXY#issuecomment-567227743, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFRTWD2F5GAGRXHYUYSOGTLQZKLDNANCNFSM4J3DJ4TA .

-- Brian Justice Managing Director (239) 823-1912 http://justicetechnologygroup.com

hagen commented 4 years ago

Hey @BrianJustice re. the bucket, thanks for pointing that out. Absolutely use whichever bucket URL is listed in the presigned post response. We've very recently (like last 2 days) deployed some major stuff for sharding - I hadn't noticed that particular change 🙊

This URL (for AU accounts) is currently: https://cliniko-files-production-ap-southeast-2.s3-ap-southeast-2.amazonaws.com/

From the guide for attachments:

To do so, make a POST request to the patient attachments endpoint, with the patient ID and s3 temp URL as parameters. The upload_url parameter is the combination of url value in the presigned post response + Key value in the s3 XML response.

So the url value from the presigned post response, and the Key element in the XML response 👌

BrianJustice commented 4 years ago

Got it. Thanks, I'll ensure we're encoding/decoding that URL properly and then resend. I appreciate all your help and the grace in which you handled my suggestions :)

On Wed, Dec 18, 2019 at 8:41 PM Hagen notifications@github.com wrote:

Hey @BrianJustice https://github.com/BrianJustice re. the bucket, thanks for pointing that out. Absolutely use whichever bucket URL is listed in the presigned post response. We've very recently (like last 2 days) deployed some major stuff for sharding - I hadn't noticed that particular change 🙊

This URL (for AU accounts) is currently:

https://cliniko-files-production-ap-southeast-2.s3-ap-southeast-2.amazonaws.com/

From the guide for attachments:

To do so, make a POST request to the patient attachments endpoint, with the patient ID and s3 temp URL as parameters. The upload_url parameter is the combination of url value in the presigned post response + Key value in the s3 XML response.

So the url value from the presigned post response, and the Key element in the XML response 👌

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/redguava/cliniko-api/issues/290?email_source=notifications&email_token=AFRTWD2CRQAHMXQ4HH5SIBDQZLGNJA5CNFSM4J3DJ4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHIDA3Q#issuecomment-567292014, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFRTWD2WSWJ6EB3JSND236DQZLGNJANCNFSM4J3DJ4TA .

-- Brian Justice Managing Director (239) 823-1912 http://justicetechnologygroup.com

BrianJustice commented 4 years ago

Hey Hagen,

So we were able to decode the URL and have successfully uploaded files however, when I go to view them in the Patient record it says that they are still processing, it has been over 2 hours now... what is the standard time for these to process? Are you able to look into a specific upload and see if there may be an issue?

On Wed, Dec 18, 2019 at 8:43 PM Brian Justice < brian.justice@justicetechnologygroup.com> wrote:

Got it. Thanks, I'll ensure we're encoding/decoding that URL properly and then resend. I appreciate all your help and the grace in which you handled my suggestions :)

On Wed, Dec 18, 2019 at 8:41 PM Hagen notifications@github.com wrote:

Hey @BrianJustice https://github.com/BrianJustice re. the bucket, thanks for pointing that out. Absolutely use whichever bucket URL is listed in the presigned post response. We've very recently (like last 2 days) deployed some major stuff for sharding - I hadn't noticed that particular change 🙊

This URL (for AU accounts) is currently:

https://cliniko-files-production-ap-southeast-2.s3-ap-southeast-2.amazonaws.com/

From the guide for attachments:

To do so, make a POST request to the patient attachments endpoint, with the patient ID and s3 temp URL as parameters. The upload_url parameter is the combination of url value in the presigned post response + Key value in the s3 XML response.

So the url value from the presigned post response, and the Key element in the XML response 👌

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/redguava/cliniko-api/issues/290?email_source=notifications&email_token=AFRTWD2CRQAHMXQ4HH5SIBDQZLGNJA5CNFSM4J3DJ4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHIDA3Q#issuecomment-567292014, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFRTWD2WSWJ6EB3JSND236DQZLGNJANCNFSM4J3DJ4TA .

-- Brian Justice Managing Director (239) 823-1912 http://justicetechnologygroup.com

-- Brian Justice Managing Director (239) 823-1912 http://justicetechnologygroup.com

hagen commented 4 years ago

That usually happens when there's a discrepancy between the Key name and the value you specified in the as the upload_url. Could you please confirm the bucket + Key value in the XML response is the same (unencoded) as the value you pass Cliniko in the finalPOST` request? You can also send me the attachment ID and I'll look it up 😊

When an attachment is stuck in processing, it means Cliniko has gone to the bucket with the upload_url you've supplied, but S3 replies with a 404. It's usually a very tiny difference - something might have been encoded, or a character replaced; maybe an additional /, and so on.

BrianJustice commented 4 years ago

Thanks Hagen,

Here are a couple of the attachment ID's:

24340805 24340786 24340785

On Thu, Jan 2, 2020 at 5:27 PM Hagen notifications@github.com wrote:

That usually happens when there's a discrepancy between the Key name and the value you specified in the as the upload_url. Could you please confirm the bucket + Key value in the XML response is the same (unencoded) as the value you pass Cliniko in the final POST` request? You can also send me the attachment ID and I'll look it up 😊

When an attachment is stuck in processing, it means Cliniko has gone to the bucket with the upload_url you've supplied, but S3 replies with a

  1. It's usually a very tiny difference - something might have been encoded, or a character replaced; maybe an additional /, and so on.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/redguava/cliniko-api/issues/290?email_source=notifications&email_token=AFRTWD36KKESSU25W6B6NALQ3ZS3ZA5CNFSM4J3DJ4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH7UPVQ#issuecomment-570378198, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFRTWD34VZBU2RJQQKP7Y6LQ3ZS3ZANCNFSM4J3DJ4TA .

-- Brian Justice Managing Director (239) 823-1912 http://justicetechnologygroup.com

BrianJustice commented 4 years ago

Hey Hagan,

Any update on this?

On Thu, Jan 2, 2020 at 7:26 PM Brian Justice < brian.justice@justicetechnologygroup.com> wrote:

Thanks Hagen,

Here are a couple of the attachment ID's:

24340805 24340786 24340785

On Thu, Jan 2, 2020 at 5:27 PM Hagen notifications@github.com wrote:

That usually happens when there's a discrepancy between the Key name and the value you specified in the as the upload_url. Could you please confirm the bucket + Key value in the XML response is the same (unencoded) as the value you pass Cliniko in the final POST` request? You can also send me the attachment ID and I'll look it up 😊

When an attachment is stuck in processing, it means Cliniko has gone to the bucket with the upload_url you've supplied, but S3 replies with a

  1. It's usually a very tiny difference - something might have been encoded, or a character replaced; maybe an additional /, and so on.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/redguava/cliniko-api/issues/290?email_source=notifications&email_token=AFRTWD36KKESSU25W6B6NALQ3ZS3ZA5CNFSM4J3DJ4TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH7UPVQ#issuecomment-570378198, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFRTWD34VZBU2RJQQKP7Y6LQ3ZS3ZANCNFSM4J3DJ4TA .

-- Brian Justice Managing Director (239) 823-1912 http://justicetechnologygroup.com

-- Brian Justice Managing Director (239) 823-1912 http://justicetechnologygroup.com

hagen commented 4 years ago

Hi @BrianJustice - thanks for the bump. Will DM you. Closing.