Closed vijayakumarmariyappan closed 4 years ago
Hi!
The doumentation is a bit unprecise. I will update it accordingly.
Before the controller is executed, the @Valid
annotations makes sure that the model is valid. The json you are providing is not valid, hence springboot rejects it with a 400 BadRequest.
The various validations are needed, since we want to guarantee a constant payload size for every request (so fake and real requests can't be distinguished, hence it is not easy to gather information on possible infected people by just looking at encrypted traffic).
So in your case there are for sure two mistakes:
1) The model for exposed should be a list of keys (plus some more properties) 2) The keys need to be a base64 encoding of 16 bytes (hence 24 characters)
For further validations please take a look at the models and what @Validation
annotations they have.
For example GaenRequest
:
public class GaenRequest {
@NotNull
@NotEmpty
@Valid
@Size(min = 14, max = 30)
@Documentation(description = "Between 14 and 30 Temporary Exposure Keys - zero or more of them might be fake keys. Starting with EN 1.5 it is possible that clients send more than 14 keys.")
private List<GaenKey> gaenKeys;
@NotNull
@Documentation(description = "Prior to version 1.5 Exposure Keys for the day of report weren't available (since they were still used throughout this day RPI=144), so the submission of the last key had to be delayed. This Unix timestamp in milliseconds specifies, which key date the last key (which will be submitted on the next day) will have. The backend then issues a JWT to allow the submission of this last key with specified key date. This should not be necessary after the Exposure Framework is able to send and handle keys with RollingPeriod < 144 (e.g. only valid until submission).")
private Integer delayedKeyDate;
public List<GaenKey> getGaenKeys() {
return this.gaenKeys;
}
public void setGaenKeys(List<GaenKey> gaenKeys) {
this.gaenKeys = gaenKeys;
}
public Integer getDelayedKeyDate() {
return this.delayedKeyDate;
}
public void setDelayedKeyDate(Integer delayedKeyDate) {
this.delayedKeyDate = delayedKeyDate;
}
}
The models themselves should also provide a little bit more feedback to what each parameter means.
Regarding the demo, I can't quite follow what you mean.
We already provide two ways to test and give insights to how the backend is working:
1) You can use the apps and point them to your backend deployments 2) Have a look at the Test files which build the correct payloads
You are of course free to provide a PR adding some postman collections or similar. This is currently for us not high priority as we are preparing for multiple API changes coming in the future.
@ubamrein Thank you for your help :)
How I configured Swagger in https://editor.swagger.io/ visited that tool site, then uploaded this below file/copy paste the content into https://editor.swagger.io/ File: https://github.com/DP-3T/dp3t-sdk-backend/blob/develop/documentation/yaml/sdk.yaml
You had mentioned as below,
**So in your case there are for sure two mistakes:
1.The model for exposed should be a list of keys (plus some more properties)
1.The model for exposed should be a list of keys (plus some more properties) tried 2 API from your swagger documentation. v1/exposed v1/exposedlist
2. The keys need to be a base64 encoding of 16 bytes (hence 24 characters) I have used this below tool and got the 24 characters for key
Though I referred this test case where they are using UTF 8 for encoding base 64. one more key was there "onset". I tried that also, but still 400 response I received. https://github.com/DP-3T/dp3t-sdk-backend/blob/develop/dpppt-backend-sdk/dpppt-backend-sdk-ws/src/test/resources/requests.http
Based on that I have tried with the below POST body, but still I'm getting 400.
Try 1: Try 1 Result:
Try 2: Try 2 Result:
What I'm trying to achieve is, working payload/POST body for your swagger documentation by using https://editor.swagger.io/. If I could see the working copy in Swagger then I can get the idea. Kindly please share the working payload for the APIs.
My Goal: Run the dp3t-sdk-backend, without modification in AWS, then use 2 APIs
As I'm stucked here, I unable to move forward. Kindly please help us regarding tihs. Thank you for your help. Please update me if any further details you needed from your side. Thank you :)
Oh so you are not trying to use the ExposureFramework?
There are two completely different controllers:
DPPPTController
: The legacy endpoint, which is not further developed. All enpoints of the form `/v1/exposed[/{timestamp}]?GAENController
: The currently used and actively developed part, basing on Google and Apple's Exposure Framework. All endpoints of the form /v1/gaen/*
The two approaches have different KeySizes as can be seen by the different ValidationUtils
they are using:
@Value("${ws.app.key_size: 32}")
int keySizeBytes;
@Bean
public ValidationUtils dpptValidationUtils() {
return new ValidationUtils(keySizeBytes, Duration.ofDays(retentionDays), releaseBucketDuration);
}
@Value("${ws.app.gaen.key_size: 16}")
int gaenKeySizeBytes;
@Bean
public ValidationUtils gaenValidationUtils() {
return new ValidationUtils(gaenKeySizeBytes, Duration.ofDays(retentionDays), releaseBucketDuration);
}
For information on what is injected take a look at the constructors of the two controllers in the WSBaseConfig
:
@Bean
public DPPPTController dppptSDKController() {
ValidateRequest theValidator = requestValidator;
if (theValidator == null) {
theValidator = new NoValidateRequest(dpptValidationUtils());
}
return new DPPPTController(dppptSDKDataService(), appSource, exposedListCacheControl, theValidator,
dpptValidationUtils(), releaseBucketDuration, requestTime);
}
@Bean
public GaenController gaenController() {
ValidateRequest theValidator = gaenRequestValidator;
if (theValidator == null) {
theValidator = backupValidator();
}
return new GaenController(insertManager(),gaenDataService(), fakeKeyService(), theValidator, gaenSigner(),
gaenValidationUtils(), Duration.ofMillis(releaseBucketDuration), Duration.ofMillis(requestTime),
Duration.ofMillis(exposedListCacheControl), keyVault.get("nextDayJWT").getPrivate());
}
For the DPPPTController
here is a working CURL request:
POST
curl -vvv -X 'POST' --data '{"key": "MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY=","keyDate": "1596693451000","fake": "0"}' -H 'Content-Type: application/json' http://localhost:8080/v1/exposed
GET
curl -vvv http://localhost:8080/v1/exposed/1596499200000
Btw. if you use Postman Collections it is easier to prepare data in JS. You can import OpenAPI 3 descriptions as well.
@ubamrein
Thank you for your quick reply.
you are not trying to use the ExposureFramework? I want to use the Apple & Google's ExposureFramework.
So you meant to say I have to use /v1/gaen/* Could you please give example for those APIs.
Thanks in advance :+1:
For the Exposure Notification there are multiple different validations going on. It is hard to just give a payload. As I said, the easiest is to use the android app set the URL to your custom backend an intercept the call with a web proxy.
This curl works on the feature/insert-manager
branch (for now)
curl -vvv -H 'Content-Type: application/json' -H 'Accept: */*' -H 'User-Agent: org.dpppt.dp3t;1.0.8;200726.2312.218;iOS;13.6' -H 'Accept-Language: de-ch' --data-binary '{"fake":1,"delayedKeyDate":2661120,"gaenKeys":[{"rollingPeriod":144,"rollingStartNumber":2659536,"transmissionRiskLevel":0,"keyData":"TfBWF24K+jDu4JyBNsYccA==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2659392,"transmissionRiskLevel":0,"keyData":"Uoq\/mwjaU+MZlszweBiLkw==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2659248,"transmissionRiskLevel":0,"keyData":"I6xK64SRRrp+zXMIFYfH8w==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2659104,"transmissionRiskLevel":0,"keyData":"rxPWEr+IlThcU6ZOZPg6UA==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2658960,"transmissionRiskLevel":0,"keyData":"AcudT9l7ccZdQ2jOjbHb9g==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2658816,"transmissionRiskLevel":0,"keyData":"peekB9dvtwebHqSN2UXLgw==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2658672,"transmissionRiskLevel":0,"keyData":"sl7gQUcrxT5LtR6zp+Pg9Q==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2658528,"transmissionRiskLevel":0,"keyData":"Gp8gqWdHvwo4YH3Dc2QVMA==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2658384,"transmissionRiskLevel":0,"keyData":"eWkACtPAD8lF90hZh7BZiw==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2658240,"transmissionRiskLevel":0,"keyData":"Lz87g8bXYs61ZLXg+g5ASQ==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2658096,"transmissionRiskLevel":0,"keyData":"PGCQ1XB8lY\/QTT9cfu\/5kQ==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2657952,"transmissionRiskLevel":0,"keyData":"hUaefAOfgJrEKtBtInI1bA==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2657808,"transmissionRiskLevel":0,"keyData":"qLQQ+6oxfsQhuTxC1jz0aQ==","fake":1},{"rollingPeriod":144,"rollingStartNumber":2657664,"transmissionRiskLevel":0,"keyData":"ntGdHxyPGUPgkgoWPNkLHA==","fake":1}]}' --compressed 'http://localhost:8080/v1/gaen/exposed'
@ubamrein Thank you so much for your help :) So kind of you.
The shared curl works. I need to explain all the pieces to the Mobile team about the payload and each key & data details. That's why I requested you. After completing this exposed API, I'm trying to get the connection details by using the below, but returns as 404. did I do any mistakes?
curl --location --request GET 'http://localhost:8080/v1/gaen/exposed/2661120'
Thank you :+1:
You should look into the newest swagger file, as everything is explained in it.
If you look at the possible responses you see for 404
:
invalid starting key date, doesn't point to midnight UTC- publishedAfter is not at the beginning of a batch release time, currently 2h
Further, as the API documentation states, your date should be in ms since UNIX Epoch and at 00:00 UTC:
Requested date for Exposed Keys retrieval, in milliseconds since Unix epoch (1970-01-01). It must indicate the beginning of a TEKRollingPeriod, currently midnight UTC.
Also, if you look at the models in the API Documentation, you'll find examples and desccriptions on what each parameter means and validations are needed.
@ubamrein Thank you for your help :)
In your previously shared CURL, I saw that delayedKeyDate as 2661120(01/31/1970 @ 7:12pm (UTC)), that was posted to server in v1/gaen/exposed API. I did the below changes in the given POST data
updated all "fake": 1 into "fake": 0, so it will not skip the key while saving. then with those data I tried in localhost, it returned 200 status code. So we exposed the list of keys.
So here to use GET method for the same, 2661120 should be given here. isn't it? I tried with publishedafter=1 & publishedafter=0. But for both I'm getting 404.
I'm using this tool to convert to timestamp. https://www.unixtimestamp.com/index.php
Could you please share the curl for GET method http://localhost:8080/v1/gaen/exposed/:timestamp?publishedafter=0 against your POST curl Example with fake=0
Please review the screenshots, update me if you need more details.
Thank you for your time :)
No, not at all..
delayedKeyDate
: this parameter is used, since we currently (or better recently) cannot get the same day key, since it is still valid on the day of submission. This means we have to delay today's key and upload it tomorrow via /v1/gaen/exposednextday
. Also, delayedKeyDate is in 10 minute intervals since UNIX Epoch. So 2661120
=> date -u -d @$(( 2661120 * 10 * 60 ))
gives you the actual time.
publishedAfter
: Restrict returned Exposed Keys to dates after this parameter. Given in milliseconds since Unix epoch (1970-01-01)
Also see (taken from README):
GET Returns a list of keys, which were used at timestamp. Note that timestamp needs to be epoch milliseconds. Since the behaviour of Android and iOS aren't the same, the optional publishedAfter parameter is added. If set only keys, which were received after publishAfter are returned. This request returns ZIP file containing export.bin and export.sig, where the keys and the signature are stored, as need by the EN framework. The class for signing and serializing is ProtoSignature.
Also note that release of keys is delayed, and aligned with the bucket boundaries (defaults to 2h).
Buckets only exist for 14 days, so everything older than 14 days will always return 404
@ubamrein Thank you so much for answering & writing effort that you put here. I will communicate with mobile team regarding this.
So far what I understood is,
If you have any better ideas, then you could share here. Thank you!
I will stop discussion here and close the issue, as the API is working as expected.
How the app and the backend is working is documented in this resp. the app repositories. If you want further inspirations check out the reference implementations by Google and Apple.
@ubamrein @martinalig
Thank you so much for your contribution to this project. You have provided demo for Android and iOS. That is good. But for Backend project found no clear steps to run. I'm sorry to say this.
I'm from Node JS platform. I can understand Java code as well. Somehow I did run the backend server by using the existing documentation. It runs. If I access the below API 1, then it responded with "Hello from DP3T WS". But for all other APIs, it returns 400 as status code. according to the documentation It says "Invalid base64 encoding in expose request". I really tried various input data inside body, but it always returns the same. Then I was thinking it has an issue in my jar file. Then I loaded "sdk.yaml" file in the path "/home/vijay/Desktop/dp3t-sdk-backend/documentation/yaml" in https://editor.swagger.io/.
I tried the below on that swagger editor, found that same behaviour as my local jar.
API 1: Request: GET http://localhost:8080/v1 OR https://demo.dpppt.org/v1 Response: Hello from DP3T WS
API 2: Request: POST http://localhost:8080/v1/exposed OR https://demo.dpppt.org/v1/exposed Body: { "key": "dGVzdA==", "keyDate": 1596672000, }
Here dGVzdA== is base64 encoded string of word "test". 1596672000 - timestamp is equivalent to 08/06/2020 @ 12:00am (UTC)- converted by https://www.unixtimestamp.com/index.php
Response: 400
I would like to know how can I use the backend app properly. is the below correct? First I'm storing the exposee's unique id(Example:email-id) in DB, by using this API(https://demo.dpppt.org/v1/exposed) Second I can get the list of friend who may have affected by his contact, by using https://demo.dpppt.org/v1/gaen/exposed/:timestamp
Did i do any mistakes on sending the data in the POST body? Kindly please help us regarding this. If you could add demo for backend, really really helpful to everyone. Thanks a lot :+1: