search

vegaprotocol / vega

A Go implementation of the Vega Protocol, a protocol for creating and trading derivatives on a fully decentralised network.
https://vega.xyz
GNU Affero General Public License v3.0
38 stars 22 forks source link

[GraphQL][DEVNET] Prepare New Market Proposal does not parse the correct dates #2113

Closed nzbaen closed 4 years ago

nzbaen commented 4 years ago

Problem encountered

Submitting a signed tx for a New Market Proposal prepared in GraphQL fails because of the date in closingDatetime and enactmentDatetime is not parsed correctly

Environment: Devnet

System response

The New Market Proposal fails with the message: PROPOSAL_ERROR_CLOSE_TIME_TOO_SOON

System version: Specify the system version (0.xx.xx) Protocol: Specify the protocol

Component: Specify the components that might be related

Steps to reproduce

Manual

Steps to reproduce the behaviour manually:

  1. Go to GraphQL playground
  2. Submit a New Market Proposal mutation (see below example) 
    mutation { prepareProposal(
partyId:"996d6750e548a8ee544841a65ff6213bd8e6b3df8984cfed92bbb615f317c806",
reference:"ETHVUSD/DEC20/DOM",
proposalTerms:{
  closingDatetime:"2020-08-26T15:29:52Z",
  enactmentDatetime:"2020-09-02T15:44:52Z",
  newMarket:{
    instrument:{
      name:"December 2020 ETH vs VUSD future DOM",
      code:"CRYPTO:ETHVUSD/DEC20",
      baseName:"DOM",
      quoteName:"VUSD",
      futureProduct:{
        maturity:"2020-12-31T23:59:59Z",
        asset:"VUSD"
      }
    }
    decimalPlaces: 5,
    riskParameters:{
      logNormal:{
        riskAversionParameter:0.001,
        tau:1,
        params:{
          mu:0,
          r:1,
          sigma:1
        }
      }
    }
metadata:[
      "asset_class:fx/crypto",
      "product:futures"
    ]
    continuousTrading:{
      tickSize:"2592000000000000"
    }
  }
}
)
{
blob
}
}
  3. Save the blob, sign the tx and submit it through REST
  4. The New Market Proposal is rejected, by looking at the list of the Market Proposals we can see the following (example of the last market created with the above dates): 
    {
  "proposal": {
    "ID": "P0000083275-0000000009",
    "reference": "ETHVUSD/DEC20/DOM",
    "partyID": "36200f07ea669c8349409de8a1bfe1027ba2f94b2fa98570d85842272bee90b4",
    "state": "STATE_REJECTED",
    "timestamp": "1597849580825039668",
    "terms": {
      "closingTimestamp": "1597850478",
      "enactmentTimestamp": "1597851378",
      "validationTimestamp": "0",
      "newMarket": {
        "changes": {
          "instrument": {
            "name": "December 2020 ETH vs VUSD future DOM",
            "code": "CRYPTO:ETHVUSD/DEC20",
            "baseName": "DOM",
            "quoteName": "VUSD",
            "future": {
              "maturity": "2020-12-31T23:59:59Z",
              "asset": "VUSD"
            }
          },
          "decimalPlaces": "5",
          "metadata": [
            "asset_class:fx/crypto",
            "product:futures"
          ],
          "openingAuctionDuration": "0",
          "logNormal": {
            "riskAversionParameter": 0.001,
            "tau": 1,
            "params": {
              "mu": 0,
              "r": 1,
              "sigma": 1
            }
          },
          "continuous": {
            "tickSize": ""
          }
        }
      }
    },
    "reason": "PROPOSAL_ERROR_CLOSE_TIME_TOO_SOON"
  },
  "yes": [],
  "no": [],
  "yesParty": {},
  "noParty": {}
}
]
}

By checking the date and transforming it into ISO we can see that the closingTimestamp and enactmentTimestamp do not reflect the parsed dates

          "closingTimestamp": "1597850478",
          "enactmentTimestamp": "1597851378",

in ISO

[16:17:18][INFO] Proposal Closing Time ...: Wed 19 Aug 16:21:18 BST 2020
[16:17:18][INFO] Proposal Enact Time .....: Wed 19 Aug 16:36:18 BST 2020

Automation

You can test this manually by cloning the QA repo and checkout the develop branch:

  1. Clone git clone git@github.com:vegaprotocol/qa.git
  2. Checkout git checkout origin develop
  3. Pull git pull origin develop --rebase
  4. Execute the automation under REST_API called proposals_gql.sh 
    ./proposals_gql.sh devnet.conf

    NOTE You can modify the .conf using your details

The automation's output will show you the commands executed and the following info:

[16:40:44][INFO] Time defined:
[16:40:44][INFO] closingDatetime: 2020-08-26T15:55:42Z
[16:40:44][INFO] enactmentDatetime: 2020-09-02T16:10:42Z
[16:40:48][INFO] ####################################################
[16:40:48][INFO] Proposal ID .............: P0000083275-0000000009
[16:40:48][INFO] Proposal Reference ......: ETHVUSD/DEC20/DOM
[16:40:48][INFO] Proposal Pary ID ........: 36200f07ea669c8349409de8a1bfe1027ba2f94b2fa98570d85842272bee90b4
[16:40:48][INFO] Proposal Closing Time ...: Wed 19 Aug 16:21:18 BST 2020
[16:40:48][INFO] Proposal Enact Time .....: Wed 19 Aug 16:36:18 BST 2020
[16:40:48][INFO] Proposal State ..........: STATE_REJECTED
[16:40:48][INFO] Proposal Reason .........: PROPOSAL_ERROR_CLOSE_TIME_TOO_SOON
[16:40:48][INFO] More info in .........: /tmp/tmp.l6ES06MQJJ

You can check the tmp file (it's a JSON file) for more info on the proposal. To change the GraphQL query just edit the proposals_gql.sh

Expected behaviour

The New Market Proposal is submitted using the specified dates

edd commented 4 years ago

just pasting some investigation here:

Console is submitting:

curl 'https://n03.s.vega.xyz/query' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0' -H 'Accept: */*' -H 'Accept-Language: en-GB,en;q=0.5' --compressed -H 'Referer: https://staging.vega.trading/' -H 'content-type: application/json' -H 'Origin: https://staging.vega.trading' -H 'DNT: 1' -H 'Connection: keep-alive' -H 'TE: Trailers' --data-raw '{"operationName":"prepareProposal","variables":{"partyId":"aef0f72ec9fd2f1990e483ad51604268b5d701b24f3d46fac3f8dae3b4ae9449","proposalTerms":{"closingDatetime":"2020-08-26T15:53:30.300Z","enactmentDatetime":"2020-09-26T00:00:00.000Z","newMarket":{"decimalPlaces":5,"metadata":["take2"],"instrument":{"code":"2fdd696ac7787d98afcbc136d4cf17bc595f7c0c6d3bf57c0ceeb5184347b725/5/20.11.28","name":"DOM/IS/STILL/WRONG","baseName":"2fdd696ac7787d98afcbc136d4cf17bc595f7c0c6d3bf57c0ceeb5184347b725","quoteName":"5","futureProduct":{"maturity":"2020-11-28T00:00:00.000Z","asset":"2fdd696ac7787d98afcbc136d4cf17bc595f7c0c6d3bf57c0ceeb5184347b725"}},"riskParameters":{"logNormal":{"riskAversionParameter":0.1,"tau":0.1,"params":{"mu":0,"r":0.1,"sigma":0.1}}},"continuousTrading":{}}}},"query":"mutation prepareProposal($partyId: String!, $proposalTerms: ProposalTermsInput!) {\n  prepareProposal(partyId: $partyId, proposalTerms: $proposalTerms) {\n    blob\n    __typename\n  }\n}\n"}'

So the time we're sending is:

{
"closingDatetime":"2020-08-26T15:53:30.300Z",
"enactmentDatetime":"2020-09-26T00:00:00.000Z",
}
{
  "data": {
    "prepareProposal": {
      "blob": "ZWJhMmQ3YzYtODZmMi00MWUyLWJhN2YtNjk3MDUwNjUzMDc3RRIkODU4Mzg4MTctOWNhYy00NTYyLTljMTItODZkMjhjZWU1NmY1GkBhZWYwZjcyZWM5ZmQyZjE5OTBlNDgzYWQ1MTYwNDI2OGI1ZDcwMWIyNGYzZDQ2ZmFjM2Y4ZGFlM2I0YWU5NDQ5IAIy0AII+oqa+gUQgIm6+wWyBsACCr0CCoUCChJET00vSVMvU1RJTEwvV1JPTkcSSzJmZGQ2OTZhYzc3ODdkOThhZmNiYzEzNmQ0Y2YxN2JjNTk1ZjdjMGM2ZDNiZjU3YzBjZWViNTE4NDM0N2I3MjUvNS8yMC4xMS4yOBpAMmZkZDY5NmFjNzc4N2Q5OGFmY2JjMTM2ZDRjZjE3YmM1OTVmN2MwYzZkM2JmNTdjMGNlZWI1MTg0MzQ3YjcyNSIBNaIGXAoYMjAyMC0xMS0yOFQwMDowMDowMC4wMDBaEkAyZmRkNjk2YWM3Nzg3ZDk4YWZjYmMxMzZkNGNmMTdiYzU5NWY3YzBjNmQzYmY1N2MwY2VlYjUxODQzNDdiNzI1EAUaBXRha2UyqgYmCZqZmZmZmbk/EZqZmZmZmbk/GhIRmpmZmZmZuT8ZmpmZmZmZuT/CDAA=",
      "__typename": "PreparedProposal"
    }
  }
}

curl 'https://wallet.s.vega.xyz/api/v1/messages' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0' -H 'Accept: */*' -H 'Accept-Language: en-GB,en;q=0.5' --compressed -H 'Referer: https://staging.vega.trading/' -H 'authorization: Bearer eyJhbGciOiJQUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MjkyNzgyODIsImlzcyI6InZlZ2Egd2FsbGV0IiwiU2Vzc2lvbiI6ImViMjdhZjAzNTJmNGEzZGQyNmQ3NzhjZTkyOTY2NmE1ZGFlMzVmODgzMTgyMzk2MzljNmJkYzY5OWI0MGY3ZTEiLCJXYWxsZXQiOiJlZGQifQ.UqEw-thJ4AdzQ2TsCSmcuGTWQPOP_1zJ00h8AK8gVfXPE7rTpmdNh2WEGGbw-VVa8XsjBNCyOzOYgv-D3gK9UFStWEmft4Itlb8MQIGjfXdhHf1Zp3IpFRB0tQKt7537yHhGH0NYignPL-lNWD0p_p16fRxePkiEKM2YZPw2MjKn_wM_0XCNXS97GY5gyPzkA_FouSbO-ywdhMON1R6xESLM06xNjiuqke9_8fAXu8Xj7Q0PoxWzSTmCNzRUqHhtXyv8mMYv5XFfiNsN_HMxxiIUqfXCrpU0fKHSxsSoc2J9JexEtk_2rczBV38HEj3vDgFlsuxpwTI8diClRAYNGYw4-SiM_Gg5mBs12outdRx-qFwMIscZB_MfedZyipv0y4hlb92bX70UT5x_yKSGZdy4OMLATTRR7WAsRyVtZ1ttfHC8z0D0wbOB7BwkvzJJq32CabDmR4BGV4Q9RKrhLZGwZeSUwiuinkBAC6tnJDirqpIgdaf_C9nbxPGHDOPdQdlLO7po8fVR9cdzmVDesc0nzskhMtD5F-XHihgOPHNwH_gM9jPNVig_JYgfamR3BFO6jzP_G4n_3CJ7VWQQZS-Ll3r50rI7LFjstY6s3cV3zhGHjsM-NNGOdiSmXtRQJRIQgjIhKW2N-bxAtWonk1b8cH8YiNamlhXwaosqaeQ' -H 'Content-Type: text/plain;charset=UTF-8' -H 'Origin: https://staging.vega.trading' -H 'DNT: 1' -H 'Connection: keep-alive' -H 'TE: Trailers' --data-raw '{"tx":"ZWJhMmQ3YzYtODZmMi00MWUyLWJhN2YtNjk3MDUwNjUzMDc3RRIkODU4Mzg4MTctOWNhYy00NTYyLTljMTItODZkMjhjZWU1NmY1GkBhZWYwZjcyZWM5ZmQyZjE5OTBlNDgzYWQ1MTYwNDI2OGI1ZDcwMWIyNGYzZDQ2ZmFjM2Y4ZGFlM2I0YWU5NDQ5IAIy0AII+oqa+gUQgIm6+wWyBsACCr0CCoUCChJET00vSVMvU1RJTEwvV1JPTkcSSzJmZGQ2OTZhYzc3ODdkOThhZmNiYzEzNmQ0Y2YxN2JjNTk1ZjdjMGM2ZDNiZjU3YzBjZWViNTE4NDM0N2I3MjUvNS8yMC4xMS4yOBpAMmZkZDY5NmFjNzc4N2Q5OGFmY2JjMTM2ZDRjZjE3YmM1OTVmN2MwYzZkM2JmNTdjMGNlZWI1MTg0MzQ3YjcyNSIBNaIGXAoYMjAyMC0xMS0yOFQwMDowMDowMC4wMDBaEkAyZmRkNjk2YWM3Nzg3ZDk4YWZjYmMxMzZkNGNmMTdiYzU5NWY3YzBjNmQzYmY1N2MwY2VlYjUxODQzNDdiNzI1EAUaBXRha2UyqgYmCZqZmZmZmbk/EZqZmZmZmbk/GhIRmpmZmZmZuT8ZmpmZmZmZuT/CDAA=","pubKey":"aef0f72ec9fd2f1990e483ad51604268b5d701b24f3d46fac3f8dae3b4ae9449","propagate":true}'

Wondering if the date format is wrong on your submission?

nzbaen commented 4 years ago

Update: this happens on devnet only, stagnet gets the correct dates

nzbaen commented 4 years ago

I've ran few tests and validated that the dates now seems to be one hour ahead:

Market proposal submission JSON dates:

closingDatetime:"2020-09-15T14:38:26Z",
enactmentDatetime:"2020-09-22T14:53:26Z"

Dates shown on the Market proposal:

[15:23:38][INFO] Proposal Closing Time ...: Tue 15 Sep 15:38:26 BST 2020
[15:23:38][INFO] Proposal Enact Time .....: Tue 22 Sep 15:53:26 BST 2020

In the script the dates are chosen with the following code:

datenow=$(curl -s "${base_url}"/statistics | jq -r .vegaTime)
dateclose=$(date -d "${datenow} + 7 days - 1 hours + 15 minutes" +"%Y-%m-%dT%H:%M:%SZ")
datenow=$(date -d "${datenow} + 14 days - 1 hours + 30 minutes" +"%Y-%m-%dT%H:%M:%SZ")

This is the part of the JSON with the market proposal's info:

  "ID": "P0000006659-0000000002",
  "reference": "ETHVUSD/DEC20/DOM",
  "partyID": "996d6750e548a8ee544841a65ff6213bd8e6b3df8984cfed92bbb615f317c806",
  "state": "STATE_OPEN",
  "timestamp": "1599575007954039949",
  "terms": {
    "closingTimestamp": "1600180706",
    "enactmentTimestamp": "1600786406",
    "validationTimestamp": "0",
    "newMarket": {
      "changes": {
        "instrument": {
          "name": "December 2020 ETH vs VUSD future DOM",
          "code": "CRYPTO:ETHVUSD/DEC20",
          "baseName": "DOM",
          "quoteName": "VUSD",
          "future": {
            "maturity": "2020-12-31T23:59:59Z",
            "asset": "ece2dcd0869fc765589f9e162fa0d1761b598def1628c4ff0f399a743693675d"
          }
        },
        "decimalPlaces": "5",
        "metadata": [
          "asset_class:fx/crypto",
          "product:futures"
        ],
        "openingAuctionDuration": "0",
        "logNormal": {
          "riskAversionParameter": 0.001,
          "tau": 1,
          "params": {
            "mu": 0,
            "r": 1,
            "sigma": 1
          }
        },
        "continuous": {
          "tickSize": ""
        }
      }
    }
  },
  "reason": "PROPOSAL_ERROR_UNSPECIFIED"
}

if we check the dates:

dom@vegatest:~/test/develop/qa/REST_API$ date -d @1600180706
Tue 15 Sep 15:38:26 BST 2020
dom@vegatest:~/test/develop/qa/REST_API$ date -d @1600786406
Tue 22 Sep 15:53:26 BST 2020
dom@vegatest:~/test/develop/qa/REST_API$

@jeremyletang is this correct?

jeremyletang commented 4 years ago

@domvega What's the issue then? Sorry I'm not sure to see what's incorrect?

jeremyletang commented 4 years ago

@domvega I mean I may not see something totally obvious but you'll need to point me to it then?

edd commented 4 years ago

@jeremyletang The submitted proposal was dated as follows:

closingDatetime:"2020-09-15T14:38:26Z",
enactmentDatetime:"2020-09-22T14:53:26Z"

The results on the proposal when fetched via REST were

Tue 15 Sep 15:38:26 BST 2020
Tue 22 Sep 15:53:26 BST 2020

I have a feeling this is a time zone issue (BST vs GMT), I just haven’t wrapped my head around if it’s actually wrong. The submitted date has no time zone, and when parsed from the output, it does have a timezone.

jeremyletang commented 4 years ago

@edd yes after chatting with @domvega it's seems like a confusion because of the timezone. I'm closing this as it's a non-issue

nzbaen commented 4 years ago

@edd, @jeremyletang and I will have a call later where he's going to try and help me understand this better. I was left at "Vega uses Vega time to avoid timezone issues" so maybe here comes the confusion (and might've been my misunderstanding of Vega time too)

edd commented 4 years ago

@edd, @jeremyletang and I will have a call later where he's going to try and help me understand this better.

After this meeting, please find somewhere in the documentation (protos, markdown in the docs folder, or the docs repo) to clarify Vega Time to everyone.

I was left at "Vega uses Vega time to avoid timezone issues" so maybe here comes the confusion (and might've been my misunderstanding of Vega time too)

I know you have a call and will get more detail, but it's not 'To avoid timezone issues', it's because we need validators to have a share idea of time it is, regardless of their location or their clock being incorrect. Ours is determined in Tendermint: "Tendermint provides a deterministic, Byzantine fault-tolerant, source of time. Time in Tendermint is defined with the Time field of the block header."

nzbaen commented 4 years ago

@edd will it be fine to add the doc under https://github.com/vegaprotocol/docs/tree/master/content/docs? the idea is to have a dedicated Vega_Time.md document for this

nzbaen commented 4 years ago

Right, I've created this document vega-time.md that will need to be reviewed and cleared/updated/merged