PANIC exit while processing transaction

[mess-maker]

OS LINUX Opensuse Tumbleweed VM on win 11 host

here are the log with ./portfoliodownloader --debug --trace -w with some errors. the program exit after having processed only 50 of the 2k+ transaction

some errors are from deposit that are skipped it's maybe a normal behaviour but it's not clear in the readme if the deposits should be listed in the csv should be great to be able to reconstruct the value over time of the porfolio in external tools to have the deposit and withdraw listed

to lighten the file I've ommited the "OK" transaction and the recurring deposit errors (same as transaction 3 hereunder) and i added the json part corresponding from the ''timeline transactions'' folder

May 18 17:57:22.893 [DEBU] [timezoneName:Europe/Paris] Runtime timezone set
May 18 17:57:23.063 [DEBU] [model:*transaction.Model] initialized repository for model
May 18 17:57:23.189 [DEBU] [model:*transaction.Model] initialized repository for model
May 18 17:57:23.189 [INFO] Downloading transactions
May 18 17:57:27.899 [INFO] 2499 transaction downloaded
May 18 17:57:27.899 [INFO] [id:transaction 1] Fetching transaction details
May 18 17:57:27.932 [INFO] [id:transaction 1] Processing transaction details
May 18 17:57:27.933 [DEBU] [id:transaction 1] resolver error: overview contains no order type nor event: unsupported response: section data entry not found (Ereignis)
May 18 17:57:27.933 [INFO] [id:transaction 1] Unsupported transaction skipped
    {
      "action": {
        "payload": "transaction 1",
        "type": "timelineDetail"
      },
      "amount": {
        "currency": "EUR",
        "fractionDigits": 2,
        "value": 200
      },
      "badge": null,
      "eventType": "PAYMENT_INBOUND_CREDIT_CARD",
      "icon": "logos/bank_visa/v2",
      "id": "transaction 1",
      "status": "EXECUTED",
      "subAmount": null,
      "subtitle": null,
      "timestamp": "2022-01-08T18:07:55.513+0000",
      "title": "Einzahlung"
    }
//this one correspond to the initial deposit to open the account at that time

May 18 17:57:27.933 [INFO] [id:transaction 2] Fetching transaction details
May 18 17:57:27.983 [INFO] [id:transaction 2] Processing transaction details
May 18 17:57:27.983 [INFO] [id:transaction 2] Transaction processed
May 18 17:57:27.983 [INFO] [id:transaction 3] Fetching transaction details
May 18 17:57:28.015 [INFO] [id:transaction 3] Processing transaction details
May 18 17:57:28.015 [DEBU] [id:transaction 3] resolver error: overview contains no order type nor event: unsupported response: section data entry not found (Ereignis)
May 18 17:57:28.015 [INFO] [id:transaction 3] Unsupported transaction skipped
{
      "action": {
        "payload": "transaction 3",
        "type": "timelineDetail"
      },
      "amount": {
        "currency": "EUR",
        "fractionDigits": 2,
        "value": 100
      },
      "badge": null,
      "eventType": "PAYMENT_INBOUND",
      "icon": "logos/timeline_plus_circle/v2",
      "id": "transaction 3",
      "status": "EXECUTED",
      "subAmount": null,
      "subtitle": null,
      "timestamp": "2022-01-17T09:26:51.758+0000",
      "title": "name and surname" (replace my real name here)
    },
// this one is a regular deposit by wire transfert

May 18 17:57:29.256 [INFO] [id:transaction 31] Fetching transaction details
May 18 17:57:29.291 [INFO] [id:transaction 31] Processing transaction details
May 18 17:57:29.291 [DEBU] [id:transaction 31] resolver error: unsupported response: %!w(<nil>)
May 18 17:57:29.291 [INFO] [id:transaction 31] Unsupported transaction skipped
 {
      "action": {
        "payload": "transaction 31",
        "type": "timelineDetail"
      },
      "amount": {
        "currency": "EUR",
        "fractionDigits": 2,
        "value": 0.26
      },
      "badge": null,
      "eventType": "CREDIT",
      "icon": "logos/FR0000035081/v2",
      "id": "transaction 31",
      "status": "EXECUTED",
      "subAmount": null,
      "subtitle": "Dividende",
      "timestamp": "2022-03-02T00:12:17.695+0000",
      "title": "Icade"
    },

May 18 17:57:30.138 [INFO] [id:transaction 50] Fetching transaction details
May 18 17:57:30.174 [INFO] [id:transaction 50] Processing transaction details 50
May 18 17:57:30.174 [DEBU] [id:transaction 50] purchase transaction resolved
May 18 17:57:30.174 [PANI] could process transaction: deserializer error: error getting overview asset: section data entry not found (Asset)
panic: (*logrus.Entry) 0xc000152d20
{
      "action": {
        "payload": "transaction 50",
        "type": "timelineDetail"
      },
      "amount": {
        "currency": "EUR",
        "fractionDigits": 2,
        "value": -395.8
      },
      "badge": null,
      "eventType": "ORDER_EXECUTED",
      "icon": "logos/FR0003500008/v2",
      "id": "transaction 50",
      "status": "EXECUTED",
      "subAmount": null,
      "subtitle": "Kauforder",
      "timestamp": "2022-03-29T09:43:31.570+0000",
      "title": "Short Faktor Optionsschein 2"
    },
goroutine 1 [running]:
github.com/sirupsen/logrus.(*Entry).log(0xc0001356c0, 0x0, {0xc00027f080, 0x71})
        /go/pkg/mod/github.com/sirupsen/logrus@v1.9.3/entry.go:260 +0x491
github.com/sirupsen/logrus.(*Entry).Log(0xc0001356c0, 0x0, {0xc000327e80?, 0xc0006f1098?, 0xc00018b080?})
        /go/pkg/mod/github.com/sirupsen/logrus@v1.9.3/entry.go:304 +0x48
github.com/sirupsen/logrus.(*Logger).Log(0xc00018b080, 0x0, {0xc000327e80, 0x1, 0x1})
        /go/pkg/mod/github.com/sirupsen/logrus@v1.9.3/logger.go:204 +0x58
github.com/sirupsen/logrus.(*Logger).Panic(...)
        /go/pkg/mod/github.com/sirupsen/logrus@v1.9.3/logger.go:253
main.main()
        /github/workspace/cmd/portfoliodownloader/public/main.go:44 +0x425

and here the transaction 50.json from ''timelineDetailV2'' folder

{
  "id": "transaction 50",
  "sections": [
    {
      "action": {
        "payload": "DE000SH0MW59",
        "type": "instrumentDetail"
      },
      "data": {
        "icon": "logos/FR0003500008/v2",
        "status": "executed",
        "subtitleText": null,
        "timestamp": "2022-03-29T09:43:31.570+0000"
      },
      "title": "Du hast 395,80 €  investiert",
      "type": "header"
    },
    {
      "action": null,
      "data": [
        {
          "detail": {
            "functionalStyle": "EXECUTED",
            "text": "Ausgeführt",
            "type": "status"
          },
          "style": "plain",
          "title": "Status"
        },
        {
          "detail": {
            "action": null,
            "text": "Kauf",
            "trend": null,
            "type": "text"
          },
          "style": "plain",
          "title": "Orderart"
        },
        {
          "detail": {
            "action": null,
            "text": "CAC",
            "trend": null,
            "type": "text"
          },
          "style": "plain",
          "title": "Basiswert"
        },
        {
          "detail": {
            "action": null,
            "text": "Short Faktor Optionsschein 2",
            "trend": null,
            "type": "text"
          },
          "style": "plain",
          "title": "Produkt"
        }
      ],
      "title": "Übersicht",
      "type": "table"
    },
    {
      "action": null,
      "data": [
        {
          "detail": {
            "action": null,
            "text": "40",
            "trend": null,
            "type": "text"
          },
          "style": "plain",
          "title": "Anteile"
        },
        {
          "detail": {
            "action": null,
            "text": "9,87 €",
            "trend": null,
            "type": "text"
          },
          "style": "plain",
          "title": "Aktienkurs"
        },
        {
          "detail": {
            "action": null,
            "text": "1,00 €",
            "trend": null,
            "type": "text"
          },
          "style": "plain",
          "title": "Gebühr"
        },
        {
          "detail": {
            "action": null,
            "text": "395,80 €",
            "trend": null,
            "type": "text"
          },
          "style": "highlighted",
          "title": "Gesamt"
        }
      ],
      "title": "Transaktion",
      "type": "table"
    },
    {
      "action": null,
      "data": [
        {
          "action": {
            "payload": "https://traderepublic-data-production.s3.eu-central-1.amazonaws.com/",
            "type": "browserModal"
          },
          "detail": "29.03.2022",
          "id": "xxxxxxxxxxxxxxxxxxx",
          "postboxType": "SECURITIES_SETTLEMENT",
          "title": "Abrechnung"
        },
        {
          "action": {
            "payload": "https://traderepublic-data-production.s3.eu-central-1.amazonaws.com/",
            "type": "browserModal"
          },
          "detail": "29.03.2022",
          "id": "yyyyyyyyyyyyyyyyyyyyy",
          "postboxType": "COSTS_INFO_BUY_V2",
          "title": "Kosteninformation"
        }
      ],
      "title": "Dokumente",
      "type": "documents"
    }
  ]
}

[dhojayev]

Thanks for the report again. Unfortunately, the response from TR is so unstructured, since it was designed for webapp and mobile app rendering, so it does not have fixed structure. Here it fails because it expects a portion of JSON to exist in the response while it's not. I will divide it into 2 split fixes:

1. The one that won't lead to panic with these transaction types.
2. The one that will add a support for above-mentioned transaction types.

Thanks again!

[dhojayev]

Could you please help me with the transaction 50? I would like to understand what "Basiswert: CAC" means in this case. Is this something that should end up in the csv file or it is not important? I am asking because I never owned such instruments. Thanks in advance!

[mess-maker]

hi thanks for thoose fix and future one

for the transaction 50 the product is a turbo (can be long or short) it's a leveraged product on and underlying stock or indice (here the underlying is the CAC 40 hence the Basiswert: CAC)

here some explanation on how they work https://www.ig.com/ie/exchange-traded-products/turbo24-trading/what-are-turbo-certificates-how-do-you-trade-them

you can find them in the derivatives section in trade republic (by searching "cac 40" or "S&P 500" for example) I think all derivatives should have this JSON section. By definition they are a derivative of an underlying

Here an ISIN of a comparable product : DE000SW7J4D0 you can search for it in trade republic to have the detailled informations

[dhojayev]

Thank you for the explanation. I implemented support for these transactions. Please note, that some data is still missing in the csv file as I am planning to add more details later once I better understand these transaction. I would really appreciate if you could run the binary again and let me know if you get more errors as you still may have transactions that the app is not familiar with yet. And one more thing: Transactions that you mentioned earlier: we are planning to add full support for those transactions soon, it just feels like it makes no sense to include those until we implement better calculation of what happens with the money deposited.

Long story short: I hope this fix would finally generate you a full csv with all purchases and sales, deposits will come soon and I can inform you once it is done.

[dhojayev]

@mess-maker FYI I added including deposit transactions in csv. The latest version v0.6.0 now.