Sync Command Failing to Add New Holding

[drkui]

Hi,

Before trying this out I added a new "Cryptocurrency BETA" investment account nicknamed BinanceUS. I downloaded crypto-sync-windows-amd64.exe in the 0,3.0 release, placed it in a folder (renamed the exe to crypto-sync), along with a config.toml file (edited with API/creds), ran the command in CMD and this is what I got:

C:\Users\[snip]\Documents\go-crypto-sync>crypto-sync.exe sync --destination pc all
2021-05-30T20:05:32.844-0700    INFO    crypto-sync.sync        crypto-sync/sync.go:49  Fetching holdings from binance-us
2021-05-30T20:05:33.535-0700    INFO    crypto-sync.sync        crypto-sync/sync.go:67  setting pricing data provider to nomics
2021/05/30 20:05:33 open cookies.json: The system cannot find the file specified.
2021/05/30 20:05:33 unexpected end of JSON input
Please enter 2FA code: [snip]
https://home.personalcapital.com/api/credential/authenticateEmailByCode
2021/05/30 20:18:10 HTTP/2.0 200 OK
Cache-Control: no-cache, no-store, must-revalidate, maxage=0
Cf-Cache-Status: DYNAMIC
Cf-Ray: [snip]
Cf-Request-Id: [snip]
Content-Language: en-US
Content-Security-Policy: frame-ancestors 'self'
Content-Type: application/json;charset=UTF-8
Date: Mon, 31 May 2021 03:18:09 GMT
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Google-Bot: noarchive
P3p: CP="Personal Capital does not have a P3P policy"
Pragma: no-cache
Server: cloudflare
X-Envoy-Upstream-Service-Time: 43

{"spHeader":{"SP_HEADER_VERSION":-1,"userStage":"D","isDelegate":false,"userGuid":"[snip]","betaTester":false,"accountsMetaData":["HAS_ON_US","HAS_CASH","HAS_INVESTMENT","HAS_CREDIT"],"success":true,"accountsSummary":{"hasCredit":true,"hasAggregated":true,"hasCash":true,"hasInvestment":true,"hasOnUs":false},"qualifiedLead":true,"developer":false,"personId":4096240,"authLevel":"DEVICE_AUTHORIZED","username":"edward+personalcap@systemli.org","status":"ACTIVE"}}
2021/05/30 20:18:10 HTTP/2.0 200 OK
Cache-Control: no-cache, no-store, must-revalidate, maxage=0
Cf-Cache-Status: DYNAMIC
Cf-Ray: [snip]
Cf-Request-Id: [snip]
Content-Language: en-US
Content-Security-Policy: frame-ancestors 'self'
Content-Type: application/json;charset=UTF-8
Date: Mon, 31 May 2021 03:18:10 GMT
Expect-Ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Google-Bot: noarchive
P3p: CP="Personal Capital does not have a P3P policy"
Pragma: no-cache
Server: cloudflare
Set-Cookie: PMData=[snip]; Max-Age=31536000; Expires=Tue, 31-May-2022 03:18:10 GMT; Path=/; Secure; HttpOnly; SameSite=Strict
Set-Cookie: REMEMBER_ME_COOKIE=[snip]; Max-Age=604799; Expires=Mon, 07-Jun-2021 03:18:09 GMT; Path=/; Secure; HttpOnly; SameSite=Strict
X-Envoy-Upstream-Service-Time: 86

{"spHeader":{"SP_HEADER_VERSION":-1,"userStage":"D","isDelegate":false,"userGuid":"[snip]","deviceName":"  #4","betaTester":false,"accountsMetaData":["HAS_ON_US","HAS_CASH","HAS_INVESTMENT","HAS_CREDIT"],"success":true,"qualifiedLead":true,"developer":false,"personId":[snip],"authLevel":"SESSION_AUTHENTICATED","username":"[snip]","status":"ACTIVE"},"spData":{"credentials":["DEVICE","OOB_PHONE","OOB_SMS","OOB_EMAIL","SITE_KNOWLEDGE","PASSWORD","PIN"],"allCredentials":[{"name":"DEVICE","status":"ACTIVE"},{"name":"OOB_PHONE","status":"ACTIVE"},{"name":"OOB_SMS","status":"ACTIVE"},{"name":"OOB_EMAIL","status":"ACTIVE"},{"name":"SITE_KNOWLEDGE","status":"ACTIVE"},{"name":"PASSWORD","status":"ACTIVE"},{"name":"PIN","status":"ACTIVE"}]}}
2021-05-30T20:18:10.823-0700    INFO    crypto-sync.personal-capital    pc/client.go:127        BNB not found in pc, create new holding
2021-05-30T20:18:11.765-0700    INFO    crypto-sync.personal-capital    pc/client.go:170        Holding=, Quantity=[snip], TotalValue=[snip]
2021-05-30T20:18:12.558-0700    FATAL   crypto-sync.personal-capital    pc/client.go:205        Invalid Input.
github.com/will7200/go-crypto-sync/internal/pc.Sync
        D:/a/go-crypto-sync/go-crypto-sync/internal/pc/client.go:205
main.(*SyncCmd).Run
        D:/a/go-crypto-sync/go-crypto-sync/cmd/crypto-sync/sync.go:80
reflect.Value.call
        C:/hostedtoolcache/windows/go/1.16.4/x64/src/reflect/value.go:476
reflect.Value.Call
        C:/hostedtoolcache/windows/go/1.16.4/x64/src/reflect/value.go:337
github.com/alecthomas/kong.callMethod
        C:/Users/runneradmin/go/pkg/mod/github.com/alecthomas/kong@v0.2.16/callbacks.go:71
github.com/alecthomas/kong.(*Context).RunNode
        C:/Users/runneradmin/go/pkg/mod/github.com/alecthomas/kong@v0.2.16/context.go:706
github.com/alecthomas/kong.(*Context).Run
        C:/Users/runneradmin/go/pkg/mod/github.com/alecthomas/kong@v0.2.16/context.go:723
main.main
        D:/a/go-crypto-sync/go-crypto-sync/cmd/crypto-sync/main.go:111
runtime.main
        C:/hostedtoolcache/windows/go/1.16.4/x64/src/runtime/proc.go:225

Also just in case, here's a redacted version of my config.toml file:

debug = false
destination = "personalcapital"
priceDataSource = "nomics"
destinationCurrencyAs = "USD"
onHoldingNotFound = "zeroQuantity"
[pricing]
    [pricing.nomics]
    apiKey = "[snip]"
    debug = false

[symbols]
    [symbols.global]
        [symbols.override]
            [symbols.override.nomics]
            IOTA = "IOT"

[holdings]
    [holdings.binance-us]
    debug = false
    apiKey = "[snip]"
    secret = "[snip]"

[destinations]
    [destinations.personalcapital]
    email = "[snip]"
    password = "[snip]"
    accountName = "BinanceUS"

My holdings in Personal Capital did not update. Anything that I might be doing wrong?

[will7200]

Hi @drkui ! Thank you for trying this out. It really helps when someone other than me gets to use this.

From what i am seeing i believe you did everything right. There was an issue with calling the Personal Capital (PC) API when creating the new holding. I have released a new version v0.3.1 that improves the error messages when calling the pc api. If you could give that a try out and tell me the error if you are still getting it.

[will7200]

I just realized that you created a "CryptoCurrency Beta" account which i wasn't aware of. You need to create a "manual investment holdings" for this to work. I'll create a new issue for crypto-sync to work with that type of account.

[drkui]

Absolutely, I think its a very useful project to be working on and will potentially help a lot of people managing their crypto on PC.

After creating a new "manual investment holdings" account named "Crypto Sync", changing the accountName to match that in the config.toml file, and trying out v0.3.1 I'm getting the following error:

C:\Users\[snip]\Documents\go-crypto-sync-master>crypto-sync.exe sync --destination pc all
2021-05-31T16:37:53.381-0700    INFO    crypto-sync.sync        crypto-sync/sync.go:71  Fetching holdings from binance-us
2021-05-31T16:37:53.906-0700    INFO    crypto-sync.sync        crypto-sync/sync.go:89  setting pricing data provider to nomics
2021-05-31T16:37:53.907-0700    ERROR   crypto-sync.personal-capital    pc/client.go:78 open cookies.json: The system cannot find the file specified.
github.com/will7200/go-crypto-sync/internal/pc.Sync
        D:/a/go-crypto-sync/go-crypto-sync/internal/pc/client.go:78
main.(*SyncCmd).Run
        D:/a/go-crypto-sync/go-crypto-sync/cmd/crypto-sync/sync.go:113
reflect.Value.call
        C:/hostedtoolcache/windows/go/1.16.4/x64/src/reflect/value.go:476
reflect.Value.Call
        C:/hostedtoolcache/windows/go/1.16.4/x64/src/reflect/value.go:337
github.com/alecthomas/kong.callMethod
        C:/Users/runneradmin/go/pkg/mod/github.com/alecthomas/kong@v0.2.16/callbacks.go:71
github.com/alecthomas/kong.(*Context).RunNode
        C:/Users/runneradmin/go/pkg/mod/github.com/alecthomas/kong@v0.2.16/context.go:706
github.com/alecthomas/kong.(*Context).Run
        C:/Users/runneradmin/go/pkg/mod/github.com/alecthomas/kong@v0.2.16/context.go:723
main.main
        D:/a/go-crypto-sync/go-crypto-sync/cmd/crypto-sync/main.go:114
runtime.main
        C:/hostedtoolcache/windows/go/1.16.4/x64/src/runtime/proc.go:225
Select Challenge Type [email, sms] (defaults to email): sms
Please enter 2FA code: [snip]
2021-05-31T16:38:25.243-0700    INFO    crypto-sync.personal-capital    pc/client.go:220        BTC not found in pc, create new holding
2021-05-31T16:38:26.831-0700    INFO    crypto-sync.personal-capital    pc/client.go:270        Holding=, Quantity=[snip], TotalValue=[snip]
2021-05-31T16:38:27.430-0700    FATAL   crypto-sync.personal-capital    pc/client.go:305        Errors:
        0: {"code":100,"details":{"fieldName":""},"message":"Invalid Input."}

github.com/will7200/go-crypto-sync/internal/pc.Sync
        D:/a/go-crypto-sync/go-crypto-sync/internal/pc/client.go:305
main.(*SyncCmd).Run
        D:/a/go-crypto-sync/go-crypto-sync/cmd/crypto-sync/sync.go:113
reflect.Value.call
        C:/hostedtoolcache/windows/go/1.16.4/x64/src/reflect/value.go:476
reflect.Value.Call
        C:/hostedtoolcache/windows/go/1.16.4/x64/src/reflect/value.go:337
github.com/alecthomas/kong.callMethod
        C:/Users/runneradmin/go/pkg/mod/github.com/alecthomas/kong@v0.2.16/callbacks.go:71
github.com/alecthomas/kong.(*Context).RunNode
        C:/Users/runneradmin/go/pkg/mod/github.com/alecthomas/kong@v0.2.16/context.go:706
github.com/alecthomas/kong.(*Context).Run
        C:/Users/runneradmin/go/pkg/mod/github.com/alecthomas/kong@v0.2.16/context.go:723
main.main
        D:/a/go-crypto-sync/go-crypto-sync/cmd/crypto-sync/main.go:114
runtime.main
        C:/hostedtoolcache/windows/go/1.16.4/x64/src/runtime/proc.go:225

[will7200]

Looks like persona capital did not respond with any helpful error message. Usually would tell me a field is causing issues. Let me run through a test account and see if I can replicate the issue. If you can also try setting the log level to -2 (so crypto-sync --log-level="-2" sync ...). This will dump out the requests and responses made to pc and set logging to debug. Please remove headers before showcasing as the cookie header is sensitive

[drkui]

Sure, here is the log file. I also snipped out the lines that started with {"spHeader": as there was so much info in those lines to sift through. log.txt

[will7200]

Thank you @drkui I narrowed it down to it because the accountId was zero. I was also able to replicate the issue in the test account. The root cause was because the account parameter was never used and the account name was hard coded to be (silly me): account.Name == "CryptoSync managed automatically" && account.AccountTypeNew == "CRYPTO_CURRENCY"

v0.3.2 fixes this by only using the accountName and raises an error if that account is not found. Thank you for your patience through this issue! Give that new version a shot and let me know if it works for you. Fixed in: 7557eb1417e067564ebecbd3f3531f81635a3b39

[drkui]

It works! No problem, glad we got that sorted out! 🎉