Error("invalid type: null, expected struct Measurements"

[sytem]

Hello. Tried this without no succes, and while Rust is new language for me, with quite limited skills to debug. Installed with instructions (Debian 9.3), made config file with login details and running it gets me error message:

sytem@summerboy:~/.config$ energiatili-import 
thread 'main' panicked at 'serde_json::from_str: Error("invalid type: null, expected struct Measurements", line: 1, column: 110108)', src/libcore/result.rs:1165:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

sytem@summerboy:~/.config$ RUST_BACKTRACE=1 energiatili-import 
thread 'main' panicked at 'serde_json::from_str: Error("invalid type: null, expected struct Measurements", line: 1, column: 110108)', src/libcore/result.rs:1165:5
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.37/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.37/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:76
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:60
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1030
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1412
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:64
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:196
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:210
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:473
  11: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:380
  12: rust_begin_unwind
             at src/libstd/panicking.rs:307
  13: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
  14: core::result::unwrap_failed
             at src/libcore/result.rs:1165
  15: energiatili_model::model::Model::from_report_html
  16: energiatili_import::main
  17: std::rt::lang_start::{{closure}}
  18: main
  19: __libc_start_main
  20: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

While using wrong userame or password in config, error is different, so I think those are not the problem

 sytem@summerboy:~/.config$ energiatili-import 
 thread 'main' panicked at 'Model not found', energiatili-model/src/model.rs:19:9

One thing that came to mind, I can see 3 different "käyttöpaikka" on website (current address and two old, might that be something I need to select.

[oherrala]

Can you try one or two things for me?

First: Log in to Energiatili, and go to "Kulutus". What's the URL you are getting?

I'm getting URL https://www.energiatili.fi/Reporting/CustomerConsumption/UserConsumptionReport?meteringPointCode=undefined&mpSourceCompanyCode=undefined&loadLastMonthData=true&showOldContracts=false.

Second: If you feed adventurous, can you try fiddling with path defined in

https://github.com/Tarlab/energiatili/blob/8c720d7941507e4ac67a5c502327df4ca8d93244/energiatili-import/src/main.rs#L10

to maybe figure out what's missing? Probably culprits are meteringPointCode and/or mpSourceCompanyCode URL parameters. If you could hardcode one or both of them in the source, then recompile and try again?

You can also turn on more logging with RUST_LOG environment variable. For example RUST_LOG=trace turns every logging nob on. More details can be found from https://docs.rs/env_logger/0.7.1/env_logger/

[sytem]

My url is at first: https://www.energiatili.fi/Reporting/CustomerConsumption/UserConsumptionReport?meteringPointCode=NNNNNN&mpSourceCompanyCode=OE&loadLastMonthData=true&showOldContracts=false where NNNNNN is six digit code (I dont know how secret that should be kept, I think that at least that can be used to end contract or something other nasty things if it can be connected to my name and address, so better be sure)

Allready tested that at least on same browser and session after that url, also url https://www.energiatili.fi/Reporting/CustomerConsumption/UserConsumptionReport found on source was giving me same output

Tried to modify url in code to: const REPORT_URL: &str = "/Reporting/CustomerConsumption/UserConsumptionReport?meteringPointCode=<NNNNNN>&mpSourceCompanyCode=OE";

Still same error.

With that logging I can see that all text from site is in finnish, could that be a problem?

It looks like the actual data is there:

var model = {"IsValid":true,"Message":null,"HasTemperatureSeries":false,"HasReactivePowerSeries":false,"PowerUnit":"kWh","DataInterval":{"Duration":"72.01:00:00","Start":new Date(1571605200000),"Stop":new Date(1577829600000),"StartValue":new Date(1571605200000),"StopValue":new Date(1577829600000),"TotalYears":0,"TotalDays":72.041666666666657,"TotalHours":72.041666666666657},"Hours":{"PMax":{"Item1":new Date(1574442000000),"Item2":5.56},"QMax":null,"Step":{"TimeZoneInfo":{"Id":"FLE Standard Time","DisplayName":"(UTC+02:00) Helsinki, Kyiv, Riga, Sofia, Tallinn, Vilnius","StandardName":"FLE Standard Time","DaylightName":"FLE Daylight Time","BaseUtcOffset":"02:00:00","AdjustmentRules":[{"DateStart":new Date(-62135596800000),"DateEnd":new Date(253402207200000),"DaylightDelta":"01:00:00","DaylightTransitionStart":{"TimeOfDay":new Date(-62135593200000),"Month":3,"Week":5,"Day":1,"DayOfWeek":0,"IsFixedDateRule":false},"DaylightTransitionEnd":{"TimeOfDay":new Date(-62135589600000),"Month":10,"Week":5,"Day":1,"DayOfWeek":0,"IsFixedDateRule":false},"BaseUtcOffsetDelta":"00:00:00"}],"SupportsDaylightSavingTime":true},"Type":4,"StepLength":"01:00:00","Start":new Date(1571605200000),"Stop":new Date(1577829600000),"StepCount":1730},"Consumptions":[{"TariffTimeZoneId":1,"TariffTimeZoneName":"Yksiaikainen","TariffTimeZoneDescription":"Yksiaikainen","Series":{"ReadingCounter":0,"Name":"Yksiaikainen","Resolution":"Hour","Data":[[1571616000000,0.09],[1571619600000,0.07],[1571623200000,0.15],[1571626800000,0.01],[1571630400000,0.16],[1571634000000,0.0],[1571637600000,0.15],[1571641200000,0.0],[1571644800000,0.16],[1571648400000,0.03],[1571652000000,0.26],[1571655600000,0.1],[1571659200000,0.26],[1571662800000,0.15],[1571666400000,0.19],[1571670000000,0.0],[1571673600000,0.0],[1571677200000,0.55],[1571680800000,0.28],[1571684400000,0.14],[1571688000000,0.26],[1571691600000,0.18] and pages more of this kind of output

[oherrala]

Ah, I looked the error more closely. There are maybe three places in the code where it could fail:

https://github.com/Tarlab/energiatili/blob/8c720d7941507e4ac67a5c502327df4ca8d93244/energiatili-model/src/model.rs#L42

https://github.com/Tarlab/energiatili/blob/8c720d7941507e4ac67a5c502327df4ca8d93244/energiatili-model/src/model.rs#L43

https://github.com/Tarlab/energiatili/blob/8c720d7941507e4ac67a5c502327df4ca8d93244/energiatili-model/src/model.rs#L49

Please, can you try one line at the time to replace the Measurements with Option<Measurements>?

My guess is that one of the items I'm expecting is not there. And since it's JSON there's null. It would help to know which one.

[sytem]

with first line changed:

  Compiling energiatili-model v0.1.0 (/home/sytem/energiatili/energiatili-model)
error[E0609]: no field `data` on type `std::option::Option<model::Measurements>`
  --> energiatili-model/src/measurement.rs:92:84
   |
92 |     let status_map: BTreeMap<i64, u8> = values_into_map(&root.consumption_statuses.data);
   |                                                                                    ^^^^

error: aborting due to previous error

For more information about this error, try `rustc --explain E0609`.
error: could not compile `energiatili-model`.

second one:

Compiling energiatili-model v0.1.0 (/home/sytem/energiatili/energiatili-model) error[E0609]: no fielddataon typestd::option::Option` --> energiatili-model/src/measurement.rs:93:81 | 93 | let temperature_map: BTreeMap<i64, f64> = values_into_map(&root.temperature.data); | ^^^^

error: aborting due to previous error

For more information about this error, try rustc --explain E0609. error: could not compile energiatili-model.

To learn more, run the command again with --verbose.`

and third one: Compiling energiatili-model v0.1.0 (/home/sytem/energiatili/energiatili-model) error[E0609]: no fielddataon typestd::option::Option` --> energiatili-model/src/measurement.rs:99:92 | 99 | let consumption_map: BTreeMap<i64, f64> = values_into_map(&consumptions.series.data); | ^^^^

error: aborting due to previous error

For more information about this error, try rustc --explain E0609. error: could not compile energiatili-model.

To learn more, run the command again with --verbose.`