Nasty error observed - Githubissues

jecollins commented 12 years ago

I'm not sure where this originates. It shows up in Visualizer. The only time I've observed it was the first time I tried to run a full sim from the 0.5.0 release package, but I have not seen it before or since. It messes up the simulation, so we don't want this surviving into the competition. Here's the relevant part of the logfile (I think):

TariffTx: customerVillageType4 Village 2 Controllable at 2009-10-25T23:00:00.000Z, PRODUCE: 0.07700000000000001@-0.007140599732870275
324419 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerVillageType4 Village 2 Controllable at 2009-10-25T23:00:00.000Z, PRODUCE: 0.07700000000000001@-0.007140599732870275
324419 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerVillageType4 Village 2 Controllable at 2009-10-25T23:00:00.000Z, PRODUCE: 0.07700000000000001@-0.007140599732870275
324421 INFO  broker.BrokerModel: 
Sample: my tariffTrans: +
TariffTx: customerOfficeComplexType1 OfficeComplex 1 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.10640000000000001@NaN
324421 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerOfficeComplexType1 OfficeComplex 1 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.10640000000000001@NaN
324422 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerOfficeComplexType1 OfficeComplex 1 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.10640000000000001@NaN
324424 INFO  broker.BrokerModel: 
Sample: my tariffTrans: +
TariffTx: customerOfficeComplexType1 OfficeComplex 2 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.215@-0.01993803821515726
324424 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerOfficeComplexType1 OfficeComplex 2 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.215@-0.01993803821515726
324424 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerOfficeComplexType1 OfficeComplex 2 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.215@-0.01993803821515726
324429 INFO  broker.BrokerModel: 
Sample: my tariffTrans: +
TariffTx: customerOfficeComplexType2 OfficeComplex 1 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.2061@-0.019112696168111214
324429 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerOfficeComplexType2 OfficeComplex 1 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.2061@-0.019112696168111214
324429 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerOfficeComplexType2 OfficeComplex 1 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.2061@-0.019112696168111214
324429 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerOfficeComplexType2 OfficeComplex 1 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.2061@-0.019112696168111214
324439 INFO  broker.BrokerModel: 
Sample: my tariffTrans: +
TariffTx: customerOfficeComplexType2 OfficeComplex 2 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.215@-0.01993803821515726
324439 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerOfficeComplexType2 OfficeComplex 2 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.215@-0.01993803821515726
324439 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerOfficeComplexType2 OfficeComplex 2 at 2009-10-25T23:00:00.000Z, PRODUCE: 0.215@-0.01993803821515726
324441 INFO  broker.BrokerModel: 
Sample: my tariffTrans: +
TariffTx: customerFrostyStorage at 2009-10-25T23:00:00.000Z, PRODUCE: 1099.383@-101.95135007950806
324441 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerFrostyStorage at 2009-10-25T23:00:00.000Z, PRODUCE: 1099.383@-101.95135007950806
324441 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerFrostyStorage at 2009-10-25T23:00:00.000Z, PRODUCE: 1099.383@-101.95135007950806
324444 INFO  broker.BrokerModel: 
Sample: my tariffTrans: +
TariffTx: customerVillageType1 Village 1 Controllable at 2009-10-25T23:00:00.000Z, PRODUCE: 0.0576@-0.005341539540432828
324444 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerVillageType1 Village 1 Controllable at 2009-10-25T23:00:00.000Z, PRODUCE: 0.0576@-0.005341539540432828
324444 INFO  broker.CustomerModel: 
 CustomerModel: my tariffTrans: +
TariffTx: customerVillageType1 Village 1 Controllable at 2009-10-25T23:00:00.000Z, PRODUCE: 0.0576@-0.005341539540432828
324452 ERROR handlers.BrokerMessageHandler: Cannot call handleMessage: java.lang.NumberFormatException: Infinite or NaN
  ..at java.math.BigDecimal.<init>(BigDecimal.java:797)
  ..at org.powertac.visualizer.Helper.roundNumberTwoDecimal(Helper.java:58)
  ..at org.powertac.visualizer.domain.broker.BrokerModel.updateCashBalance(BrokerModel.java:77)
  ..at org.powertac.visualizer.services.handlers.BrokerMessageHandler.handleMessage(BrokerMessageHandler.java:128)
  ..at

There are some tariff transactions with NAN in them; is this what's causing the problem, or is it the last transaction?

juricababic commented 12 years ago

According to stack trace at the bottom, it seems that Visualizer received NaN cash balance for a broker. That's why number formatting fails. I have used BigDecimal to trim unnecessary decimals from numbers. I can surely get rid of this formatting method because, in meanwhile, I have learned how to trim numbers in the presentation layer.

I still think there should not be any NaNs in the game.

Thoughts?

jecollins commented 12 years ago

This is happening when OfficeComplex customers with interruptible consumption call usePower with a negative power quantity (this can happen when they get curtailed by the balancing market), but with a customer count of 0. So the price gets divided by the customer count and we get NaN as a result. But the model should not be calling usePower on the tariff subscription if there are no subscribed customers. I guess we could block that in TariffSubscription, but the error is in the OfficeComplex customer model.

chrysopoulos18 commented 12 years ago

Hello there.

I am buffled with this one, because that bug was taken care a long time ago. If you remember, Markus had the same problem and I had taken care of that by adding this lines for both consumer models.

if (sub.getCustomerCommitted() > 0) sub.usePower(summary) if (sub2.getCustomerCommitted() > 0) sub2.usePower(summary)

where sub and sub2 are the subscriptions for the base and the controllable consumption.

How is possible that the models have called this function without any customers subscribed????

Is it possible that a curtailment happens when nobody is present in a subscription? How can you curtain so much power that you produce, as I can see from the log of the visualizer?

I am sorry for my somewhat stupid questions, but I am trying to find where this problem may be stemming from, and in my code I don't see any other place that I call usePower, except for these two lines.

jecollins commented 12 years ago

The most likely possibility that I can envison is that the subscription count was updated to 0 after the customer used power (in phase 2), but before the balancing process was run (in phase 3). If you look again at the time sequence for balancing actions, you will see that the balancing process is creating a tariff transaction to "correct" for the curtailment. If the subscription count has changed in the meantime, this is a problem.

I suppose we could simply check for this case in TariffSubscription, but in that case we would not observe the full effects of power shifting. The other alternative would be to update subscriptions before using power, but in that case you also might not see and act on a shift. What do you think?

chrysopoulos18 commented 12 years ago

This is a tricky situation indeed. Both alternatives have pros and cons, so I cannot decide over them.

In my implementations, I first check the revoked tarifs, then the weather, after that the curtailment and in the end I consume power. I think that is the natural flow of things. I don't think that I can do much more in there.

So, the problem comes to the Tariff Subscription. As you said, the curtailment is estimated from before, and this is difficult to change too. So it all comes up to the time scheduling of the subscription update. Both ways are a bit dirty... I don't know which might be the correct way to handle this.

I was also wandering what might happen, if someone was curtailing his power consumption for a tariff (meaning that he moved the consumption to the next timeslot) but then in the next was not a subscriber to this tariff, due to the newly published tariffs that were better that the one he was in.

jecollins commented 12 years ago

First, regarding curtailments and shifting. If a customer gets curtailed in timeslot n, and then switches its subscription, the shifting should still happen. It just won't show up on the original subscription.

Second, the compensating tariff transaction that is issued by the balancing market should be with regard to the subscription situation at the time the power being curtailed was used. One way to model that might be to store, in the tariff subscription, not only the current customer count, but the count as it was earlier in the current timeslot before subscription decisions might have changed the number. We would need to decide how to represent and maintain that number, and use it correctly when curtailing.

jecollins commented 12 years ago

Here's a potential solution: The problem is that subscriptions are changing between the time the customer uses power and the time the balancing market runs. The most dramatic lossage from that is the error we see, but I'm sure there are more subtle errors also. But time in the simulation is discrete, so we are free to re-order events so they make sense. I suggest that we defer the processing of subscription changes until after the balancing market has run. One obvious place to do that would be when Accounting processes subscription changes.

Currently, new subscriptions go through the TariffMarket, and could easily be deferred. Unsubscribes go through the subscriptions themselves, but could be delegated to the TariffMarket as well to be deferred.

The other issue here, if we really want to do this right, is that curtailments should happen to the customers who were subscribed at the time the curtailed power was used. If you look at the [sequence diagram}(../wiki/Controllable-capacity) at the bottom of the page, you see that customers retrieve curtailments from their current subscriptions in the timeslot following the usage. This may need to be fixed somehow in the individual customer models, because the effect will be different in Prashant's statistical models than in Antionios' behavioral models. That's because for Antonios there will be specific customers that are curtailed, while for Prashant it's just a potentially different number of anonymous customers.

chrysopoulos18 commented 12 years ago

I agree that the only solution is to reorder the flow of events and I also agree that some changes might be needed for the customer models.

As I understood the curtailment procedure, if I am not mistaken, the customers who are moving the load are paid for getting the load in the next timeslot. So even if they are not subscribed to this tariff in the next timeslot, they are paid for it, which is correct. So, maybe there's no need for change in the customer models.

Another thing that could be done is to divide the the power "trade-off" to two seperate procedures. One would be the shifting of the power loads to the next timeslot and another would be to shift loads to the current timeslot from the next one.

This may need two different events, or the balancing event as defined now may need a parameter to seperate the two procedures. This way the customers will have to move their loads from n timeslot to the n+1 timeslot, even if they will not be in the same subscription of that, and in n+1 timeslot the customers will have to either do nothing (since they have moved their load in n+1 timeslot before) or move load from the n+2 timeslot to the n+1 timeslot.

Does this make sense, or I am completely lost?

jecollins commented 12 years ago

I think we should avoid shifting forward in time. What we have now is a good approximation of what happens in the real world. Based on our conversations, here's a plan:

unsubscribe events should go through the tariff market, as subscribe events already do. This will not require a change in the customer models; the subscription can simply delegate the call to the tariffMarket.
the tariffMarket should not process subscribe and unsubscribe events immediately when customers act, but rather during Phase 4. The tariffMarket is currently activated during Phase 3 to put it after customer activation and before Accounting (see the timeline info). Since we still want subscription events to occur before Accounting runs, this may require some coordination with Accounting - for example, Accounting could trigger the subscription updates.
when a subscription is asked for the maximum possible curtailment, (see the balancing control sequence, it will not include curtailments from customers who have already unsubscribed. This is an argument for keeping the unsubscribe operation in the subscription.

So changes are needed in TariffSubscription, TariffMarketService, and a minor change in Accounting. It should not be necessary to make any changes in the customer models.

Does this make sense?

jecollins commented 12 years ago

One more thought. Rather than ask Accounting to meddle wtih the TariffMarket, it would be cleaner to just add another phase to the timeslot phase processor and put TariffMarket in phase 4, and Accounting in phase 5. This will require that the number of phases be configurable (in the Spring config, where the timeslot phases are set).

chrysopoulos18 commented 12 years ago

This makes perfect sense.

So the only thin we will have to change in the customer models will be to change the unsubscribing to pass through Tariff Market, I guess. If anything else is needed, let me know. I will be more than glad to help with it.

Cheers.

jecollins commented 12 years ago

No. The customer models do not need to change at all. We'll just have TariffSubscription delegate unsubscribe events to the TariffMarket. We might want to update the customer models later to interact directly with the TariffMarket, but for now we can minimize the number of changes (and the associated risk) by using the delegation method.

preddy commented 12 years ago

This seems consistent with what we discussed by email and I like the delegation through TariffSubscription idea for minimizing the short term impact. Thanks.

jecollins commented 12 years ago

One minor modification to the plan above. Rather than unnecessarily couple Accounting to TariffMarket, let's just add another timeslot phase and put TariffMarket between DistributionUtility and Accounting.

jecollins commented 12 years ago

I have implemented and pushed a fix for this that follows the outline above, with one small addition. That is to push through subscriptions, rather than deferring them in the case where there are no subscriptions. This only happens in the first timeslot, when the customer models first subscribe to the default tariff, and then need to use power before the TariffMarket is activated for the first time.

There are test errors in household-customer and officecomplex-customer, probably because TariffMarket.subscribeToTariff() no longer returns a subscription - it is now a void method.

jecollins commented 12 years ago

The NaN problem has not gone away, but of course it waited to show up again until I had put together all of the 0.5.1 release. At least is has not been released...

jecollins commented 12 years ago

The problem is that the Visualizer has its own powertac.xml. It works with the one under server-main, but not the one under Visualizer.

jecollins commented 12 years ago

The separate powertac.xml is indeed a problem, but it's apparently not the only problem. The symptom now is that the bogus PRODUCE transaction (caused by a balancing event, presumably) happens in timeslot 406, but the subscription count for that customer went to 0 in timeslot 404.

jecollins commented 12 years ago

I think this one is done, finally. We now check for non-zero subscription count before returning available curtailment from a subscription.

powertac / powertac-server

Nasty error observed #531