search

SEED-platform / seed

Standard Energy Efficiency Data (SEED) Platform™ is a web-based application that helps organizations easily manage data on the energy performance of large groups of buildings.
Other
110 stars 55 forks source link

Error Report on Meter Data Upload #3179

Closed dreneden1 closed 1 year ago

dreneden1 commented 2 years ago

Is your feature request related to a problem? Please describe. We are having lots of problems when we try to upload meter data using the "Upload Portfolio Manager Meter Usage" button. We are specifically trying to upload data from the "data request" format from ENERGY STAR Portfolio Manager. See attached file as an example. When we upload the "monthly usage" tab, we often get the error that the data isn't in the required format "An error occurred while processing the file. Please ensure that your file meets the required specifications.", but we don't have any details on what the required specifications are or where/how we didn't meet them. Uploading Monthly Data.xlsx

Describe the solution you'd like ENERGY STAR Portfolio Manager has a similar feature where you can upload data via spreadsheet. When the file doesn't meet the required specs, you get an error report that specifies exactly why the file didn't upload.

image

Describe alternatives you've considered The meter data upload works better when we use the "Download my portfolio" option from ESPM and then upload this file using the "Upload Portfolio Manager Meter Usage" button; however, we don't have all of the data in this format since participants don't "share their property" with us, they simply respond to the data request.

Additional context

image

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity within 60 days. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity within 60 days. It will be closed if no further activity occurs. Thank you for your contributions.

RDmitchell commented 2 years ago

This would be a very good enhancement to add.

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity within 60 days. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity within 60 days. It will be closed if no further activity occurs. Thank you for your contributions.

isalanglois commented 1 year ago

@RDmitchell Can we follow-up to see if this is still an issue? @haneslinger addressed some issues with this feature, and may be working now

RDmitchell commented 1 year ago

@isalanglois -- I don't have a good ESPM data set that would allow me to test this. Maybe we should have Dan test on the staging server?

RDmitchell commented 1 year ago

@isalanglois -- I can test uploading the file he references, I just don't have a dataset in ESPM that would show what he is showing at the beginning of the issue description. So I will start with that.

But I can't do much testing until tomorrow afternoon.

RDmitchell commented 1 year ago

I can import the file referenced at the beginning of this issue, as long as I switch the tabs so that the Information and Metrics tab is first, so that the program can create the property records with the PM Property ID, and the Monthly Usage (meter data) 2nd. image image

haneslinger commented 1 year ago

sweet, I think we are good to close then?

RDmitchell commented 1 year ago

@nllong -- I guess we need to let @dreneden1 know when this is added to the release patch?

dreneden1 commented 1 year ago

@RDmitchell please do!

What was the solution in the end? Will there be a more detailed error message when the upload fails? I noticed in the screenshot above a "delete meters" feature - that's great! Is that for a different ticket?

RDmitchell commented 1 year ago

@dreneden1 -- we have implemented delete meters, and will move it to the staging server soon. I will ping you when it's there so you can test.

Not sure about the more detailed error message on failed upload. @nllong / @isalanglois -- we might need to add another issue for that feature.

dreneden1 commented 1 year ago

@RDmitchell I'm confused... This ticket was about adding an error report when files are uploaded and rejected. It seems like the solution was to implement the ability to delete meters (which is a great and much needed feature). How are those two related?

RDmitchell commented 1 year ago

@haneslinger -- @dreneden1 is correct, I don't think we addressed his original issue, which is a way to see what the problem is with his file when he is uploading it from Portfolio Manager and gets the message "An error occurred while processing your file".

In this particular case, with this particular file, the problem was that SEED couldn't interpret the data properly because the tab order was incorrect, and could be fixed by reversing the tab order. Which is what I did and then the file could be imported.

But the bigger issue is to give users more information about what is wrong with the file so that they know how to fix it.

@nllong -- We may have to think about this a bit because there are many different possible problems with files. Does SEED have enough information to give the user an accurate error message about what is wrong with the file?

haneslinger commented 1 year ago

Hey @dreneden1,

I've just tried uploading the linked file and I didn't have any issues. Which version of seed are you on? Is this still plaguing you? If so, could we set up a time to go threw it together so I can look at your logs?

dreneden1 commented 1 year ago

Hi @haneslinger, can you try with this one? One of our clients is trying to upload it and there is no problem with the annual data, but when we come to add the meter data, we are getting that error message.

BBO Participants - 2019 - Ottawa Data Upload Issue.xlsx

RDmitchell commented 1 year ago

@haneslinger -- on the LBNL production instance, which is SHA 09736c1d6, there are issues importing meter data. Not sure if these two things are related, but thought I would chime in.

Actually, importing ESPM meter data generally seems to work on production

See this issue: #3450

haneslinger commented 1 year ago

Thank you both for the extra information! I have been able to recreate and error with BBO Participants - 2019 - Ottawa Data Upload Issue.xlsx on develop and I am investigating.

dreneden1 commented 1 year ago

Hi @haneslinger - any luck?

The monthly data upload was working a couple of weeks ago, with very similar files. I wonder if ENERGY STAR Portfolio Manager changed something in their report formatting which is screwing with the monthly data upload.

This is quite urgent as I am hoping to wrap up our program by tomorrow, but I need to be able to add meter data. Are there any other options?

anchapin commented 1 year ago

@dreneden1 it's working now with the PR (#3716) that I just created.

dreneden1 commented 1 year ago

@haneslinger thanks for prompting the fix, and @anchapin thanks for the fix!

@haneslinger circling back to the original topic of this issue, will there now be an error report when a file doesn't upload?

isalanglois commented 1 year ago

#3734 A new ticket has been created to address the error messaging

RDmitchell commented 1 year ago

@dreneden1 -- We have added error messaging when importing meters.

In the future, when you get errors it would probably be good to let us know (you could make a new issue as you did here) and we can look into whether we can make the error messages more helpful to the user.

dreneden1 commented 1 year ago

Sounds good - thank you @RDmitchell !