search

SEED-platform / seed

Standard Energy Efficiency Data (SEED) Platform™ is a web-based application that helps organizations easily manage data on the energy performance of large groups of buildings.
Other
107 stars 55 forks source link

UI for Data Import of PM files #116

Closed RDmitchell closed 9 years ago

RDmitchell commented 9 years ago

Need to add some UI to make user select the file type at data import

image

meghasandesh commented 9 years ago

Robin,

I am including a link to Balsamiq - the web based tool I am using for the wireframe designs. You can follow the progress here: https://lbl.mybalsamiq.com/projects/seedproject/grid I'll include screenshots of finalized mockups directly in the issue

meghasandesh commented 9 years ago

custom select

Robin,

I have attached a mock of the dropdown menu here. Let me know what you think.

Megha

dangunter commented 9 years ago

But, in the short-term, how about another option in the upload dialog for PM data

https://lbl.mybalsamiq.com/projects/seedproject/Upload%20PM%20data

Dan Gunter

On Sun, Mar 1, 2015 at 4:15 PM, meghasandesh notifications@github.com wrote:

[image: custom select] https://cloud.githubusercontent.com/assets/1052210/6433838/1647f94c-c02e-11e4-9b57-ead31d189634.png

Robin,

I have attached a mock of the dropdown menu here. Let me know what you think.

Megha

Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-76640346.

meghasandesh commented 9 years ago

Dan,

Do you see any issues with the dropdown solution ?

meghasandesh commented 9 years ago

image

Minor edits made to the interface after consulting with Robin

dangunter commented 9 years ago

The additional information I am expecting, to do PM mapping, is program name and program version (I am adding 2 fields in the DB for this). This is separate from the data format (csv, xls, etc.). There is also a field "source_type" but that seems to be munging format and program source, so I am going to ignore that and expect new info. Probably the fastest thing would be for me to make whatever UI changes I need to test out the new interface and then leave it to you to clean up the appearance or refactor the interaction later.

-Dan

Dan Gunter

On Tue, Mar 3, 2015 at 3:51 PM, meghasandesh notifications@github.com wrote:

image https://cloud.githubusercontent.com/assets/1052210/6475287/29c96a24-c1bd-11e4-9e94-e95cba405a11.png

Minor edits made to the interface after consulting with Robin

Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-77065538.

meghasandesh commented 9 years ago

That sounds reasonable. I'd be willing to do it On Mar 4, 2015 7:15 AM, "Dan Gunter" notifications@github.com wrote:

The additional information I am expecting, to do PM mapping, is program name and program version (I am adding 2 fields in the DB for this). This is separate from the data format (csv, xls, etc.). There is also a field "source_type" but that seems to be munging format and program source, so I am going to ignore that and expect new info. Probably the fastest thing would be for me to make whatever UI changes I need to test out the new interface and then leave it to you to clean up the appearance or refactor the interaction later.

-Dan

Dan Gunter

On Tue, Mar 3, 2015 at 3:51 PM, meghasandesh notifications@github.com wrote:

image < https://cloud.githubusercontent.com/assets/1052210/6475287/29c96a24-c1bd-11e4-9e94-e95cba405a11.png

Minor edits made to the interface after consulting with Robin

Reply to this email directly or view it on GitHub <https://github.com/SEED-platform/seed/issues/116#issuecomment-77065538 .

— Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-77175680.

RDmitchell commented 9 years ago

I guess one question is how we will know the program version. I don’t think we can expect the user to know this. It would be nice if somehow that information was included in the file we are uploading, but we would have to talk to each organization responsible for their dataset to see if that is possible.

RDmitchell commented 9 years ago

Megha -- I think the current design is fine. We can implement it when Dan is read to test his PM mapping functionality.

rebrownlbnl commented 9 years ago

Maybe we can tell the PM program version based on the export date? Or is it encoded in the file? Or are there particular field names that are only associated with a specific version?

Rich Brown Building Technology and Urban Systems Department Lawrence Berkeley National Laboratory Sent from my mobile phone

On Mar 4, 2015, at 11:34 AM, RDmitchell notifications@github.com wrote:

I guess one question is how we will know the program version. I don’t think we can expect the user to know this. It would be nice if somehow that information was included in the file we are uploading, but we would have to talk to each organization responsible for their dataset to see if that is possible.

From: meghasandesh [mailto:notifications@github.com notifications@github.com] Sent: Wednesday, March 04, 2015 7:17 AM To: SEED-platform/seed Cc: RDmitchell Subject: Re: [seed] UI for Data Import of PM files (#116)

That sounds reasonable. I'd be willing to do it On Mar 4, 2015 7:15 AM, "Dan Gunter" notifications@github.com wrote:

The additional information I am expecting, to do PM mapping, is program name and program version (I am adding 2 fields in the DB for this). This is separate from the data format (csv, xls, etc.). There is also a field "source_type" but that seems to be munging format and program source, so I am going to ignore that and expect new info. Probably the fastest thing would be for me to make whatever UI changes I need to test out the new interface and then leave it to you to clean up the appearance or refactor the interaction later.

-Dan

Dan Gunter

On Tue, Mar 3, 2015 at 3:51 PM, meghasandesh notifications@github.com wrote:

image <

https://cloud.githubusercontent.com/assets/1052210/6475287/29c96a24-c1bd-11e4-9e94-e95cba405a11.png

Minor edits made to the interface after consulting with Robin

Reply to this email directly or view it on GitHub <https://github.com/SEED-platform/seed/issues/116#issuecomment-77065538 .

— Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-77175680.

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77176060> .Image removed by sender.

— Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-77205084.

RDmitchell commented 9 years ago

I will look into this. I suspect that the API version of the PM file will have some versioning (I would hope), but not sure about the custom report options of getting the data. But there are definitely header lines in that custom report version, so I will figure out if they are useful.

Robin

On Fri, Mar 6, 2015 at 10:24 AM, Rich Brown notifications@github.com wrote:

Maybe we can tell the PM program version based on the export date? Or is it encoded in the file? Or are there particular field names that are only associated with a specific version?

Rich Brown Building Technology and Urban Systems Department Lawrence Berkeley National Laboratory Sent from my mobile phone

On Mar 4, 2015, at 11:34 AM, RDmitchell notifications@github.com wrote:

I guess one question is how we will know the program version. I don’t think we can expect the user to know this. It would be nice if somehow that information was included in the file we are uploading, but we would have to talk to each organization responsible for their dataset to see if that is possible.

From: meghasandesh [mailto:notifications@github.com notifications@github.com] Sent: Wednesday, March 04, 2015 7:17 AM To: SEED-platform/seed Cc: RDmitchell Subject: Re: [seed] UI for Data Import of PM files (#116)

That sounds reasonable. I'd be willing to do it On Mar 4, 2015 7:15 AM, "Dan Gunter" notifications@github.com wrote:

The additional information I am expecting, to do PM mapping, is program name and program version (I am adding 2 fields in the DB for this). This is separate from the data format (csv, xls, etc.). There is also a field "source_type" but that seems to be munging format and program source, so I am going to ignore that and expect new info. Probably the fastest thing would be for me to make whatever UI changes I need to test out the new interface and then leave it to you to clean up the appearance or refactor the interaction later.

-Dan

Dan Gunter

On Tue, Mar 3, 2015 at 3:51 PM, meghasandesh notifications@github.com wrote:

image <

https://cloud.githubusercontent.com/assets/1052210/6475287/29c96a24-c1bd-11e4-9e94-e95cba405a11.png

Minor edits made to the interface after consulting with Robin

Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77065538 .

— Reply to this email directly or view it on GitHub <https://github.com/SEED-platform/seed/issues/116#issuecomment-77175680 .

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77176060> .Image removed by sender.

— Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-77205084.

— Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-77608953.

Robin Mitchell Building Technology and Urban Systems Department Environmental Energy Technologies Division Lawrence Berkeley National Laboratory

meghasandesh commented 9 years ago

Hi Robin,

I am currently working with Dan to ensure that the front end communicates info about the file to the backend. We are solving a few hurdles which will simplify the file upload as well as matching. The new UI should be up soon.

On Fri, Mar 6, 2015 at 10:32 AM, RDmitchell notifications@github.com wrote:

I will look into this. I suspect that the API version of the PM file will have some versioning (I would hope), but not sure about the custom report options of getting the data. But there are definitely header lines in that custom report version, so I will figure out if they are useful.

Robin

On Fri, Mar 6, 2015 at 10:24 AM, Rich Brown notifications@github.com wrote:

Maybe we can tell the PM program version based on the export date? Or is it encoded in the file? Or are there particular field names that are only associated with a specific version?

Rich Brown Building Technology and Urban Systems Department Lawrence Berkeley National Laboratory Sent from my mobile phone

On Mar 4, 2015, at 11:34 AM, RDmitchell notifications@github.com wrote:

I guess one question is how we will know the program version. I don’t think we can expect the user to know this. It would be nice if somehow that information was included in the file we are uploading, but we would have to talk to each organization responsible for their dataset to see if that is possible.

From: meghasandesh [mailto:notifications@github.com notifications@github.com] Sent: Wednesday, March 04, 2015 7:17 AM To: SEED-platform/seed Cc: RDmitchell Subject: Re: [seed] UI for Data Import of PM files (#116)

That sounds reasonable. I'd be willing to do it On Mar 4, 2015 7:15 AM, "Dan Gunter" notifications@github.com wrote:

The additional information I am expecting, to do PM mapping, is program name and program version (I am adding 2 fields in the DB for this). This is separate from the data format (csv, xls, etc.). There is also a field "source_type" but that seems to be munging format and program source, so I am going to ignore that and expect new info. Probably the fastest thing would be for me to make whatever UI changes I need to test out the new interface and then leave it to you to clean up the appearance or refactor the interaction later.

-Dan

Dan Gunter

On Tue, Mar 3, 2015 at 3:51 PM, meghasandesh <notifications@github.com

wrote:

image <

https://cloud.githubusercontent.com/assets/1052210/6475287/29c96a24-c1bd-11e4-9e94-e95cba405a11.png

Minor edits made to the interface after consulting with Robin

Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77065538 .

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77175680 .

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77176060> .Image removed by sender.

— Reply to this email directly or view it on GitHub <https://github.com/SEED-platform/seed/issues/116#issuecomment-77205084 .

— Reply to this email directly or view it on GitHub <https://github.com/SEED-platform/seed/issues/116#issuecomment-77608953 .

Robin Mitchell Building Technology and Urban Systems Department Environmental Energy Technologies Division Lawrence Berkeley National Laboratory

— Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-77610282.

RDmitchell commented 9 years ago

Do you really mean "matching" ? I am thinking you mean "mapping" ? The classic confusion.

On Fri, Mar 6, 2015 at 11:09 AM, meghasandesh notifications@github.com wrote:

Hi Robin,

I am currently working with Dan to ensure that the front end communicates info about the file to the backend. We are solving a few hurdles which will simplify the file upload as well as matching. The new UI should be up soon.

On Fri, Mar 6, 2015 at 10:32 AM, RDmitchell notifications@github.com wrote:

I will look into this. I suspect that the API version of the PM file will have some versioning (I would hope), but not sure about the custom report options of getting the data. But there are definitely header lines in that custom report version, so I will figure out if they are useful.

Robin

On Fri, Mar 6, 2015 at 10:24 AM, Rich Brown notifications@github.com wrote:

Maybe we can tell the PM program version based on the export date? Or is it encoded in the file? Or are there particular field names that are only associated with a specific version?

Rich Brown Building Technology and Urban Systems Department Lawrence Berkeley National Laboratory Sent from my mobile phone

On Mar 4, 2015, at 11:34 AM, RDmitchell notifications@github.com wrote:

I guess one question is how we will know the program version. I don’t think we can expect the user to know this. It would be nice if somehow that information was included in the file we are uploading, but we would have to talk to each organization responsible for their dataset to see if that is possible.

From: meghasandesh [mailto:notifications@github.com notifications@github.com] Sent: Wednesday, March 04, 2015 7:17 AM To: SEED-platform/seed Cc: RDmitchell Subject: Re: [seed] UI for Data Import of PM files (#116)

That sounds reasonable. I'd be willing to do it On Mar 4, 2015 7:15 AM, "Dan Gunter" notifications@github.com wrote:

The additional information I am expecting, to do PM mapping, is program name and program version (I am adding 2 fields in the DB for this). This is separate from the data format (csv, xls, etc.). There is also a field "source_type" but that seems to be munging format and program source, so I am going to ignore that and expect new info. Probably the fastest thing would be for me to make whatever UI changes I need to test out the new interface and then leave it to you to clean up the appearance or refactor the interaction later.

-Dan

Dan Gunter

On Tue, Mar 3, 2015 at 3:51 PM, meghasandesh < notifications@github.com

wrote:

image <

https://cloud.githubusercontent.com/assets/1052210/6475287/29c96a24-c1bd-11e4-9e94-e95cba405a11.png

Minor edits made to the interface after consulting with Robin

Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77065538 .

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77175680 .

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77176060

.Image removed by sender.

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77205084 .

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77608953 .

Robin Mitchell Building Technology and Urban Systems Department Environmental Energy Technologies Division Lawrence Berkeley National Laboratory

— Reply to this email directly or view it on GitHub <https://github.com/SEED-platform/seed/issues/116#issuecomment-77610282 .

— Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-77616860.

Robin Mitchell Building Technology and Urban Systems Department Environmental Energy Technologies Division Lawrence Berkeley National Laboratory

meghasandesh commented 9 years ago

Yes, I meant to say mapping! Freudian slip!

On Fri, Mar 6, 2015 at 11:20 AM, RDmitchell notifications@github.com wrote:

Do you really mean "matching" ? I am thinking you mean "mapping" ? The classic confusion.

On Fri, Mar 6, 2015 at 11:09 AM, meghasandesh notifications@github.com wrote:

Hi Robin,

I am currently working with Dan to ensure that the front end communicates info about the file to the backend. We are solving a few hurdles which will simplify the file upload as well as matching. The new UI should be up soon.

On Fri, Mar 6, 2015 at 10:32 AM, RDmitchell notifications@github.com wrote:

I will look into this. I suspect that the API version of the PM file will have some versioning (I would hope), but not sure about the custom report options of getting the data. But there are definitely header lines in that custom report version, so I will figure out if they are useful.

Robin

On Fri, Mar 6, 2015 at 10:24 AM, Rich Brown notifications@github.com wrote:

Maybe we can tell the PM program version based on the export date? Or is it encoded in the file? Or are there particular field names that are only associated with a specific version?

Rich Brown Building Technology and Urban Systems Department Lawrence Berkeley National Laboratory Sent from my mobile phone

On Mar 4, 2015, at 11:34 AM, RDmitchell notifications@github.com wrote:

I guess one question is how we will know the program version. I don’t think we can expect the user to know this. It would be nice if somehow that information was included in the file we are uploading, but we would have to talk to each organization responsible for their dataset to see if that is possible.

From: meghasandesh [mailto:notifications@github.com notifications@github.com] Sent: Wednesday, March 04, 2015 7:17 AM To: SEED-platform/seed Cc: RDmitchell Subject: Re: [seed] UI for Data Import of PM files (#116)

That sounds reasonable. I'd be willing to do it On Mar 4, 2015 7:15 AM, "Dan Gunter" notifications@github.com wrote:

The additional information I am expecting, to do PM mapping, is program name and program version (I am adding 2 fields in the DB for this). This is separate from the data format (csv, xls, etc.). There is also a field "source_type" but that seems to be munging format and program source, so I am going to ignore that and expect new info. Probably the fastest thing would be for me to make whatever UI changes I need to test out the new interface and then leave it to you to clean up the appearance or refactor the interaction later.

-Dan

Dan Gunter

On Tue, Mar 3, 2015 at 3:51 PM, meghasandesh < notifications@github.com

wrote:

image <

https://cloud.githubusercontent.com/assets/1052210/6475287/29c96a24-c1bd-11e4-9e94-e95cba405a11.png

Minor edits made to the interface after consulting with Robin

Reply to this email directly or view it on GitHub <

https://github.com/SEED-platform/seed/issues/116#issuecomment-77065538

.

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77175680 .

— Reply to this email directly or view it on GitHub <

https://github.com/SEED-platform/seed/issues/116#issuecomment-77176060

.Image removed by sender.

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77205084 .

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77608953 .

Robin Mitchell Building Technology and Urban Systems Department Environmental Energy Technologies Division Lawrence Berkeley National Laboratory

— Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-77610282 .

— Reply to this email directly or view it on GitHub <https://github.com/SEED-platform/seed/issues/116#issuecomment-77616860 .

Robin Mitchell Building Technology and Urban Systems Department Environmental Energy Technologies Division Lawrence Berkeley National Laboratory

— Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-77618807.

RDmitchell commented 9 years ago

I am now testing (on seedtest) the new merged Portfolio Mapping changes, and I see that in the mapping screen there are now 2 rows of headers -- this was when I imported the covered buildings list, so nothing to do with Portfolio Manager files. See image below image

meghasandesh commented 9 years ago

I added the extra headers to help with display of duplicate fields and mapped fields. I'll add in a filter to not have the extra headers when not required On Mar 29, 2015 1:26 PM, "RDmitchell" notifications@github.com wrote:

I am now testing (on seedtest) the new merged Portfolio Mapping changes, and I see that in the mapping screen there are now 2 rows of headers -- this was when I imported the covered buildings list, so nothing to do with Portfolio Manager files. See image below [image: image] https://cloud.githubusercontent.com/assets/6314950/6887532/30965920-d617-11e4-8967-7f5bac7bcdbd.png

— Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-87469341.

RDmitchell commented 9 years ago

After having gone through the mapping stage, including saving the mapping, when I come back to mapping via the Data navigation link, there are several deselected fields. What does that mean?

image

RDmitchell commented 9 years ago

Here is the suggested mapping when I import a PM file. Because of how we still (I think) have matching set up, we need to keep the 4 sacred mappings, at least for now. So the Portfolio Mapping JSON file needs to have these mappings from PM to SEED:

Address 1 == Address Line 1 Property ID == PM Property ID

image

RDmitchell commented 9 years ago

Also, in the screen shot above, very few of the fields are marked as BEDES (ie, the Green Checkmark in the BEDES column). I thought that if the field name came from the new PM BEDES > SEED field mapping list, it would be marked with the BEDES green checkmark. I assume that all these SEED field names have come from that list (although I haven't gone through the list field by field to check), so I would have assumed that when the mapping screen first appears, all the fields would have the green check mark ??

RDmitchell commented 9 years ago

The program is not remembering the changes that I am making to SEED field names for files other than Portfolio Manager (I understand that for now, the program will not remember the PM changes, although in the future we will probably want to make that work).

This is the third time I have imported the sample data file "covered-buildings-sample.csv" to the same user account. Each time I have changed four of the fields, State Of Washington Unique Facilities Identifier (UFI) to State, Premises Street Address to Address Line 1, Property Type to Premises Occupancy Classification, and UBI to Tax Lot ID. image

This is what I change them to

image

If I now save the mapping, then delete the data set and reimport that file, the program doesn't remember the changes I made. We need to keep the feature that the program remembers the file mapping for the non-PM data that is imported.

dangunter commented 9 years ago

Right now, all files are treated as PM files, i.e a static mapping. I thought we agreed to that. For PM files, no attempt is made to use any information from previous mappings.

Dan Gunter

On Sun, Mar 29, 2015 at 2:08 PM, RDmitchell notifications@github.com wrote:

The program is not remembering the changes that I am making to SEED field names for files other than Portfolio Manager (I understand that for now, the program will not remember the PM changes, although in the future we will probably want to make that work).

This is the third time I have imported the sample data file "covered-buildings-sample.csv" to the same user account. Each time I have changed four of the fields, State Of Washington Unique Facilities Identifier (UFI) to State, Premises Street Address to Address Line 1, Property Type to Premises Occupancy Classification, and UBI to Tax Lot ID. [image: image] https://cloud.githubusercontent.com/assets/6314950/6887688/2a44cb2e-d61c-11e4-9cf4-6bb9f4116030.png

This is what I change them to

[image: image] https://cloud.githubusercontent.com/assets/6314950/6887702/737cd912-d61c-11e4-8cc7-548989598efe.png

If I now save the mapping, then delete the data set and reimport that file, the program doesn't remember the changes I made. We need to keep the feature that the program remembers the file mapping for the non-PM data that is imported.

Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-87478002.

rebrownlbnl commented 9 years ago

My understanding was that Megha is adding the pulldown to the UI so the user can designate if the file is type "PM" or not. If type=PM, use the fixed mapping, else use the existing approach of guessing the mapping and using the "remembered" mappings.

I think using the fixed PM mapping for all file types will cause some illogical mappings for non-PM file types, which the user will have to manually override, and it won't remember those overrides. Could get tedious.

Can we implement this conditional approach?

Rich

Rich Brown Building Technology and Urban Systems Division Lawrence Berkeley National Laboratory Office: (510) 486-5896 Mobile: (510) 502-5968 REBrown@lbl.gov

www.lbl.gov/~rebrown

On Sun, Mar 29, 2015 at 9:44 PM, Dan Gunter notifications@github.com wrote:

Right now, all files are treated as PM files, i.e a static mapping. I thought we agreed to that. For PM files, no attempt is made to use any information from previous mappings.

Dan Gunter

On Sun, Mar 29, 2015 at 2:08 PM, RDmitchell notifications@github.com wrote:

The program is not remembering the changes that I am making to SEED field names for files other than Portfolio Manager (I understand that for now, the program will not remember the PM changes, although in the future we will probably want to make that work).

This is the third time I have imported the sample data file "covered-buildings-sample.csv" to the same user account. Each time I have changed four of the fields, State Of Washington Unique Facilities Identifier (UFI) to State, Premises Street Address to Address Line 1, Property Type to Premises Occupancy Classification, and UBI to Tax Lot ID. [image: image] < https://cloud.githubusercontent.com/assets/6314950/6887688/2a44cb2e-d61c-11e4-9cf4-6bb9f4116030.png

This is what I change them to

[image: image] < https://cloud.githubusercontent.com/assets/6314950/6887702/737cd912-d61c-11e4-8cc7-548989598efe.png

If I now save the mapping, then delete the data set and reimport that file, the program doesn't remember the changes I made. We need to keep the feature that the program remembers the file mapping for the non-PM data that is imported.

Reply to this email directly or view it on GitHub <https://github.com/SEED-platform/seed/issues/116#issuecomment-87478002 .

— Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-87543280.

RDmitchell commented 9 years ago

I did not realize that all files would be treated as PM files, and I'm not sure that is really going to work. It basically means that we are taking away functionality that used to be there, for the non-PM files.

What would be the level of effort to treat ONLY the PM files with the static list?

Robin

On Sun, Mar 29, 2015 at 9:44 PM, Dan Gunter notifications@github.com wrote:

Right now, all files are treated as PM files, i.e a static mapping. I thought we agreed to that. For PM files, no attempt is made to use any information from previous mappings.

Dan Gunter

On Sun, Mar 29, 2015 at 2:08 PM, RDmitchell notifications@github.com wrote:

The program is not remembering the changes that I am making to SEED field names for files other than Portfolio Manager (I understand that for now, the program will not remember the PM changes, although in the future we will probably want to make that work).

This is the third time I have imported the sample data file "covered-buildings-sample.csv" to the same user account. Each time I have changed four of the fields, State Of Washington Unique Facilities Identifier (UFI) to State, Premises Street Address to Address Line 1, Property Type to Premises Occupancy Classification, and UBI to Tax Lot ID. [image: image] < https://cloud.githubusercontent.com/assets/6314950/6887688/2a44cb2e-d61c-11e4-9cf4-6bb9f4116030.png

This is what I change them to

[image: image] < https://cloud.githubusercontent.com/assets/6314950/6887702/737cd912-d61c-11e4-8cc7-548989598efe.png

If I now save the mapping, then delete the data set and reimport that file, the program doesn't remember the changes I made. We need to keep the feature that the program remembers the file mapping for the non-PM data that is imported.

Reply to this email directly or view it on GitHub <https://github.com/SEED-platform/seed/issues/116#issuecomment-87478002 .

— Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-87543280.

Robin Mitchell Building Technology and Urban Systems Department Environmental Energy Technologies Division Lawrence Berkeley National Laboratory

RDmitchell commented 9 years ago

Yes, per my previous comment, which I saw before seeing Rich's, I agree with Rich.

We need to distinguish between PM files and other files, and we should not treat them the same.

I thought we had this conversation at one point, that the non-PM files would still use the fuzzy mapping logic to make the suggestion about what the SEED field would be.

And remembering the new field names if a user changes them is actually quite important. Some of the non-PM fields can have lots of fields themselves, and if the user has to re-map every time they load one of those files, which could be multiple times a day, week, etc, will be problematic. Again, it is taking away useful functionality that was there before we made this change.

Robin

dangunter commented 9 years ago

Fine. For the record, we did have a conversation, Robin, in which you specifically said to always use the static mapping. You may have been present, Rich, I don't remember. But I can change it back pretty easily. To be clear, PM files will not remember any changes made by the user.

I'll add an issue so I can remember what needs to be done here.

Dan Gunter

On Sun, Mar 29, 2015 at 11:03 PM, RDmitchell notifications@github.com wrote:

Yes, per my previous comment, which I saw before seeing Rich's, I agree with Rich.

We need to distinguish between PM files and other files, and we should not treat them the same.

I thought we had this conversation at one point, that the non-PM files would still use the fuzzy mapping logic to make the suggestion about what the SEED field would be.

And remembering the new field names if a user changes them is actually quite important. Some of the non-PM fields can have lots of fields themselves, and if the user has to re-map every time they load one of those files, which could be multiple times a day, week, etc, will be problematic. Again, it is taking away useful functionality that was there before we made this change.

Robin

On Sun, Mar 29, 2015 at 10:11 PM, Rich Brown notifications@github.com wrote:

My understanding was that Megha is adding the pulldown to the UI so the user can designate if the file is type "PM" or not. If type=PM, use the fixed mapping, else use the existing approach of guessing the mapping and using the "remembered" mappings.

I think using the fixed PM mapping for all file types will cause some illogical mappings for non-PM file types, which the user will have to manually override, and it won't remember those overrides. Could get tedious.

Can we implement this conditional approach?

Rich

Rich Brown Building Technology and Urban Systems Division Lawrence Berkeley National Laboratory Office: (510) 486-5896 Mobile: (510) 502-5968 REBrown@lbl.gov

www.lbl.gov/~rebrown

On Sun, Mar 29, 2015 at 9:44 PM, Dan Gunter notifications@github.com wrote:

Right now, all files are treated as PM files, i.e a static mapping. I thought we agreed to that. For PM files, no attempt is made to use any information from previous mappings.

Dan Gunter

On Sun, Mar 29, 2015 at 2:08 PM, RDmitchell notifications@github.com wrote:

The program is not remembering the changes that I am making to SEED field names for files other than Portfolio Manager (I understand that for now, the program will not remember the PM changes, although in the future we will probably want to make that work).

This is the third time I have imported the sample data file "covered-buildings-sample.csv" to the same user account. Each time I have changed four of the fields, State Of Washington Unique Facilities Identifier (UFI) to State, Premises Street Address to Address Line 1, Property Type to Premises Occupancy Classification, and UBI to Tax Lot ID. [image: image] <

https://cloud.githubusercontent.com/assets/6314950/6887688/2a44cb2e-d61c-11e4-9cf4-6bb9f4116030.png

This is what I change them to

[image: image] <

https://cloud.githubusercontent.com/assets/6314950/6887702/737cd912-d61c-11e4-8cc7-548989598efe.png

If I now save the mapping, then delete the data set and reimport that file, the program doesn't remember the changes I made. We need to keep the feature that the program remembers the file mapping for the non-PM data that is imported.

Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-87478002 .

Reply to this email directly or view it on GitHub < https://github.com/SEED-platform/seed/issues/116#issuecomment-87543280 .

Reply to this email directly or view it on GitHub <https://github.com/SEED-platform/seed/issues/116#issuecomment-87549242 .

Robin Mitchell Building Technology and Urban Systems Department Environmental Energy Technologies Division Lawrence Berkeley National Laboratory

Reply to this email directly or view it on GitHub https://github.com/SEED-platform/seed/issues/116#issuecomment-87560661.

RDmitchell commented 9 years ago

Having the static list for PM files, and not remembering changes made by the user, is fine for now. Not using the static file, but using the old matching method for non PM files, and remembering changes would be the desired behavior for any non-PM file.

And in the PM static list, please changes the mapping of Address 1 from Premises Street Address to Address Line 1.

Thanks

Robin

RDmitchell commented 9 years ago

I added the previous request, ie, getting Address Line 1 and PM Property ID into the PM static list, to Dan's original issue # 104, which I think is where it really belongs, rather than in this issue.

RDmitchell commented 9 years ago

When I define three different fields with the same field name, the program highlights them in red, but then it allows me to click Map Your Data. The Map Your Data button should not be "live" if there are duplicate field definitions.

image

Then, when I look at the mapping view list, the program selects the last field that had the duplicate field name, so in this example it has set Release Date to City.

image

Then when I go back to the Mapping Screen (after having saved the mapping, so clicking on the Mapping button under the dataset name), I see that there are several fields that are shown as NOT mapped (such as Release Date), even though they are mapped, and the City fieldname in the original dataset is supposedly mapped to Premises City in the SEED db, but when I look at the building list, there is no Premises City, even if I look at List Settings.

image

Looking at the List Settings tab in the Building List, there is no Premises City field.

image

And looking at the Building List detail, there is no Premises City field, and City is mapped to Release Date, even though the Mapping screen does not show that.

image

RDmitchell commented 9 years ago

I am assigning this to myself for testing reminders when Megha has fixed a few of the mapping bugs.

RDmitchell commented 9 years ago

am closing this bug. There are still a few outstanding issues but the functionality is good enough for now and I will write up separate issues for any outstanding work on this.